Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 5-8 and 12 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Kuoch (US Patent Pub. No.: US 2021/0394676 A1).
Regarding claim 1, Kuoch teaches a method for detecting surroundings (FIG. 1 a schematic illustration of a communication system with two mobile communication devices, each associated with a corresponding vehicle for providing visual information about at least part of an environment, a view onto which is at least in part concealed by one of the vehicles, according to an embodiment of the invention. [0043].
PNG
media_image1.png
622
1080
media_image1.png
Greyscale
), wherein a first image data stream (Moreover, it is preferred, that the at least one image of the environment including at least part of the first vehicle is provided in form of a second video stream by means of the at least one sensor device. [0023]) is generated by a first camera of an ego vehicle (FIG. 1 shows a schematic illustration of a communication system 1 comprising in this example two mobile communication devices, namely a first mobile communication device 2 and a second mobile communication device 3, each associated with a respective vehicle, namely a first vehicle 4 and a second vehicle 5 (which reads on “an ego vehicle”), wherein the first vehicle 4 is travelling ahead of the second vehicle 5. Each of the mobile communication devices 2, 3 comprises a processing unit 6, 7, at least one sensor device, which is configured as a camera 8, 9 and a communication unit 10, 11. [0050]), the method comprising: receiving a second image data stream (Preferably, the visual information is provided in form of at least part of a first video stream captured by a first device, which is associated with the first vehicle, especially a first mobile communication device within the first vehicle. [0020]), which is generated by a second camera of a further vehicle (FIG. 1 shows a schematic illustration of a communication system 1 comprising in this example two mobile communication devices, namely a first mobile communication device 2 and a second mobile communication device 3, each associated with a respective vehicle, namely a first vehicle 4 (which reads on “a further vehicle”) and a second vehicle 5, wherein the first vehicle 4 is travelling ahead of the second vehicle 5. Each of the mobile communication devices 2, 3 comprises a processing unit 6, 7, at least one sensor device, which is configured as a camera 8, 9 and a communication unit 10, 11. [0050]), wherein a first visual field represented by the first image data stream overlaps with a second visual field represented by the second image data stream (Preferably, the mobile communication devices 2, 3, which are preferably configured as smartphones but can also be configured as a tablet PC, a laptop, a notebook, and so on, are positioned behind the windshield of the associated vehicles 4, 5, such that the respective fields of view FOV1 (which reads on “a second visual field represented by the second image data stream”), FOV2 (which reads on “a first visual field represented by the first image data stream”) of the cameras 8, 9 of the respective mobile communication devices 2, 3 cover at least part 16a, 16b of the environment 16 in the front of the respective vehicles 4, 5. [0051]); on the basis of the first image data stream (First of all, the camera 8 of the second mobile communication device 3 continuously captures images of the part 16b of the environment 16 within its field of view FOV2. [0053]) and the second image data stream (After that, the second mobile communication device 3 sends a request by means of its communication unit 10 to the cloud server 12 for requesting the visual information, which is concealed by the first vehicle 4. [0053]), identifying a region which is obscured for the first camera in the first visual field (First of all, the camera 8 of the second mobile communication device 3 continuously captures images of the part 16b of the environment 16 within its field of view FOV2. The captured images can be analyzed and based on such an analysis vehicles like the second vehicle 4 can be detected within those captured images. [0053]. In this example, the view of the camera 8 of the second mobile communication device 3 onto at least part of the environment 16 is concealed by the first vehicle 4 travelling ahead. [0052]), wherein the region is not obscured for the second camera (In this example, the view of the camera 8 of the second mobile communication device 3 onto at least part of the environment 16 is concealed by the first vehicle 4 travelling ahead. [0052]. Since the obscuring object is the vehicle 4 itself, the region ahead of the vehicle 4 is not obscured for the camera of the vehicle 4.); on the basis of the second image data stream (After that, images captured by the camera 9 of the first mobile communication device 2 are streamed in form an environmental live video via the cloud server 12 to the second mobile communication device 3. [0053]), generating substitute image data which corresponds to the region that is obscured for the first camera (The first image 29, namely the image of the received first video stream is then overlaid over the second image 25a within the defined area 30. [0064].
PNG
media_image2.png
512
906
media_image2.png
Greyscale
); and on the basis of the first image data stream (FIG. 6 shows on the right hand side an image 25a provided by the built in camera video stream module 18 captured by the camera 8 of the second mobile communication device 3 and the received visual information in form of an image 29 of the streamed first video provided by the camera 9 of the first mobile communication device 2 and on the left hand side the composed image 31. [0064]), generating a combined image data stream which represents the first visual field (The received visual information in form of the first video stream provided by the first mobile communication device 2 is now embedded into this second video stream image by image by means of the second mobile communication device 3 and displayed on the display device 14. [0053]), wherein the substitute image data is displayed (The finally composed image 31 is then displayed on the display device 14 of the second mobile communication device 3. [0064]) in a region of the combined image data stream (The first image 29, namely the image of the received first video stream is then overlaid over the second image 25a within the defined area 30. [0064]), which corresponds to the region obscured for the first camera (This area 30 constitutes the part of the second image 25a, which represents the first vehicle 4. [0064]).
Regarding claim 5, Kuoch teaches the method as claimed in claim 1, wherein the substitute image data is superimposed on the first image data stream (The first image 29, namely the image of the received first video stream is then overlaid over the second image 25a within the defined area 30. [0064]) in order to generate the combined image data stream (The received visual information in form of the first video stream provided by the first mobile communication device 2 is now embedded into this second video stream image by image by means of the second mobile communication device 3 and displayed on the display device 14. [0053]).
Regarding claim 6, Kuoch teaches the method as claimed in claim 5, wherein the substitute image data is superimposed in a partially transparent manner on original image data of the first image data stream which corresponds to the region obscured for the first camera in the first image data stream (For example it would be possible that the displaying of the visual information is modified by increasing a degree of transparency, according to which the visual information is displayed as an overlay over the at least one image of the environment. Therefore parts of the at least one image behind that overlay image are visible, and also such a detected object, in case it would be behind that overlay, can therefore be seen then. [0010]).
Regarding claim 7, Kuoch teaches the method as claimed in claim 1, wherein the further vehicle (FIG. 1 vehicle 4
PNG
media_image3.png
622
1080
media_image3.png
Greyscale
) is located in the first visual field (FIG. 1 FOV2) and the region obscured for the first camera is obscured by the further vehicle for the first camera (In this example, the view of the camera 8 of the second mobile communication device 3 onto at least part of the environment 16 is concealed by the first vehicle 4 travelling ahead. [0052]).
Regarding claim 8, Kuoch teaches the method as claimed in claim 1, wherein the region obscured for the first camera is identified (Based on the captured images of this second video stream the vehicle detection and selection module 20 vehicles are detected and one of them is selected. [0062]) by at least one vehicle computing unit of the ego vehicle (To perform this see through function an application running on the processing unit 6, 7 of the first and second mobile communication devices 2, 3 can cause the corresponding processing units 6, 7 to execute a method as described in the following. [0052]); and/or the substitute image data is determined (In case several vehicles are detected, then the most central on the captured images is selected, in this case the first vehicle 4, for which the visual see through information shall be requested. [0053]) by the at least one vehicle computing unit of the ego vehicle (To perform this see through function an application running on the processing unit 6, 7 of the first and second mobile communication devices 2, 3 can cause the corresponding processing units 6, 7 to execute a method as described in the following. [0052]); and/or the combined image data stream is generated (To provide this composed image 31 the video composition and rendering module 23 first determines a certain area in the image 25a captured by the second mobile communication device 3 as part of the second video stream, which in the following is therefore called second image 25a. This area 30 constitutes the part of the second image 25a, which represents the first vehicle 4. The first image 29, namely the image of the received first video stream is then overlaid over the second image 25a within the defined area 30. [0064]) by the at least one vehicle computing unit of the ego vehicle (To perform this see through function an application running on the processing unit 6, 7 of the first and second mobile communication devices 2, 3 can cause the corresponding processing units 6, 7 to execute a method as described in the following. [0052]).
Regarding claim 12, Kuoch in the combination teaches a surroundings detection system (FIG. 1 a schematic illustration of a communication system with two mobile communication devices, each associated with a corresponding vehicle for providing visual information about at least part of an environment, a view onto which is at least in part concealed by one of the vehicles, according to an embodiment of the invention. [0043].
PNG
media_image1.png
622
1080
media_image1.png
Greyscale
), comprising: a first camera for an ego vehicle (FIG. 1 shows a schematic illustration of a communication system 1 comprising in this example two mobile communication devices, namely a first mobile communication device 2 and a second mobile communication device 3, each associated with a respective vehicle, namely a first vehicle 4 and a second vehicle 5 (which reads on “an ego vehicle”), wherein the first vehicle 4 is travelling ahead of the second vehicle 5. Each of the mobile communication devices 2, 3 comprises a processing unit 6, 7, at least one sensor device, which is configured as a camera 8, 9 and a communication unit 10, 11. [0050]), which is configured to generate a first image data stream (Moreover, it is preferred, that the at least one image of the environment including at least part of the first vehicle is provided in form of a second video stream by means of the at least one sensor device. [0023]); at least one communication interface for the ego vehicle for wireless data transmission (It is an object of the present invention to provide a method for providing visual information about at least part of an environment, the view onto which is at least in part concealed by a first vehicle travelling ahead, a computer program product, a mobile communication device and a communication system, which allow for see-through functions, which can enhance the safety even more. [0004]); and at least one computing unit (It is an object of the present invention to provide a method for providing visual information about at least part of an environment, the view onto which is at least in part concealed by a first vehicle travelling ahead, a computer program product, a mobile communication device and a communication system, which allow for see-through functions, which can enhance the safety even more. [0004]), wherein the at least one computing unit is configured to: receive a second image data stream (Preferably, the visual information is provided in form of at least part of a first video stream captured by a first device, which is associated with the first vehicle, especially a first mobile communication device within the first vehicle. [0020]) which is generated by means of a second camera of a further vehicle (FIG. 1 shows a schematic illustration of a communication system 1 comprising in this example two mobile communication devices, namely a first mobile communication device 2 and a second mobile communication device 3, each associated with a respective vehicle, namely a first vehicle 4 (which reads on “a further vehicle”) and a second vehicle 5, wherein the first vehicle 4 is travelling ahead of the second vehicle 5. Each of the mobile communication devices 2, 3 comprises a processing unit 6, 7, at least one sensor device, which is configured as a camera 8, 9 and a communication unit 10, 11. [0050]) via the at least one communication interface (It is an object of the present invention to provide a method for providing visual information about at least part of an environment, the view onto which is at least in part concealed by a first vehicle travelling ahead, a computer program product, a mobile communication device and a communication system, which allow for see-through functions, which can enhance the safety even more. [0004]), wherein a first visual field represented by the first image data stream overlaps with a second visual field represented by the second image data stream (Preferably, the mobile communication devices 2, 3, which are preferably configured as smartphones but can also be configured as a tablet PC, a laptop, a notebook, and so on, are positioned behind the windshield of the associated vehicles 4, 5, such that the respective fields of view FOV1 (which reads on “a second visual field represented by the second image data stream”), FOV2 (which reads on “a first visual field represented by the first image data stream”) of the cameras 8, 9 of the respective mobile communication devices 2, 3 cover at least part 16a, 16b of the environment 16 in the front of the respective vehicles 4, 5. [0051]); on the basis of the first image data stream (First of all, the camera 8 of the second mobile communication device 3 continuously captures images of the part 16b of the environment 16 within its field of view FOV2. [0053]) and the second image data stream (After that, the second mobile communication device 3 sends a request by means of its communication unit 10 to the cloud server 12 for requesting the visual information, which is concealed by the first vehicle 4. [0053]), identify a region which is obscured for the first camera in the first visual field (First of all, the camera 8 of the second mobile communication device 3 continuously captures images of the part 16b of the environment 16 within its field of view FOV2. The captured images can be analyzed and based on such an analysis vehicles like the second vehicle 4 can be detected within those captured images. [0053]. In this example, the view of the camera 8 of the second mobile communication device 3 onto at least part of the environment 16 is concealed by the first vehicle 4 travelling ahead. [0052]), which region is not obscured for the second camera (In this example, the view of the camera 8 of the second mobile communication device 3 onto at least part of the environment 16 is concealed by the first vehicle 4 travelling ahead. [0052]. Since the obscuring object is the vehicle 4 itself, the region ahead of the vehicle 4 is not obscured for the camera of the vehicle 4.); on the basis of the second image data stream (After that, images captured by the camera 9 of the first mobile communication device 2 are streamed in form an environmental live video via the cloud server 12 to the second mobile communication device 3. [0053]), generate substitute image data which corresponds to the region that is obscured for the first camera (The first image 29, namely the image of the received first video stream is then overlaid over the second image 25a within the defined area 30. [0064].
PNG
media_image2.png
512
906
media_image2.png
Greyscale
); and on the basis of the first image data stream (FIG. 6 shows on the right hand side an image 25a provided by the built in camera video stream module 18 captured by the camera 8 of the second mobile communication device 3 and the received visual information in form of an image 29 of the streamed first video provided by the camera 9 of the first mobile communication device 2 and on the left hand side the composed image 31. [0064]), generate a combined image data stream which represents the first visual field (The received visual information in form of the first video stream provided by the first mobile communication device 2 is now embedded into this second video stream image by image by means of the second mobile communication device 3 and displayed on the display device 14. [0053]), and at the same time to display the substitute image data (The finally composed image 31 is then displayed on the display device 14 of the second mobile communication device 3. [0064]) in a region of the combined image data stream (The first image 29, namely the image of the received first video stream is then overlaid over the second image 25a within the defined area 30. [0064]), which corresponds to the region obscured for the first camera (This area 30 constitutes the part of the second image 25a, which represents the first vehicle 4. [0064]).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 9-11 and 13-14 are rejected under 35 U.S.C. 103 as being unpatentable over Kuoch (US Patent Pub. No.: US 2021/0394676 A1) hereinafter Kuoch, in view of Du (US Patent Pub. No.: US 2019/0052842 A1), hereinafter Du.
Regarding claim 9, Kuoch teaches the method as claimed in claim 1, wherein the combined image data stream is displayed (Moreover, at least one image of the environment including at least part of the vehicle is captured and the at least one image of the environment and the received visual information embedded in the at least one image is displayed on the display device. [0001]).
Kuoch does not teach the following limitations as further recited, but Du further teaches on a vehicle-external display device by means of a vehicle-external computer system (One of the networked devices that can communicate with the vehicle 12 via the wireless communication system 28 is the networked wireless device 57. The networked wireless device 57 can include computer processing capability, a transceiver capable of communicating using a short-range wireless protocol, and a visual display 59. [0026].
PNG
media_image4.png
700
528
media_image4.png
Greyscale
).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kuoch to incorporate the teachings of Du to display the combined image data stream on a vehicle-external display device by means of a vehicle-external computer system in order to improve autonomous driving vehicle performance in a heterogeneous capability environment by sharing capabilities among different vehicles.
Regarding claim 10, Kuoch in the combination teaches a method for tele-operated guidance of an ego vehicle, comprising: carrying out a method for detecting surroundings as claimed in claim 9 (Moreover, at least one image of the environment including at least part of the vehicle is captured and the at least one image of the environment and the received visual information embedded in the at least one image is displayed on the display device. [0001]).
Du in the combination further teaches in response to the display of the combined image data stream (The system employed by a first vehicle 510 is operative to use a camera system, sensor system or other detection system to capture images and objects, such as vehicles or other road users, in its proximate area, to receive images and other data from other road uses, and to combine these images and data. The system may be further operative to generate a display of the combined data. [0054]) on the vehicle-external display device (One of the networked devices that can communicate with the vehicle 12 via the wireless communication system 28 is the networked wireless device 57. The networked wireless device 57 can include computer processing capability, a transceiver capable of communicating using a short-range wireless protocol, and a visual display 59. [0026]), capturing a user input (A live expert or advisor, e.g. the advisor 86 illustrated in FIG. 1, can optionally review the object prediction output 39 and provide additional input and/or override automatic driving operations and assume operation of the vehicle if desired or required by a vehicle situation. [0044]) by the vehicle-external computer system (One of the networked devices that can communicate with the vehicle 12 via the wireless communication system 28 is the networked wireless device 57. The networked wireless device 57 can include computer processing capability, a transceiver capable of communicating using a short-range wireless protocol, and a visual display 59. [0026]); and controlling the ego vehicle at least partially automatically depending on the user input (A live expert or advisor, e.g. the advisor 86 illustrated in FIG. 1, can optionally review the object prediction output 39 and provide additional input and/or override automatic driving operations and assume operation of the vehicle if desired or required by a vehicle situation. The interpretation module 48 generates an interpreted output 49 that includes additional input provided by the live expert, if any. The [0044]).
Regarding claim 11, Du in the combination teaches the method as claimed in claim 10, wherein the vehicle-external computer system is used to transmit a control command to the ego vehicle (One of the networked devices that can communicate with the vehicle 12 via the wireless communication system 28 is the networked wireless device 57. The networked wireless device 57 can include computer processing capability, a transceiver capable of communicating using a short-range wireless protocol, and a visual display 59. [0026]), depending on the user input (A live expert or advisor, e.g. the advisor 86 illustrated in FIG. 1, can optionally review the object prediction output 39 and provide additional input and/or override automatic driving operations and assume operation of the vehicle if desired or required by a vehicle situation. [0044]); and the ego vehicle is guided at least partially automatically depending on the control command (A live expert or advisor, e.g. the advisor 86 illustrated in FIG. 1, can optionally review the object prediction output 39 and provide additional input and/or override automatic driving operations and assume operation of the vehicle if desired or required by a vehicle situation. The interpretation module 48 generates an interpreted output 49 that includes additional input provided by the live expert, if any. The [0044]).
Regarding claim 13, Du in the combination teaches the surroundings detection system as claimed in claim 12, wherein the surroundings detection system comprises a vehicle-external display device and the at least one computing unit contains a vehicle-external computer system (One of the networked devices that can communicate with the vehicle 12 via the wireless communication system 28 is the networked wireless device 57. The networked wireless device 57 can include computer processing capability, a transceiver capable of communicating using a short-range wireless protocol, and a visual display 59. [0026].
PNG
media_image4.png
700
528
media_image4.png
Greyscale
), and the vehicle-external computer system is configured to generate the combined image data stream (The system employed by a first vehicle 510 is operative to use a camera system, sensor system or other detection system to capture images and objects, such as vehicles or other road users, in its proximate area, to receive images and other data from other road uses, and to combine these images and data. The system may be further operative to generate a display of the combined data. [0054]) and to display it on the vehicle-external display device (One of the networked devices that can communicate with the vehicle 12 via the wireless communication system 28 is the networked wireless device 57. The networked wireless device 57 can include computer processing capability, a transceiver capable of communicating using a short-range wireless protocol, and a visual display 59 (which reads on “the vehicle-external display device”). [0026]); or the at least one computing unit contains at least one vehicle computing unit, which is configured to receive the second image data stream via the at least one communication interface, to identify the region obscured for the first camera, to generate the substitute image data, to generate the combined image data stream and to transmit the combined image data stream via the at least one communication interface to the vehicle-external computer system and the vehicle-external computer system is configured to display the combined image data stream on the vehicle- external display device (The claim language is interpreted as disjunctive.).
Regarding claim 14, Du in the combination teaches a vehicle guidance system for tele-operated guidance of an ego vehicle, wherein the vehicle guidance system has a surroundings detection system as claimed in claim 13 (The system employed by a first vehicle 510 is operative to use a camera system, sensor system or other detection system to capture images and objects, such as vehicles or other road users, in its proximate area, to receive images and other data from other road uses, and to combine these images and data. The system may be further operative to generate a display of the combined data. [0054]); the vehicle-external computer system is configured to capture user input (A live expert or advisor, e.g. the advisor 86 illustrated in FIG. 1, can optionally review the object prediction output 39 and provide additional input and/or override automatic driving operations and assume operation of the vehicle if desired or required by a vehicle situation. [0044]) in response to the display of the combined image data stream (The system employed by a first vehicle 510 is operative to use a camera system, sensor system or other detection system to capture images and objects, such as vehicles or other road users, in its proximate area, to receive images and other data from other road uses, and to combine these images and data. The system may be further operative to generate a display of the combined data. [0054]) on the vehicle-external display device (One of the networked devices that can communicate with the vehicle 12 via the wireless communication system 28 is the networked wireless device 57. The networked wireless device 57 can include computer processing capability, a transceiver capable of communicating using a short-range wireless protocol, and a visual display 59 (which reads on “the vehicle-external display device”). [0026]); and the vehicle guidance system has an ego vehicle guidance system for the ego vehicle, which is configured to guide the ego vehicle at least partially automatically depending on the user input (A live expert or advisor, e.g. the advisor 86 illustrated in FIG. 1, can optionally review the object prediction output 39 and provide additional input and/or override automatic driving operations and assume operation of the vehicle if desired or required by a vehicle situation. The interpretation module 48 generates an interpreted output 49 that includes additional input provided by the live expert, if any. [0044]).
Claims 2-4 are rejected under 35 U.S.C. 103 as being unpatentable over Kuoch (US Patent Pub. No.: US 2021/0394676 A1) hereinafter Kuoch, in view of Rameau (A Real-time Augmented Reality System to See-Through Cars, IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS VOL. 22, NO. 11, NOVEMBER 2016), hereinafter Rameau.
Regarding claim 2, Kuoch teaches all of the elements of the claimed invention as stated in claim 1 except for the following limitations as further recited. However, Rameau teaches wherein a deviation of one or more extrinsic camera parameters and/or one or more intrinsic camera parameters between the first camera and the second camera (We also assume a fully calibrated vision system on both cars, where the 3×3 intrinsic camera matrices of the front cameras L (left) and R (right) are respectively noted KL and KR; while KB stands for the rear car intrinsic matrix. Page 2396 right column first paragraph) is at least partially compensated by transforming the second image data stream according to a transformation parameter set (
PNG
media_image5.png
218
812
media_image5.png
Greyscale
. Page 2398 left column 1st paragraph); and the region obscured for the first camera is identified on the basis of the first image data stream (The ROI is determined using the extremal positions of these reprojected points on the image. Therefore, the synthetic patch is cropped appropriately and stitched on the current rear car image. Page 2398 right column 4th paragraph) and the transformed second image data stream (Therefore, all the pixels in the front car images can be reprojected to form a synthetic image of the occluded region. Page 2396 right column 3rd paragraph).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Kuoch to incorporate the teachings of Rameau to partially compensate a deviation of the extrinsic or intrinsic camera parameters between the first camera and the second camera by transforming the second image data stream according to a transformation parameter set and identify the region obscured for the first camera on the basis of the first image data stream and the transformed second image data stream in order to create a seamless see-through effect which preserves the structure of the scene.
Regarding claim 3, Rameau in the combination teaches the method as claimed in claim 2, wherein the combined image data stream is generated on the basis of the first image data stream and the transformed second image data stream (To tackle this problem in a more effective way, we propose to consider the front car as a cuboid where we assume that the position of the left camera is roughly known (see Fig. 4(a)). Then, the 8 corner points of this cuboid PCi can be reprojected on the rear car image like in Fig. 4(b) (in practice a box slightly bigger than the car dimensions is preferable to ensure a good coverage). The ROI is determined using the extremal positions of these reprojected points on the image. Therefore, the synthetic patch is cropped appropriately and stitched on the current rear car image. Page 2398 right column 4th paragraph.
PNG
media_image6.png
466
828
media_image6.png
Greyscale
), wherein the substitute image data is generated on the basis of the transformed second image data stream (Therefore, all the pixels in the front car images can be reprojected to form a synthetic image of the occluded region. Page 2396 right column 3rd paragraph).
Regarding claim 4, Rameau in the combination teaches the method as claimed in claim 2, wherein at least one feature is identified (Note that using relatively large field of view lenses is a good way to extract more common features between the two vehicles and to be more robust to moving objects in the scene. Page 2398 right column last paragraph), which is represented by both the first image data stream and the second image data stream (Note that using relatively large field of view lenses is a good way to extract more common features between the two vehicles and to be more robust to moving objects in the scene. Page 2398 right column last paragraph); and the transformation parameter set is determined depending on a comparison of a representation of the at least one feature in the first image data stream with a representation of the at least one feature in the second image data stream (To summarize: at every new image acquisition, the visual odometry is performed on the front car, thus, the new 3D points and their corresponding ORB features are transferred to the rear vehicle. Page 2399 right column 2nd paragraph).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LEI ZHAO whose telephone number is (703)756-1922. The examiner can normally be reached Monday - Friday 8:00 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, VU LE can be reached at (571)272-7332. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/LEI ZHAO/Examiner, Art Unit 2668
/VU LE/Supervisory Patent Examiner, Art Unit 2668
Prosecution Insights