DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-12 and 17-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Weimer (20120038903) in view of Takahashi (20190377073) and Kamilov (20170200273).
Referring to claims 1, Weimer shows a sensor system (see figure 3A) comprising:
at least one time-of-flight (ToF) sensor configured to receive light from a scene (see figure 3A note Ref 324 also see paragraph 16);
at least one light source configured to emit a structured light pattern (see figure 3A Ref 304 and 316,330 also see the light pattern projected onto the scene Ref 210 and 212) ;
and a controller that carries out operations see figure 3A Ref 308), the operations comprising:
dynamically adjusting the structured light pattern based on one or more highly reflective regions in the scene (see paragraph 41 note the lesser number of beams that are emitted to a particular reflective surface); causing the at least one light source to illuminate at least a portion of the scene with the structured light pattern (see paragraph 41 also see figures 3A-3F); and causing the at least one ToF sensor to provide time of flight information indicative of a depth map of the scene based on the structured light pattern (see paragraph 16 note the LIDAR device generates a map or survey of ground features).
However Weimer fails to show the highly reflective surface is a retroreflector. Takahashi shows a similar device that includes saturation mitigation that specifically shows that a highly reflective surface can include a retroreflector (see paragraph 35). It would have been obvious to have a highly reflective surface include a retroreflector because this is well known to cause saturation as taught by Takahashi. However neither Weimer nor Takahashi shows causing the imaging sensor to provide information indicative of an image of the scene; and
determining a high-resolution depth map of the scene based on the depth map of the scene and the image of the scene, wherein the high-resolution depth map of the scene includes depth information with a higher spatial resolution than that of the depth map of the scene.
Kamilov shows a similar device that includes causing the imaging sensor to provide information indicative of an image of the scene (see figure 3 Ref 320 and 325); and
determining a high-resolution depth map of the scene based on the depth map of the scene and the image of the scene, wherein the high-resolution depth map of the scene includes depth information with a higher spatial resolution than that of the depth map of the scene (see the calibration and fusion as shown in figure 3 Ref 330 also see the comparison of the resolution see paragraph 52-56). It would have been obvious to include the image sensor and difference in resolution of the high-resolution depth map and the standard depth map because this allows the output of relatively low and inexpensive sensors to have high resolution outputs that are delivered to a display through the sensor fusion as taught by Kamilov.
Referring to claim 2, Weimer shows dynamically adjusting the structured light pattern comprises lowering illumination levels for portions of the scene where the one or more retroreflector regions are present (see paragraph 41).
Referring to claim 3, Weimer shows the at least one ToF sensor comprises a plurality of complementary metal-oxide semiconductor (CMOS) or charge-coupled device (CCD) photosensitive elements (see paragraph 29).
Referring to claim 4, Weimer shows the structured light pattern comprises at least one of: a predetermined spatial distribution of light, a predetermined temporal distribution of light, or a predetermined spectral distribution of light (see paragraph 36).
Referring to claim 5, Weimer shows the structured light pattern comprises at least one of: a predetermined light pulse repetition rate, a predetermined light pulse duration, a predetermined light pulse intensity, or a predetermined light pulse duty cycle (see paragraph 9 note the alteration of the illuminating light can include controlling the relative intensity of the beams).
Referring to claim 6, Weimer shows the at least one light source comprises at least one of: a laser diode, a light-emitting diode, a plasma light source, a strobe light, a solid-state laser, or a fiber laser (see paragraph 31).
Referring to claims 7 and 19, Weimer shows dynamically adjusting the structured light pattern comprises selecting a desired structured light pattern from among a plurality of possible structured light patterns, wherein causing the at least one light source to illuminate at least a portion of the scene with the structured light pattern comprises illuminating the portion of the scene according to the desired structured light pattern (see figures 2A-2J note the different patterns that can be selected for different imaged scenes, also note paragraphs 17-21).
Referring to claim 8, Weimer shows an imaging sensor, wherein the imaging sensor comprises a plurality of photosensitive elements, wherein the plurality of photosensitive elements comprises at least one million photosensitive elements, wherein the operations further comprise causing the imaging sensor to provide information indicative of an image of the scene based on the structured light pattern (see figure 3C note the camera 340 also see paragraph 37).
Referring to claim 9, Weimer shows the operations further comprise determining a high- resolution depth map of the scene based on the depth map of the scene and the image of the scene (see paragraph 3 also see the images used in figure 3C camera 340 also see figure 3E Ref 356, see paragraph 37).
Referring to claim 10, Weimer shows at least one ToF sensor, the imaging sensor, and the at least one light source are coupled to a common substrate (see figure 3A-3F and figure 1C Ref 104).
Referring to claims 11 and 17, Weimer shows operations further comprise determining at least one inference about the scene based on the depth map of the scene (see paragraph 18 note the determination of a landing zone based on the depth map of the scene).
Referring to claims 12 and 18, Weimer shows the at least one inference comprises information about objects in an environment of a vehicle or an operating context of the vehicle (see paragraph 18 also see paragraph 20).
Referring to claim 20, Weimer shows dynamically adjusting the structured light pattern further comprises adjusting the structured light pattern based on an amount of ambient light or a time of day (see paragraph 16).
Claim(s) 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Weimer (20120038903) in view of Takahashi (20190377073), Kamilov (20170200273), and Nehmadi (20180232947).
Referring to claim 13, Weimer fails to show but Nehmadi shows the controller comprises at least one deep neural network, wherein the determining the at least one inference is performed by the at least one deep neural network (see paragraph 53). It would have been obvious to include the DNN as taught by Nehmadi because this allows for fast and accurate execution of object detection, classification and semantic segmentation (see paragraph 53).
Claim(s) 14-16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Weimer (20120038903) in view of Takahashi (20190377073), Kamilov (20170200273), and Stettner (20150258990).
Referring to claim 14, Weimer shows a system comprising:
a sensor system configured to be coupled to a vehicle (see figures 1A-1C), wherein each sensor system comprises: at least one time-of-flight (ToF) sensor (see paragraph 16;
at least one imaging sensor, wherein the at least one ToF sensor and the at least one imaging sensor are configured to receive light from a scene (see figure 3C Ref 340 also see figure 3E note Ref 340 and 356);
at least one light source configured to emit a structured light pattern (see figures 2A-2J);
and a controller that carries out operations, the operations comprising: dynamically adjusting the structured light pattern based on one or more highly reflective regions in the scene (see paragraph 41 note the lesser number of beams that are emitted to a particular reflective surface);
causing the at least one light source to illuminate at least a portion of the scene with the structured light pattern (see paragraph 41 also see figures 3A-3F);
causing the at least one ToF sensor to provide time of flight information indicative of a depth map of the scene based on the structured light pattern (see paragraph 16 note the LIDAR device generates a map or survey of ground features);
and causing the imaging sensor to provide information indicative of an image of the scene based on the structured light pattern (see paragraph 16 note the LIDAR device generates a map or survey of ground features).
However Weimer fails to show the highly reflective surface is a retroreflector. Takahashi shows a similar device that includes saturation mitigation that specifically shows that a highly reflective surface can include a retroreflector (see paragraph 35). It would have been obvious to have a highly reflective surface include a retroreflector because this is well known to cause saturation as taught by Takahashi.
However Weimer fails to show multiple sensor systems configured to be coupled to a vehicle. Stettner shows a similar device that includes multiple sensor systems configured to be coupled to a vehicle (see figures 2-3 also see each sensor system LRU and SRU are shown in detail in figure 5). It would have been obvious to include multiple sensor systems coupled to a vehicle because this allows for complete coverage of the environment surrounding the vehicle.
Referring to claim 15, Weimer shows the operations further comprise determining a high- resolution depth map of the scene based on the depth map of the scene and the image of the scene (see see paragraph 3 also see the images used in figure 3C camera 340 also see figure 3E Ref 356, see paragraph 37).
Referring to claim 16, Weimer shows at least one of the sensor systems comprises at least one ToF sensor and at least one imaging sensor in a common housing (see figure 1A Ref 104 also see figure 1C Ref 104).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LUKE D RATCLIFFE whose telephone number is (571)272-3110. The examiner can normally be reached M-F 9:00AM-5:00PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Isam Alsomiri can be reached on 571-272-6970. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/LUKE D RATCLIFFE/Primary Examiner, Art Unit 3645
Prosecution Insights