DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-10 are rejected under 35 U.S.C. 103 as being unpatentable over Goldstein et al. (US 20220057519 A1; hereafter Goldstein) in view of David et al. (US 20050269481 A1; hereafter David).
As of Claim 1: Goldstein teaches a system (100) for object recognition, the system comprising: a red, green, blue (RGB) image sensor (122) configured to generate RGB images of a field of view (FOV) (¶¶0063,0064,0076 and note that apparatus 100 includes an imaging device 104 configured to detect a subject 308 in a subject area. Imaging device 104 may include an optical camera 108. ); a co-located, near infrared (NIR) image sensor (124) configured to capture NIR images of the FOV; a co-located NIR laser (110) configured to emit NIR pulses towards the FOV (¶¶0068,0072 and note that FIG. 1, light radar component 116 may perform ToF calculation, by firing pulses of light and measuring time required for a backscattered and/or reflected pulse to return. Time may be measured using an oscillator-based clock, where a faster clock signal may enable more accurate measure of the time a pulse takes to return to detector. ToF may alternatively or additionally be measured using an amplitude modulated continuous wave (AMCW) technique, whereby light is emitted continuously from light source with a varying amplitude, and a phase of returning detected light is compared to a phase of transmitted light. For instance, light source may cast a modulated illumination in a near-infrared (NIR) or short-wave infrared (SWIR) spectrum onto a scene); and a processor (140) having associated non-transient memory with instructions that when executed by the processor perform a process comprising steps of: a) receiving one or more RGB images from the RGB image sensor (¶¶0075-0076,0101,0192 and note that device 120 may generate images, which may be combined with and/or used to supplement images taken using optical camera 108, infrared camera, 112, light radar component 116, or any combination thereof. Also, FIG. 1, object detection and/or edge detection may alternatively or additionally be performed using light radar data, RF sensor or radar component data, and/or 3D camera, sensor or computational method data. For instance, and without limitation, processor 136 and/or remote device 140 may receive, from light radar component 116 and/or radar component 208, raw modulated images.); b) receiving multiple NIR non-pulse images from the NIR image sensor (¶¶0065,0075,0087 and note that imaging device 104 may include an infrared camera 112); c) receiving multiple NIR pulse-enhanced images, each including reflections of NIR laser pulses from retro-reflectors in the FOV, wherein each NIR pulse-enhanced image is generated from multiple NIR image sensor exposures, wherein the number of NIR image sensor exposures is N, wherein each NIR image sensor exposure is synchronized with a respective laser pulse, (¶¶0087,0164,0194-0199);
d) determining, from the multiple NIR non-pulse images and the multiple NIR pulse-enhanced images, multiple respective NIR pulse-only images, comparing a brightness of the NIR pulse-only images with a preset threshold, and, when the brightness is insufficient, increasing the number N of NIR image sensor exposures and repeating steps a-d (¶¶0070,0071,0099,0630);
e) determining, from the multiple NIR pulse-only images, multiple respective retro-reflector images, each pixel of each retro-reflector image indicating whether a corresponding point in the FOV is part of a retro-reflector (¶¶0167-0169);
f) determining a distance image, each pixel of the distance image indicating a distance range from the NIR image sensor to a point corresponding to the pixel in the FOV;
g) determining, from the multiple NIR retro-reflector images, a velocity image, each pixel of the velocity image indicating a velocity of a retro-reflector at a point corresponding to the pixel in the FOV (¶¶0067,0681and note that an automated threat detection and deterrence apparatus with position dependent deterrence is configured to identify a behavior of an individual as a function of at least a datum regarding the individual, determine at least a deterrent that impacts the behavior, identify at least a spatiotemporal element related to the individual and the at least deterrent, generate a safety modifier as a function of the spatiotemporal element, wherein generating includes identifying a distance parameter as a function of the spatiotemporal element, determining a velocity parameter as a function of the spatiotemporal element, and generating the safety modifier as a function of the distance parameter and velocity parameter, modify the at least a deterrent as a function of the safety modifier, and initiate a modified deterrent.);
h) generating a multi-mode image, wherein each pixel of the multi-mode image has a set of values derived from corresponding pixels in multiple images (¶¶0055,0076,0871 and note that), where the multiple images include at least one of the one or more RGB images (¶¶0064,00,0087 and note that), at least one of the multiple NIR non-pulse images, at least one of the multiple NIR pulse-only images, at least one of the retro-reflector images (¶¶0169,0193 and note that), as well as the distance image, the velocity image, and a map of x, y coordinates of the FOV, wherein each pixel of the multi-mode image corresponds to one of the x, y coordinates (¶¶0087,0121,0452,0464,0666 and note that); and i) applying the multi-mode image to a trained ML model to recognize objects in the multi-mode image (Fig. 4 and ¶¶0106,0109,0118,0121,0134,0146,0169 and note that).
David is a similar or analogous system to the claimed invention as evidenced David teaches a system may be required to operate in an environment saturated with sources of light in the visible and near IR wavelengths (e.g., headlights of other cars, roadside lights, other active night vision systems), while overcoming the challenge of eliminating blinding resulting from such sources of light, and without encountering radiation and other safety problems in influencing the system that would have prompted a predictable variation of Goldstein by applying David’s known principal of wherein each synchronized laser pulse of each NIR image sensor exposure has a duration of 2*(Rmax -Rmin)/C, wherein each NIR image sensor exposure has a duration equal to the laser pulse and offset from the laser pulse by 2*Rmin/C, where C is the speed of light, Rmin is a minimum range for object detection, and Rmax is a maximum range for object detection (¶¶0042,0074-0076,0087,0088,0100,0101,0107,0108 and note that the laser pulse width (T.sub.laser) may be determined in accordance with the depth of the field from which some minimum level of reflections is required (R.sub.o-R.sub.min) divided by the speed of light in the relevant medium (C) and multiplied by two. R.sub.o is the range from which, for the first time, reflections arrived at the system's image intensifier while it is at an "ON" position, wherein those reflections are the end result of the whole span of the pulse width passing in its entirety, over the target located at this R.sub.o range. Up to R.sub.min range, reflections emitting from targets within this range will encounter an "OF" image intensifier.)
In view of the motivations such as a light source providing non-visible light pulses and a camera having an image intensifier enabled to gate selected received images thereby further driver's visibility range and one of ordinary skill in the art would have implemented the claimed variation of the prior art system of Goldstein.
Therefore, the claimed invention would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention
As of Claim 2: Goldstein in view of David further teaches a step of generating multiple multi-mode images and applying the multiple multi-mode images to an untrained object recognition machine learning (ML) model to generate the trained ML model to recognize objects in multi-mode images (Goldstein ¶¶0106-0108,0117,0124,0134-0136,0150,0296).
As of Claim 3: Goldstein in view of David further teaches the ML model correlates objects with surface types, wherein surface types are categorized by reflectiveness, and wherein reflectiveness is determined as being proportional to a pixel value of the at least one of the multiple NIR pulse-only images (Goldstein ¶¶0162-0165,0170,0226 and note that).
As of Claim 4: Goldstein in view of David further teaches the ML model provides object recognition for one of an advanced driver-assistance system (ADAS), an autonomous driving system, an anti-collision system, a train system, and a drone detection system (Goldstein ¶¶0366-0369 and note that).
As of Claim 5: Goldstein in view of David further teaches the ML model is trained to detect retro-reflecting objects including drones, observation systems, video cameras, optical lenses, and binoculars (Goldstein ¶¶0126,0161-0162,0187 and note that).
As of Claim 6: Goldstein in view of David further teaches brightness of pixels of the NIR pulse-only images is proportional to an amount of return laser pulse captured from an object and to a distance of the object (Goldstein ¶¶0073,0077,0101 and note that).
As of Claim 7: Goldstein in view of David further teaches the RGB and NIR sensors are separate image sensors (Goldstein ¶¶0066-0067 and note that).
As of Claim 8: Goldstein in view of David further teaches the RGB and red (NIR) image sensors are a merged sensor including both RGB and NIR sensitive pixel elements in a single chip (Goldstein ¶¶00,00,00 and note that).
As of Claim 9: Goldstein in view of David further teaches the FOV is a mutual subset of total fields of view of the RGB and NIR image sensors (Goldstein ¶¶0079,0095 and note that).
As of Claim 10: Claim 10 is a method of object recognition, implemented by a processor (140) having associated non-transient memory with instructions that when executed by the processor perform steps of claim 1 and all limitations area addressed in Claim 1. Moreover, Goldstein teaches in ¶0697 non-transient memory with instructions that when executed by the processor perform steps of claim 10.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MEKONNEN D DAGNEW whose telephone number is (571)270-5092. The examiner can normally be reached on 8:00AM-5:00PM M-Th.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Lin Ye can be reached on 571-272-7372. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MEKONNEN D DAGNEW/Primary Examiner, Art Unit 2638
Prosecution Insights