DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 24 is objected to because of the following informalities: The claim recites the limitation: “… processor is further programmed t: …”.
The limitation should recite: “… processor is further programmed to: … (emphasis added)”. Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3, 9-12, 19, 20 are rejected under 35 U.S.C. 103 as being unpatentable over Numata (US 2022/0150424 A1) in view of Zhang et al., hereinafter referred to as Zhang (“A CMOS SPAD Imager with Collision Detection and 128 Dynamically Reallocating TDCs for Single-Photon Counting and 3D Time-of-Flight Imaging”; Published: 11/17/2018; Pgs. 19; URL: https://doi.org/10.3390/s18114016).
As per claim 1, Numata discloses a method for detecting changes via an event camera (Numata: Abstract.), the method comprising:
monitoring a plurality of pixel measurements from an image sensor of a camera (Numata: Paras. [0016], [0020] disclose a pixel unit 110 constituting the event-based sensor 101 that includes a photoelectric conversion element 1011 [image sensor], counts the number of photons incident on the photoelectric conversion unit 111 [monitoring a plurality of pixel measurements] by the pixel counter 112.),
wherein the image sensor comprises a single photon avalanche diode (SPAD) sensor array (Numata: Para. [0020] discloses the photoelectric conversion unit 111 includes an avalanche photodiode (SPAD).),
determining an estimate of intensity (luminance) for a scene using a current frame (Numata: Paras. [0024], [0029] disclose the differential count value calculated by the comparator 116 corresponds to the reciprocal number of incidence frequency of photons and measures “a change in incidence frequency of photons”, that is, a change in luminance.);
detecting changes of the plurality of pixel measurements using the estimate of intensity for the scene (Numata: Paras. [0021]-[0022], [0024] disclose the memory 115 uses the comparator 116 to determine a differential count value between the current count value of the time counter 113 and the past count value of the time counter 113. In a case where the differential count value is equal to or greater than the second threshold a change in luminance is detected.);
maintaining a stored flux value for each of a first plurality of pixels, wherein the first plurality of pixels has not changed intensities (Numata: Paras. [0021]-[0022] disclose the memory 115 stores the past count values of the time counter 113. In a case where the differential count value is smaller than the second threshold [i.e., first plurality of pixels has not changed intensities], the second determination circuit 117 does not send a request signal.);
transmitting an intensity change value (outputs the count value associated with luminance changes) for each of a second plurality of pixels (pixels associated with luminance change that equal or exceed the second threshold), wherein the second plurality of pixels have changed intensities (Numata: Paras. [0021]-[0023] disclose in a case where the differential count value is equal to or greater than the second threshold [i.e., the second plurality of pixels have changed intensities], the second determination circuit 117 sends a request signal and outputs the count value of the time counter 113 to the horizontal read circuit 122.);
determining if a change in the scene has occurred based on the intensity change value for each of the second plurality of pixels (Numata: Paras. [0018], [0029], [0031]-[0032] disclose determining a scene change (such as intrusion or theft) by detecting a change in luminance by comparing the required times and differentiating between detection of a moving body/subject and changes in brightness of the whole screen or specific regions of the screen.); and
triggering an event (outputs address event signal) in response to the change in the scene (Numata: Paras. [0017], [0042] disclose outputting an address event signal indicating a change in the luminance. The outputs caused by the movement of the subject are detected as specific events.).
However, Numata does not explicitly disclose “… wherein the camera is configured to generate a high frame rate of incoming data that exceeds a readout bandwidth of the camera”.
Further, Zhang is in the same field of endeavor and teaches wherein the camera is configured to generate a high frame rate of incoming data that exceeds a readout bandwidth of the camera (Zhang: Section 1. Introduction, Pg. 1 discloses "a SPAD sensor with 32×32 pixels" and Section 1. Introduction, Pg. 2 discloses "enabling a photon throughput of 222 million counts per second (Mcps) and 465 Mcps [claimed high frame rate] … The sensor was measured and characterized in a 2D scanning LiDAR system [claimed camera]" and Zhang: Section 1. Introduction, Pg. 2 discloses "the IO bandwidth is the major limitation to the photon throughput … Therefore, to avoid the distortion due to photon pileup, the pixel activity must be restricted to 1-3% [describes claimed incoming data exceeds a readout bandwidth]" and further discloses "Instead of transmitting every event to the computer, the histogram of each point was constructed in the field-programmable gate array (FPGA) and transmitted … thus reducing the required transmission bandwidth [claimed incoming data exceeds a readout bandwidth]".).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, and having the teachings of Numata and Zhang before him or her, to modify the camera system of Numata to include the incoming data exceeding readout bandwidth of camera feature as described in Zhang. The motivation for doing so would have been to improve detection throughput of valid events by providing a configuration that enables advanced readout efficiency.
As per claim 11, Numata discloses a system for detecting changes (Numata: Abstract.), the system comprising:
a camera (event-based sensor 101) comprising:
an image sensor (Numata: Para. [0016] discloses photoelectric conversion element 1011.) comprising a single photon avalanche diode (SPAD) sensor array (Numata: Para. [0020] discloses the photoelectric conversion unit 111 includes an avalanche photodiode (SPAD).),
;
a processor (102) electrically coupled to the image sensor (1011), the processor programmed to (Numata: Fig. 1):
monitor a plurality of pixel measurements from the image sensor (Numata: Paras. [0016], [0020] disclose a pixel unit 110 constituting the event-based sensor 101 that includes a photoelectric conversion element 1011 [image sensor], counts the number of photons incident on the photoelectric conversion unit 111 [monitoring a plurality of pixel measurements] by the pixel counter 112.);
determine an estimate of intensity for a scene using a current frame (Numata: Paras. [0024], [0029] disclose the differential count value calculated by the comparator 116 corresponds to the reciprocal number of incidence frequency of photons and measures “a change in incidence frequency of photons”, that is, a change in luminance.);
detect changes of the plurality of pixel measurements using the estimate of intensity for the scene (Numata: Paras. [0021]-[0022], [0024] disclose the memory 115 uses the comparator 116 to determine a differential count value between the current count value of the time counter 113 and the past count value of the time counter 113. In a case where the differential count value is equal to or greater than the second threshold a change in luminance is detected.);
maintain a stored flux value for each of a first plurality of pixels, wherein the first plurality of pixels has not changed intensities (Numata: Paras. [0021]-[0022] disclose the memory 115 stores the past count values of the time counter 113. In a case where the differential count value is smaller than the second threshold [i.e., first plurality of pixels has not changed intensities], the second determination circuit 117 does not send a request signal.);
transmit an intensity change value for each of a second plurality of pixels, wherein the second plurality of pixels have changed intensities (Numata: Paras. [0021]-[0023] disclose in a case where the differential count value is equal to or greater than the second threshold [i.e., the second plurality of pixels have changed intensities], the second determination circuit 117 sends a request signal and outputs the count value of the time counter 113 to the horizontal read circuit 122.);
determine if a change in the scene has occurred based on the intensity change value for each of the second plurality of pixels (Numata: Paras. [0018], [0029], [0031]-[0032] disclose determining a scene change (such as intrusion or theft) by detecting a change in luminance by comparing the required times and differentiating between detection of a moving body/subject and changes in brightness of the whole screen or specific regions of the screen.); and
trigger an event in response to the change in the scene (Numata: Paras. [0017], [0042] disclose outputting an address event signal indicating a change in the luminance. The outputs caused by the movement of the subject are detected as specific events.).
However, Numata does not explicitly disclose “… wherein the camera is configured to generate a high frame rate of incoming data that exceeds a readout bandwidth of the camera”.
Further, Zhang is in the same field of endeavor and teaches wherein the camera is configured to generate a high frame rate of incoming data that exceeds a readout bandwidth of the camera (Zhang: Section 1. Introduction, Pg. 1 discloses "a SPAD sensor with 32×32 pixels" and Section 1. Introduction, Pg. 2 discloses "enabling a photon throughput of 222 million counts per second (Mcps) and 465 Mcps [claimed high frame rate] … The sensor was measured and characterized in a 2D scanning LiDAR system [claimed camera]" and Zhang: Section 1. Introduction, Pg. 2 discloses "the IO bandwidth is the major limitation to the photon throughput … Therefore, to avoid the distortion due to photon pileup, the pixel activity must be restricted to 1-3% [describes claimed incoming data exceeds a readout bandwidth]" and further discloses "Instead of transmitting every event to the computer, the histogram of each point was constructed in the field-programmable gate array (FPGA) and transmitted … thus reducing the required transmission bandwidth [claimed incoming data exceeds a readout bandwidth]".).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, and having the teachings of Numata and Zhang before him or her, to modify the camera system of Numata to include the incoming data exceeding readout bandwidth of camera feature as described in Zhang. The motivation for doing so would have been to improve detection throughput of valid events by providing a configuration that enables advanced readout efficiency.
As per claim 3, Numata discloses the method of claim 1, further comprising rendering an event camera image on a device (Numata: Paras. [0016]-[0017], [0029] disclose rendering luminance changes of a frame image of a subject captured by the information processing apparatus 100 using the event-based sensor 101 on display unit 104.).
As per claim 9, Numata discloses the method of claim 1, wherein determining the estimate of intensity for the scene using the current frame also uses an estimate of flux (incidence frequency of photons) at locations corresponding to each of the plurality of pixel measurements from the image sensor (Numata: Paras. [0024], [0029] disclose the differential count value calculated by the comparator 116 corresponds to the reciprocal number of incidence frequency of photons, and thus the photoelectric conversion element 1011 according to the present exemplary embodiment has a function of measuring “a change in incidence frequency of photons”, that is, a change in luminance detected as an address event in real time at each pixel address with respect to the output frame.).
As per claim 10, Numata discloses the method of claim 1, further comprising rendering an optical image of the scene if the change in the scene has occurred, wherein the optical image corresponds to a time of the change, and wherein the optical image comprises a third plurality of pixels corresponding to cumulative adaptive exposures (Numata: Fig. 4 & Paras. [0018], [0029] disclose the event-based sensor 101 also measures required time (the number of clocks) until the number of photons becomes equal to or greater than a first threshold, detects a change in luminance by comparing the required times and outputs frame images of the subject.).
As per claim 12, the claim(s) recites analogous limitations to claim(s) 3 above, and is/are therefore rejected on the same premise.
As per claim 19, the claim(s) recites analogous limitations to claim(s) 9 above, and is/are therefore rejected on the same premise.
As per claim 20, the claim(s) recites analogous limitations to claim(s) 10 above, and is/are therefore rejected on the same premise.
Allowable Subject Matter
Claims 4-7, 13-16, 21-24 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure and can be viewed in the list of references. Any inquiry concerning this communication or earlier communications from the examiner should be directed to PEET DHILLON whose telephone number is (571)270-5647. The examiner can normally be reached M-F: 5am-1:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sath V. Perungavoor can be reached at 571-272-7455. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PEET DHILLON/Primary Examiner
Art Unit: 2488
Date: 07-02-2026
Prosecution Insights