DETAILED ACTION
This office action is in response to application filed on April 8, 2024.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been received.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 04/08/2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
The listing of references in the specification is not a proper information disclosure statement. 37 CFR 1.98(b) requires a list of all patents, publications, or other information submitted for consideration by the Office, and MPEP § 609.04(a) states, “the list may not be incorporated into the specification but must be submitted in a separate paper.” Therefore, unless the references have been cited by the examiner on form PTO-892, they have not been considered.
In this particular case, the examiner notes that applicant describes in the specification the following references (see p. 5, lines 17-22):
Xinyu Zheng et al in “Modeling and Fabrication of a Nano-multiplication-region Avalanche Photodiode”, January 2007, esto.nasa.gov, which has been provided to the Office for consideration and has been listed in an Information Disclosure Statement.
Kang L. Wang et al in “Towards Ultimate Single Photon Counting Imaging CMOS Applications”, Workshop for Astronomy and Space Sciences, January 5 and 6, 2011, which has not been provided to the Office for consideration and has not been listed in an Information Disclosure Statement.
Appropriate action is required.
Response to Amendment
Preliminary amendments filed on April 8, 2024 have been entered.
The abstract of the disclosure has been amended.
The specification has been amended.
Claims 1-20 have been examined.
Drawings
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5) because they do not include the following reference sign(s) mentioned in the description:
In Figure 3: item ‘305’ as described in page 7, lines 3-7 (it appears item ‘205’ in Figure 3 should be renamed ‘305’).
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
The following title is suggested: “Multi-modal Imaging based on Photon Arrival Time”.
The abstract of the disclosure is objected to because it includes the language “(Fig. 4)” at the end of the description. A corrected abstract of the disclosure is required and must be presented on a separate sheet, apart from any other text. See MPEP § 608.01(b).
The disclosure is objected to because of the following informalities:
Page 3, lines 13-14: Language “Before a detailed description of the embodiments under reference of Fig. 1 to Fig .... , general explanations are made.” should read “Before a detailed description of the embodiments under reference of Fig. 1 to Fig. [[...]]16 , general explanations are made.” in order to correct for minor informalities.
Page 3, line 21: Language “diode, etc.), loudspeakers, etc., a (wireless) interface, etc., as it is generally known for electronic” should read “diode, etc.), loudspeakers, etc., a (wireless) interface, etc.), as it is generally known for electronic” in order to correct for minor informalities (i.e., add closing parenthesis).
Page 7, lines 6-8: Language “... , 210. Each sub-pixel 306, ... , 210 is allocated to a different task. In the specific example of Fig. 2, the super-pixel 305 comprises five sub-pixels: sub-pixels 306, 307, 308 sense red, green, blue, subpixel 309 senses the infrared components of the light and sub-pixel 210 is allocated to range” should read “... , [[210]]310. Each sub-pixel 306, ... , [[210]]310is allocated to a different task. In the specific example of Fig. [[2]]3, the super-pixel 305 comprises five sub-pixels: sub-pixels 306, 307, 308 sense red, green, blue, subpixel 309 senses the infrared components of the light and sub-pixel [[210]]310 is allocated to range” in accordance with the details of Figure 3.
Page 7, line 14: Language “subpixels as shown in the example depicted in Fig. 2 above …” should read “subpixels as shown in the example depicted in Fig. [[2]]3 above …” in order to correct for minor informalities (e.g., it appears Figure 3 is discussed in this paragraph).
Page 11, line 7: Language “SP AD array 901 in Fig. 8, the wrap-around counter 908 …” should read “SP AD array 901 in Fig. [[8]]9, the wrap-around counter 908 …” in order to correct for minor informalities (e.g., it appears Figure 9 is discussed in this paragraph).
Page 11, line 18: Language “sensor, then, at 1003, the timer value of the global counter 902 is obtained …” should read “sensor, then, at 1003, the timer value of the global counter [[902]]1002 is obtained …” in accordance with the details of Figure 10.
Page 11, line 27: Language “… As indicated by the arrows emerging from SPAD array 1001 in Fig. 8 …” should read “As indicated by the arrows emerging from SPAD array 1001 in Fig. [[8]]10 …”in order to correct for minor informalities (e.g., it appears Figure 10 is discussed in this paragraph).
Page 11, line 31: Language “implementation example of Fig. 8 a memory for storing …” should read “implementation example of Fig. [[8]]10, a memory for storing …”in order to correct for minor informalities (e.g., it appears Figure 10 is discussed in this paragraph).
Page 13, line 2: Language “If photons arrive faster than the resolution of the timer, than a double photon capture problem may” should read “If photons arrive faster than the resolution of the timer, then a double photon capture problem may” in order to correct for minor informalities.
Page 13, lines 12-13: Language “… The ,,linear“ signal represents the true signal. It goes into the correction in the sense that this is the “target” that is expected to be achieve” should read “… The “linear” signal represents the true signal. It goes into the correction in the sense that this is the “target” that is expected to be achieved” in order to correct for minor informalities.
Page 13, line 21: Language “approach described with regard to Fig. 2 above .” should read “approach described with regard to Fig. 2 above.” in order to correct for minor informalities (e.g., remove space before period).
Page 13, line 25: Language “an arrival time of 1000 photons per pixel …” should read “an arrival time of 10 photons per pixel …” in accordance with the language described in the paragraph (e.g., all other examples described in the paragraph refer to an arrival time of 10 photons per pixel).
Page 13, line 28: Language should end with a period.
Page 14, line 3: Language “reduces as compared to the output error without correction” should read “reduced as compared to the output error without correction.” in order to correct for minor informalities.
Appropriate correction is required.
Claim Objections
Claim 1 is objected to because of the following informalities:
Claim language should read “An electronic device comprising circuitry configured to determine times that pass between photon arrivals at a pixel of an imaging sensor or within larger areas of the imaging sensor, and to convert the times into intensity signals” in order to provide appropriate antecedence basis.
Appropriate correction is required.
Claim 3 is objected to because of the following informalities:
Claim language should read “The electronic device of claim 1, wherein the circuitry is configured to perform processing of [[the]] arrival time” in order to provide appropriate antecedence basis.
Appropriate correction is required.
Claim 4 is objected to because of the following informalities:
Claim language should read “The electronic device of claim 3, wherein the processing of the arrival time comprises performing arrival time averaging, applying [[an]] a low pass (LP) filter, and/or determining events” in order to clarify the recited subject matter.
Appropriate correction is required.
Claim 8 is objected to because of the following informalities:
Claim language should read “The electronic device of claim 1, wherein the circuitry is configured to determine intensity signals for [[the]] pixels of the imaging sensor based on sensor saturation” in order to provide appropriate antecedence basis.
Appropriate correction is required.
Claim 9 is objected to because of the following informalities:
Claim language should read “The electronic device of claim 1, wherein the circuitry is configured to determine the intensity signals for the pixel of the imaging sensor based on per pixel wrap-around counting” in order to provide appropriate antecedence basis.
Appropriate correction is required.
Claim 10 is objected to because of the following informalities:
Claim language should read “The electronic device of claim 1, comprising a global counter configured to measure [[the]] time for the imaging sensor” in order to provide appropriate antecedence basis.
Appropriate correction is required.
Claim 14 is objected to because of the following informalities:
Claim language should read “The electronic device of claim 1, wherein the imaging sensor uses Single Photon Avalanche Diodes (SPAD) technologies” in order to clarify the recited subject matter.
Appropriate correction is required.
Claim 16 is objected to because of the following informalities:
Claim language should read “The electronic device of claim 1, in which [[the]] pixels are synchronous pixels” in order to provide appropriate antecedence basis.
Appropriate correction is required.
Claim 19 is objected to because of the following informalities:
Claim language should read “A method comprising determining times that pass between photon arrivals at a pixel of an imaging sensor or within larger areas of the imaging sensor, and to convert the times into intensity signals” in order to provide appropriate antecedence basis.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 recites “An electronic device comprising circuitry configured to determine times that pass between photon arrivals at a pixel of an imaging sensor or within larger areas of the imaging sensor, and to convert these times into intensity signals” which is unclear as to what the scope of the claim is regarding “larger areas” (e.g., what is defined as “larger areas”? Larger in comparison to what? Larger than a pixel?).
Similar language is recited in independent claim 19, with none of the dependent claims clarifying the recited subject matter.
The specification recites similar language or appears to suggest larger areas compared to a pixel (see page 6, lines 19-22).
For examination purposes, claim language is interpreted as any area that is larger than a pixel.
Claim 16 recites “The electronic device of claim 1, in which the pixels are synchronous pixels” which is unclear as to what is meant by synchronous (e.g., synchronous with respect to time, to event, etc.?).
The original disclosure describes “The pixels may be synchronous pixels. For example, in synchronous pixels, events are in general not related to a global timing but may happen individually at each pixel, i.e. independently of the generation of events in other pixels of the sensor” (see page 4, lines 28-30), however, there is no clear definition regarding what makes a pixel synchronous.
For examination purposes, claim language is interpreted as described in the prior art of record (see below).
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 20 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim(s) does/do not fall within at least one of the four categories of patent eligible subject matter because the claim is directed to “a computer program” and according to the MPEP: “Non-limiting examples of claims that are not directed to any of the statutory categories include: Products that do not have a physical or tangible form, such as information (often referred to as “data per se”) or a computer program per se (often referred to as “software per se”) when claimed as a product without any structural recitations” (see MPEP 2106.03).
In order to overcome this rejection, the following language is suggested: “A non-transitory computer program”.
Examiner’s Note
Claims 1-19 were evaluated for patent eligibility under 35 U.S.C. 101 using the SUBJECT MATTER ELIGIBILITY TEST FOR PRODUCTS AND PROCESSES described in the 2024 Guidance Update on Patent Subject Matter Eligibility, Including on Artificial Intelligence (see also 2019 Revised Patent Subject Matter Eligibility Guidance) to determine patent eligibility under 35 U.S.C. 101.
Regarding claim 1, the examiner submits that under Step 1 of the test for evaluating claims for eligibility under 35 U.S.C. 101, the claim is to a machine, which is one of the statutory categories of invention.
Continuing with the analysis, under Step 2A - Prong One of the test, the examiner submits that claim 1 does not recite a judicial exception, therefore, the claim qualifies as eligible subject matter under 35 U.S.C.101 (see 2019 Revised Patent Subject Matter Eligibility Guidance – Revised Step 2A, see also MPEP 2106.04).
Similarly, independent claim 19 is directed to patent eligible subject matter as explained above with regards to claim 1.
Regarding the dependent claims 2-18, they were found to be patent eligible under 35 U.S.C. 101 by incorporating the eligible subject matter of their corresponding independent claims.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-4, 9-12, 14, 16 and 19 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Ingle (Ingle et al., “Passive Inter-Photon Imaging”, arXiv:2104.00059v2 [cs.CV], April 11, 2021, 27 pages, IDS reference), hereinafter ‘Ingle’.
Regarding claim 1.
Ingle discloses:
An electronic device (Suppl. Figs. 5 and 10 – “IP-SPAD hardware prototype”) comprising circuitry configured to determine times that pass between photon arrivals at a pixel of an imaging sensor or within larger areas of the imaging sensor (p. 2, section “Intensity from Inter-Photon Timing”: a hardware prototype consisting of a single-photon avalanche diode (SPAD) pixel mounted on two translation stages measures inter-photon timing information (see also p. 3, col. 2, par. 2; p. 3-4, section “3.1 Flux Estimator”)), and to convert these times into intensity signals (Abstract: intensity is obtained based on inter-photon timing information (see also p. 1, col. 2, par. 2)).
Regarding claim 2.
Ingle discloses all the features of claim 1 as described above.
Ingle further discloses:
the circuitry is configured to determine photon arrival times at the pixel of the imaging sensor (p. 1, section “How to Measure Inter-Photon Timing”: SPADs time-tag detected photons (see also Fig. 1b; p. 3, section “3.1 Flux Estimator”; p. 6, section “4.2 Single-Pixel IP-SPAD Hardware Prototype”, par. 2; section “Supplementary Note 7 Pixel Designs for Passive SPAD Imaging”, p. 16, last paragraph)).
Regarding claim 3.
Ingle discloses all the features of claim 1 as described above.
Ingle further discloses:
the circuitry is configured to perform processing of the arrival time (p. 1, col. 2, par. 2; p. 16, last paragraph: photon time-stamps are acquired and stored for modeling statistics of photon detection of successive photons, to relate average inter-photon delay and incident flux (see also section “Supplementary Document for “Passive Inter-Photon Imaging””, p. 1, regarding photon arrivals following a Poisson process)).
Regarding claim 4.
Ingle discloses all the features of claim 3 as described above.
Ingle further discloses:
the processing of the arrival time comprises performing arrival time averaging, applying an LP filter, and/or determining events (p. 3, section “3.1 Flux Estimator”: after each photon detection event, the IP-SPAD is unable to detect photons for a period of time in order to reset and store the timestamp of the most recent detected photon (see also section “Supplementary Document for “Passive Inter-Photon Imaging””, p. 1; section “Supplementary Note 4 Hardware Prototype”, p. 11, par. 1)).
Regarding claim 9.
Ingle discloses all the features of claim 1 as described above.
Ingle further discloses:
the circuitry is configured to determine intensity signals for the pixel of the imaging sensor based on per pixel wrap-around counting (p. 3, section “3.1 Flux Estimator”, par. 1: IP-SPAD pixel captures photons and increments total photon count, while repeating this process for an exposure time (in light of current application details at page 7, lines 27-31; see also section “Supplementary Note 7 Pixel Designs for Passive SPAD Imaging”, p. 17, section “SPAD Array Designs for Passive Imaging”, par. 2)).
Regarding claim 10.
Ingle discloses all the features of claim 1 as described above.
Ingle further discloses:
a global counter configured to measure the time for the whole sensor (Suppl. Fig. 10; section “Supplementary Note 7 Pixel Designs for Passive SPAD Imaging”, p. 16, last par. - p. 17, par. 2: data registers are used to store running average of inter-photon times).
Regarding claim 11.
Ingle discloses all the features of claim 1 as described above.
Ingle further discloses:
a global counter and per pixel memories for storing timer values (Suppl. Fig. 10; section “Supplementary Note 7 Pixel Designs for Passive SPAD Imaging”, p. 16, last par.; p. 17, section “SPAD Array Designs for Passive Imaging”, par. 2: a digital counter in the single pixel architecture is used for storing the number of detected photons while TDCs are used to acquire and store individual photon time-stamps in registers).
Regarding claim 12.
Ingle discloses all the features of claim 1 as described above.
Ingle further discloses:
a global counter and per pixel wrap-around counters (Suppl. Fig. 10; section “Supplementary Note 7 Pixel Designs for Passive SPAD Imaging”, p. 16, last par.; p. 17, section “SPAD Array Designs for Passive Imaging”, par. 2: a digital counter in the single pixel architecture is used for storing the number of detected photons while TDCs and counter in each pixel are used to acquire, count and store individual photon time-stamps in registers).
Regarding claim 14.
Ingle discloses all the features of claim 1 as described above.
Ingle further discloses:
the imaging sensor uses SPAD technologies (p. 2, section “Intensity from Inter-Photon Timing”: the hardware prototype consisting of a single-photon avalanche diode (SPAD) pixel mounted on two translation stages to measure inter-photon timing information (see also p. 3, col. 2, par. 2; p. 3-4, section “3.1 Flux Estimator”)).
Regarding claim 16.
Ingle discloses all the features of claim 1 as described above.
Ingle further discloses:
the pixels are synchronous pixels (p. 1, col. 2, par. 2; p. 7, section “4.3 Hardware Experiment Results – HDR Imaging”: IP-SPAD prototype captures brightness information corresponding to dark regions as well as bright pixels, which implies that information detected in one pixel is independent from detection in other pixels (in light of current application details at page 4, lines 28-30)).
Regarding claim 19.
Ingle discloses:
A method comprising determining times that pass between photon arrivals at a pixel of an imaging sensor or within larger areas of the imaging sensor (p. 2, section “Intensity from Inter-Photon Timing”: a hardware prototype consisting of a single-photon avalanche diode (SPAD) pixel mounted on two translation stages measures inter-photon timing information (see also p. 3, col. 2, par. 2; p. 3-4, section “3.1 Flux Estimator”)), and to convert these times into intensity signals (Abstract: intensity is obtained based on inter-photon timing information (see also p. 1, col. 2, par. 2)).
Claim Rejections - 35 USC § 103
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 5-8, 13, 15 and 17-18 are rejected under 35 U.S.C. 103 as being unpatentable over Ingle.
Regarding claim 5.
Ingle discloses all the features of claim 1 as described above.
Ingle does not explicitly disclose:
the circuitry is configured to determine derivatives of arrival time averages.
However, Ingle teaches:
“Intuitively, as the brightness increases, the time-of-darkness between consecutive photon detections decreases. By modeling the statistics of photon arrivals, we derive a theoretical expression that relates the average inter-photon delay and the incident flux. The key observation is that because photon arrivals are stochastic, the average inter-photon time decreases asymptotically as the incident flux increases. Using this novel temporal intensity cue, we design algorithms to estimate brightness from as few as one photon timestamp per pixel to extremely high brightness, beyond the saturation limit of conventional sensors” (p. 1, col. 2, par. 2: modeling statistics of photon detection of successive photons is used to derive relationship between average inter-photon delay (analogous to derivatives of arrival time averages in light of current application details at p. 10, lines 9-12) and incident flux (see also p. 17, par. 2 regarding storing running average of inter-photon times)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Ingle to configure the circuitry to determine derivatives of arrival time averages, in order to reduce the computational time of analysis by the circuitry while still providing improved results.
Regarding claim 6.
Ingle discloses all the features of claim 1 as described above.
Ingle does not explicitly disclose:
the circuitry is configured to determine derivatives of arrival times.
However, Ingle teaches:
“Intuitively, as the brightness increases, the time-of-darkness between consecutive photon detections decreases. By modeling the statistics of photon arrivals, we derive a theoretical expression that relates the average inter-photon delay and the incident flux. The key observation is that because photon arrivals are stochastic, the average inter-photon time decreases asymptotically as the incident flux increases. Using this novel temporal intensity cue, we design algorithms to estimate brightness from as few as one photon timestamp per pixel to extremely high brightness, beyond the saturation limit of conventional sensors” (p. 1, col. 2, par. 2: modeling statistics of photon detection of successive photons is used to derive relationship between average inter-photon delay (analogous to derivatives of arrival times in light of current application details at p. 10, lines 9-12) and incident flux).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Ingle to configure the circuitry to determine derivatives of arrival times, in order to reduce the computational time of analysis by the circuitry while still providing improved results.
Regarding claim 7.
Ingle discloses all the features of claim 1 as described above.
Ingle does not explicitly disclose:
the circuitry is configured to determine averages of derivatives of arrival times.
However, Ingle teaches:
“Intuitively, as the brightness increases, the time-of-darkness between consecutive photon detections decreases. By modeling the statistics of photon arrivals, we derive a theoretical expression that relates the average inter-photon delay and the incident flux. The key observation is that because photon arrivals are stochastic, the average inter-photon time decreases asymptotically as the incident flux increases. Using this novel temporal intensity cue, we design algorithms to estimate brightness from as few as one photon timestamp per pixel to extremely high brightness, beyond the saturation limit of conventional sensors” (p. 1, col. 2, par. 2: modeling statistics of photon detection of successive photons is used to derive relationship between average inter-photon delay (analogous to averages of derivatives of arrival times in light of current application details at p. 10, lines 9-12) and incident flux (see also p. 17, par. 2 regarding storing running average of inter-photon times)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Ingle to configure the circuitry to determine averages of derivatives of arrival times, in order to reduce the computational time of analysis by the circuitry while still providing improved results.
Regarding claim 8.
Ingle discloses all the features of claim 1 as described above.
Ingle does not explicitly disclose:
the circuitry is configured to determine intensity signals for the pixels of the imaging sensor based on sensor saturation.
However, Ingle teaches:
“Intuitively, as the brightness increases, the time-of-darkness between consecutive photon detections decreases. By modeling the statistics of photon arrivals, we derive a theoretical expression that relates the average inter-photon delay and the incident flux. The key observation is that because photon arrivals are stochastic, the average inter-photon time decreases asymptotically as the incident flux increases. Using this novel temporal intensity cue, we design algorithms to estimate brightness from as few as one photon timestamp per pixel to extremely high brightness, beyond the saturation limit of conventional sensors” (p. 1, col. 2, par. 2: modeling statistics of photon detection of successive photons is used to derive relationship between average inter-photon delay and incident flux, which is employed to estimate brightness even beyond saturation limit of conventional sensors (see also Figs. 5 and 8; Suppl. Fig. 9; p. 7, section “4.3 Hardware Experiment Results”, par. 1)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Ingle to configure the circuitry to determine intensity signals for the pixels of the imaging sensor based on sensor saturation, in order to improve detail capture in extremely bright regions.
Regarding claim 13.
Ingle discloses all the features of claim 1 as described above.
Ingle further discloses:
a global counter (Suppl. Fig. 10; section “Supplementary Note 7 Pixel Designs for Passive SPAD Imaging”, p. 16, last par.; p. 17, section “SPAD Array Designs for Passive Imaging”, par. 2: a digital counter in the single pixel architecture is used for storing the number of detected photons).
Ingle does not explicitly disclose:
a saturation counter.
However, Ingle teaches:
“Intuitively, as the brightness increases, the time-of-darkness between consecutive photon detections decreases. By modeling the statistics of photon arrivals, we derive a theoretical expression that relates the average inter-photon delay and the incident flux. The key observation is that because photon arrivals are stochastic, the average inter-photon time decreases asymptotically as the incident flux increases. Using this novel temporal intensity cue, we design algorithms to estimate brightness from as few as one photon timestamp per pixel to extremely high brightness, beyond the saturation limit of conventional sensors” (p. 1, col. 2, par. 2: modeling statistics of photon detection of successive photons is used to derive relationship between average inter-photon delay and incident flux, which is employed to estimate brightness even beyond saturation limit of conventional sensors (see also Figs. 5 and 8; Suppl. Fig. 9; p. 7, section “4.3 Hardware Experiment Results”, par. 1)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Ingle to implement a saturation counter, in order to improve detail capture in extremely bright regions while preventing damage of the electronic device.
Regarding claim 15.
Ingle discloses all the features of claim 14 as described above.
Ingle does not explicitly disclose:
the imaging sensor is a giga-pixel image sensor.
However, Ingle teaches:
“The theoretical analysis and experimental results in this paper were restricted to a single SPAD pixel. For most passive imaging applications, in practice, there will be a need to scale this method to large form factor SPAD arrays with thousands of pixels” (p. 17, section “SPAD Array Designs for Passive Imaging”, par. 1: prototype can be scaled to form SPAD arrays with thousands of pixels (analogous to a giga-pixel image sensor)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Ingle to implement the imaging sensor as a giga-pixel image sensor, in order to provide improved imaging systems for high-resolution demanding applications.
Regarding claim 17.
Ingle discloses all the features of claim 1 as described above.
Ingle does not explicitly disclose:
the imaging sensor is configured to support different modalities.
However, Ingle teaches:
“Using this novel temporal intensity cue, we design algorithms to estimate brightness from as few as one photon timestamp per pixel to extremely high brightness, beyond the saturation limit of conventional sensors” (p. 1, col. 2, par. 2: algorithms estimate brightness on a pixel from one detected photon up to detection beyond saturation (analogous to different modalities)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Ingle to configure the imaging sensor to support different modalities, in order to provide flexible imaging systems that adjust to different applications.
Regarding claim 18.
Ingle discloses all the features of claim 1 as described above.
Ingle does not explicitly disclose:
the imaging sensor is configured to be applied in Event-Based Vision Sensors (EVS).
However, Ingle teaches:
“The proposed techniques, aided by the emergence of single-photon sensors such as single-photon avalanche diodes (SPADs) with picosecond timing resolution, will have implications for a wide range of imaging applications: robotics, consumer photography, astronomy, microscopy and biomedical imaging” (Abstract: proposed technology can be used in high-quality fields (analogous to applied in Event-Based Vision Sensors)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Ingle to configure the imaging sensor to be applied in Event-Based Vision Sensors (EVS), in order to provide flexible imaging systems that adjust to different applications.
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Ingle, in view of Ingle (US 20200036918 A1, IDS reference), hereinafter “Ingle’918”.
Regarding claim 20.
Ingle discloses all the features of claim 19 as described above.
Ingle does not explicitly disclose:
A computer program comprising instructions which are configured to, when executed on a processor, perform the method of claim 19.
Ingle’918 teaches:
“In accordance with some embodiments of the disclosed subject matter, a non-transitory computer readable medium containing computer executable instructions that, when executed by a processor, cause the processor to perform a method for high dynamic range imaging using an image sensor comprising a pixel, the pixel comprising a single photon detector configured to detect the arrival of a single photon, and having a dead time τd and a counter coupled to an output of the single photon detector, wherein the counter is configured to increment in response to a signal indicative of detection of a photon output by the single photon detector, the method comprising: reading out, from the pixel, a value stored by the counter after an exposure time has elapsed; and calculating an intensity value for the pixel based on the value and the dead time τd” ([0026]: memory contains instructions to be executed by a processor to perform a method for high dynamic range imaging including calculating the intensity value of a pixel based on detecting photons arrival).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Ingle in view of Ingle’918, to implement a computer program comprising instructions which are configured to, when executed on a processor, perform the method of claim 19, in order to facilitate the application and analysis of the prototype.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Cheimets; Peter N., US 20090190001 A1, PHOTON COUNTING IMAGING SYSTEM
Reference discloses an imaging device including photon sensors to capture arrival times of photons on different image planes, while also capturing photon colors.
HYNECEK; Jaroslav, US 20190244986 A1, SINGLE-PHOTON AVALANCHE DIODE IMAGE SENSOR WITH PHOTON COUNTING AND TIME-OF-FLIGHT DETECTION CAPABILITIES
Reference discloses an imaging sensor using single-photon avalanche diode technology as well as time of flight detection capabilities.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LINA CORDERO whose telephone number is (571)272-9969. The examiner can normally be reached 9:30 am - 6:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ANDREW SCHECHTER can be reached at 571-272-2302. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/LINA CORDERO/Primary Examiner, Art Unit 2857
Prosecution Insights