DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1, 3-21 are pending.
Response to Arguments
Applicant's arguments filed 04/08/2026 have been fully considered.
Regarding claims 1 and 16, the amendment necessitates a new ground of rejection. See below for detail claim mapping.
Regarding claims 16 and 20, the amendment fails to overcome the rejection. See detail below for new mapping.
However, the amendment to claim 1 has overcome rejections for claims 3-7, and 13-15. See allowable subject matter below.
Allowable Subject Matter
Claims 3-7 and 13-15 objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 3, prior art alone or as combined, fails to teach the multi-bit confidence being based on comparing with a threshold value, or comparing with neighboring pixel reading, when considered together with all limitations of claim 1.
Regarding claim 6, prior art alone or as combined, fails to teach a compression flag to switch between differential amplification of first and second single ended signal and direct reading of the first and second single ended signal, as considered together with everything in claim 1.
Regarding claim 13, prior art alone or as combined fails to teach calibrating the readout circuitry mismatch using know values. Previously, Wojewoda, US 20080278359 A1 (“Wojewoda”) was considered. However, Wojewoda teaches calibration of analog-to-digital converter (ADC) to correct mismatch and offset. There is no reason to apply the teach to the two-single-ended pixel circuitry as the underline issue is different. Moreover, given Bamji directly measures the mismatch and offset ([0077]), there is no need for such additional calibration. Therefore, the amendment to claim 1 has overcome the rejection for claim 13.
Claims 4, 5, 7, 14, and 15 are allowable due to claim dependency.
Information Disclosure Statement
The information disclosure statement (IDS) filed 6/8/2026 and 2/4/2026 are in compliance with the provisions of 37 CFR 1.97 and 1.98. Accordingly, the information disclosure statement are being considered by the examiner.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 8-11, 16, and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Bamji et al., US 20180176492 A1 (“Bamji”), and Watanabe et al., US 20210368123 A1(“Watanabe”).
Regarding claim 1, Bamji teaches a time of flight (ToF) camera system (Figs. 1, 2, 6) comprising:
an image sensor comprising a plurality of differential imaging pixels (Fig. 6, [0065]); and
an image acquisition system coupled to the image sensor (control logic 672) and configured to:
readout the plurality of differential imaging pixels by reading out a first single ended signal and a second single ended signal from each of the plurality of differential imaging pixels (Figs. 6-7, [0074], signal at A-side and B-side);
for each of at least some of the plurality of differential imaging pixels, determine pixel data based on the first single ended signal and the second single ended signal, wherein the pixel data comprises a difference value indicative of a difference between the first single ended signal and the second single ended signal ([0077], Vdiff)
…
output the pixel data to a processor for the determination of the ToF image frame (Fig. 8, [0092]).
Bamji fails to teach
a multi-bit confidence value indicative of reliability of the difference value for use in a determination of a ToF image frame,
wherein the confidence value is based on a plurality of comparisons involving at least one of the first single ended signal or the second single ended signal, and wherein each comparison of the plurality of comparisons corresponds to one or more bits of the confidence value that are indicative of a result of the comparison.
However, Watanabe teaches a method to improve dynamic range by adjusting exposure time based on receiving light intensity (Watanabe, Figs 9 and 10, [0128]). In particular, plurality of comparisons are discussed, comparing between the different taps to determine the max value, comparing the max Q between HG, MG, and LG. Based on the comparing, the result from HG, MG and LG is selected (Figs. 11-12).
It would have been obvious to one of ordinary skill in the art to have modified Bamji’s system, in view of Watanabe’s teaching, to adjust exposure time to three different ways based on the plurality of comparison. Since three options would be resulted from the comparison, it would be obvious to one of ordinary skill in the art to at least use 2 bits to record whether HG, MG, or LG is most reliable.
Therefore, Bamji, as modified in view of Watanabe, teaches
a multi-bit confidence value indicative of reliability of the difference value for use in a determination of a ToF image frame (as discussed above regarding Watanabe, 2 bits to indicate HG, MG, or LG is more reliable),
wherein the confidence value is based on a plurality of comparisons involving at least one of the first single ended signal or the second single ended signal, and wherein each comparison of the plurality of comparisons corresponds to one or more bits of the confidence value that are indicative of a result of the comparison (comparing max value of A and B in two comparisons as shown in Watanabe Fig. 10).
Regarding claim 8, Bamji, as modified in view of Watanabe, teaches the ToF camera system of claim 1, wherein the processor is configured to determine a ToF image based on the pixel data received from the image acquisition system (Bamji, Fig. 8, [0092]).
Regarding claim 9, Bamji, as modified in view of Watanabe, teaches the ToF camera system of claim 1, wherein the ToF camera system is a continuous wave ToF camera system (Bamji, Fig. 2, [0028], modulated signal 230).
Regarding claim 10, Bamji, as modified in view of Watanabe, teaches the ToF camera system of claim 1, wherein the image acquisition system comprises first readout circuitry for reading out the first single ended signal and second readout circuitry for reading out the second single ended signal (Bamji, Fig. 6, [0070], readout circuitry 550 and 650).
Regarding claim 11, Bamji, as modified in view of Watanabe, teaches the ToF camera system of claim 10, further configured to correct an offset between the first single ended signal and the second single ended signal caused by mismatch of the first readout circuitry and the second readout circuitry (Bamji, [0076], [0077]).
Regarding claim 16, it is a method claim corresponding to system claim 1. It is rejected for the same reason.
Regarding claim 21, Bamji, as modified in view of Watanabe, teaches the ToF camera system of claim 1, wherein the confidence value is further indicative of at least one of signal strength, a saturated differential imaging pixel, or a readout problem (Watanabe, Figs 9 and 10, [0128], Figs 11-12, signal strength, a saturated differential imaging are both covered).
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Bamji, and Watanabe as applied to claims 11 above, and further in view of Yasunobu Hitomi, US 20190227174 A1 (“Hitomi”).
Regarding claim 12, Bamji, as modified in view of Watanabe, teaches the ToF camera system of claim 11. However, the combination does not teach:
the image sensor comprises at least one row of blank pixels configured such that incident light on the image sensor does not result in charge accumulation in the blank pixels, and wherein correcting the offset between the first single ended signal and the second single ended signal for a particular imaging pixel comprises:
determining a difference between a first single ended signal of a blank pixel that is in the same pixel column as the particular imaging pixel and a second single ended signal of the blank pixel that is in the same pixel column as the particular imaging pixel; and
correcting the offset between the first single ended signal and the second single ended signal for the particular imaging pixel using the determined difference.
On the other hand, Hitomi teaches configuring one or more rows of blank pixels such that incident light on the image sensor does not result in charge accumulation in the blank pixels ([0034]). The purpose is to measure dark offset and uncontrollable charge accumulation. This will help determine the optimal integration time for best signal to noise ratio (SNR) (Figs 7-8, [0059], [0080], [0090]).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention, to have further modified Bambi’s system, in view of Hitomi‘s teaching, to configure one or more rows of blank pixels such that incident light on the image sensor does not result in charge accumulation in the blank pixels to measure dark offset and uncontrollable charge accumulation, and to determine optimal integration time for best SNR.
Therefore, Bamji, as modified in view of Watanabe and Hitomi, teaches
wherein the image sensor comprises at least one row of blank pixels configured such that incident light on the image sensor does not result in charge accumulation in the blank pixels, and wherein correcting the offset and wherein correcting the offset between the first single ended signal and the second single ended signal for a particular imaging pixel comprises (Hitomi, Figs 7-8, [0059], [0080], [0090]):
determining a difference between a first single ended signal of a blank pixel that is in the same pixel column as the particular imaging pixel and a second single ended signal of the blank pixel that is in the same pixel column as the particular imaging pixel (Bamji, [0076], [0077]); and
correcting the offset between the first single ended signal and the second single ended
signal for the particular imaging pixel using the determined difference (Bamji, [0077]).
Claims 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over RAJASEKARAN, US 20210044767 Al, ("Rajasekaran"), Yaghmai, (US 20100231771 Al), ("Yaghmai"), and Schemmann et.al., WO 2007135175 A2, ("Schemmann").
Regarding claim 17, Rajasekaran teaches a camera system (Figs. 6-8) comprising:
an image sensor configured to receive light reflected by an object being imaged, wherein the image sensor comprises a plurality of differential imaging pixels (Figs, 6-8, [0026], [0029]); and
an image acquisition system (Fig. 6, controller 608, [0027]) coupled to the image sensor and configured to:
control charge accumulation timing of the plurality of differential imaging pixels such that a first side of the differential imaging pixels accumulates charge for a first period of time and a second side of the differential imaging pixels accumulates charge for a second period of time, wherein the first period of time is longer than the second period of time (Fig. 8, [0030], different integration time for PGA and PGB);
readout from each of the plurality of differential imaging pixels a first single ended signal indicative of a charge accumulated by the first side of the differential imaging pixel and a second single ended signal indicative of a charge accumulated by the second side of the differential imaging pixel (Fig. 8, [0032], readout 808),
the camera system configured to determine an image using the signals readout from the image sensor (Fig. 11, [0035]), wherein determining the image comprises using, for each of the plurality of differential pixels, determining a corresponding pixel of the image based on […] how close the first single ended signal is to saturation ([0036], selecting pixel charge from either PGA or PGB depending on whether PGA is close to a saturation threshold).
However, Rajasekaran fails to teach the pixel charge is determined based on a combination of a first weighted value of the first single ended signal and a second weighted value of a normalized version of the second single ended signal, with the combination being defined by how close the first single ended signal is to saturation.
However, Schemmann teaches determining a pixel image based on a combining weighted average of a bright and less bright integration signal based on a saturation level (p. 6 2nd para., p. 7 ll. 19 – 32; p. 26 ll. 6-15).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the invention, to have modified Rajasekaran’s system, in view of Schemmann’s teaching, to further determine a pixel image based on a combination of a first weighted value of the first single ended signal and a second weighted value of […] the second single ended signal, with the combination being defined by how close the first single ended signal is to saturation by how close the first single ended signal is to saturation.
The motivation to do so is applying a known technique to a known device for predictable result. Here, the predictable result is more accurately determining the charge exposure and improving image dynamic range (Schemmann, p. 7 ll. 19 – 32).
However, Rajasekaran, as modified in view of Schemmann does not teach determining a pixel image with a normalized version of the second single ended signal.
On the other hand, Yaghmai, teaches determine image of a pixel with a normalized of version of low exposure period signal based on the ratio of exposure T1/T2 ([0050]).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention, to have further modified Rajasekaran’s system, in view of Yaghmai’s teaching, to apply a normalization to the low exposure light signal (PGB) when there is saturation so as to approximate the PGA when saturation could not be properly determined. In other words, to better capture the received light intensity for distance measurement.
Regarding claim 18, Rajasekaran, as modified in view of Schemmann and Yaghmai, teaches the system of claim 17, wherein determining the image further comprises:
for each of the plurality of differential imaging pixels, determining from the first single ended signal whether or not the first side of the differential imaging pixel is saturated (Rajasekaran, [0036]), and
if the first side of the differential imaging pixel is determined not to be saturated, use the first single ended signal for the determination of the image (Rajasekaran, [0036]), otherwise
if the first side of the differential imaging pixel is determined to be saturated, use the second single ended signal for the determination of the image (Rajasekaran, [0036]).
Regarding claim 19, Rajasekaran, as modified in view of Schemmann and Yaghmai, teaches the system of claim 18, wherein determining whether or not the first side of the differential imaging pixel is saturated comprises comparing the first single ended signal to a saturation threshold (Rajasekaran, [0036]).
Regarding claim 20, Rajasekaran, as modified in view of Schemmann and Yaghmai, teaches the system of claim 17, wherein the system is further configured to determine a size of a first weighting applied to the first single ended signal to obtain the first weighted value and a size of a second weighting applied to the second single ended signal to obtain the second weighted value, based on how close the first single ended signal is to saturation (Schemmann, p. 6, ll. 13-26, p. 7 ll. 19 – 32).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YUQING XIAO whose telephone number is (571)270-3603. The examiner can normally be reached on 8AM-5PM EST M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Joseph Thomas can be reached at (571)272-8004. The fax phone number for the organization where this application or proceeding is assigned is 571-2730-4603.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YUQING XIAO/Supervisory Patent Examiner, Art Unit 3645
Prosecution Insights