Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments with respect to claim(s) 1 have been considered but are moot in view of the new grounds of rejection.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1 is/are rejected under 35 U.S.C. 103 as being unpatentable over Waligorsky et al. (Waligorsky) (US 2016/0198147) in view of Krupka et al. (Krupka) (US 2014/0368613).
Regarding claim 1, Waligorsky discloses an image sensor for distance measurement ([0012] time of flight sensor 100), the image sensor comprising:
a pixel array comprising a plurality of unit pixels (FIG. 2, [0012], a pixel array including pixels);
a readout circuit configured to read out pixel signals from the pixel array in units of sub-frames and generate raw data ([0006], [0014], [0094], sub frames for each phase of the image light modulation is readout to produce raw image data during intervals IF1-IF4; [0087], [0094], [0142], 4 sub images are captured are different phases);
a preprocessing circuit configured to preprocess the raw data to generate phase data ([0049], [0069], [0077], calculations using phase data are performed) based on at least two sub-frames having different phase shifts in response to different photogate signals (FIG. 18, [0069], discrete raw frames having phase shifts of 0, 90, 180, and 270 degrees are processed to determine phase data);
a calibration circuit configured to generate correction data by performing a calibration operation on the phase data ([0049], phase correction (calibration) is performed, [0099], a correction may be applied to the depth values according to embodiments, to compensate for the error introduced by the phase transition);
an image signal processor (120) configured to generate depth information using the correction data ([0099], processor 120 can be configured to compute a first phase correction from an attribute of the first row that the depth first pixel is in, and then compute an adjusted first depth value from the first depth value and the first phase correction. Then a depth image of the object may be rendered, which includes the adjusted first depth value); and
an output interface circuit configured to output depth data comprising the depth information in units of depth frames ([0099], a depth image of the object may be rendered, which includes the adjusted first depth value).
Waligorsky is silent about a memory configured to store the phase data.
Krupka from the same or similar field of endeavor discloses a memory configured to store the phase data ([0054] measured phase data is stored).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Krupka into the teachings of Waligorsky for later retrieval and processing of the phase data.
Claim(s) 2 is/are rejected under 35 U.S.C. 103 as being unpatentable over Waligorsky et al. (Waligorsky) (US 2016/0198147) in view of Krupka et al. (Krupka) (US 2014/0368613), and further in view of Xu (CN 110488240 A).
Regarding claim 2, Waligorsky in view of Krupka discloses the image sensor of claim 1 (see claim 1 above).
Waligorsky in view of Krupka is silent about wherein the plurality of unit pixels have a 4-tap structure comprising first to fourth taps that respectively are configured to generate first to fourth pixel signals according to first to fourth photogate signals.
Xu from the same or similar field endeavor discloses wherein the plurality of unit pixels have a 4-tap structure comprising first to fourth taps that respectively are configured to generate first to fourth pixel signals according to first to fourth photogate signals (pg 6. Para. 4, each puxel has 4 taps for collecting 4 respective images).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Xu into the teachings of Waligorsky in view of Krupka for faster image collection and processing by allowing parallel capture and processing of 4 image signals.
Claim(s) 5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Waligorsky et al. (Waligorsky) (US 2016/0198147) in view of Krupka et al. (Krupka) (US 2014/0368613). And further in view of Min et al. (Min) (US 2012/0133737).
Regarding claim 5, Waligorsky in view of Krupka discloses the image sensor of claim 1 (see claim 1 above).
Waligorsky in view of Krupka is silent about wherein the plurality of unit pixels have a 2-tap structure comprising two taps that respectively are configured to generate a first pixel signal and a second pixel signal according to a first photogate signal and a second photogate signal.
Min from the same or similar field of endeavor discloses wherein the plurality of unit pixels have a 2-tap structure comprising two taps that respectively are configured to generate a first pixel signal and a second pixel signal according to a first photogate signal and a second photogate signal ([0050] a 2 tap pixel having two photogates PX1 and PX2).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Min into the teachings of Waligorsky in view of Krupka for higher resolution range finding
Claim(s) 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Waligorsky et al. (Waligorsky) (US 2016/0198147) in view of Krupka et al. (Krupka) (US 2014/0368613), and further in view of Takahashi et al. (Takahashi) (US 2017/0261611).
Regarding claim 6, Waligorsky in view of Krupka discloses the image sensor of claim 1 (See claim 1 above).
Waligorsky in view of Krupka is silent about wherein, in a first sub-frame, the pixel array is configured to generate the pixel signals according to a control signal having a first modulation frequency, and in a second sub-frame, the pixel array is configured to generate the pixel signals according to a control signal having a second modulation frequency that is different from the first modulation frequency.
Takahashi from the same or similar field of endeavor discloses wherein, in a first sub-frame, the pixel array is configured to generate the pixel signals according to a control signal having a first modulation frequency, and in a second sub-frame, the pixel array is configured to generate the pixel signals according to a control signal having a second modulation frequency that is different from the first modulation frequency ([0059], a first and a second sub frame use different modulation frequencies).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Takahashi into the teachings of Waligorsky in view of Krupka for higher resolution range finding (Takahashi: [0006]).
Claim(s) 7 and 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Waligorsky et al. (Waligorsky) (US 2016/0198147) in view of Krupka et al. (Krupka) (US 2014/0368613), and further in view of Kang (KR 20210032669 A).
Regarding claims 7 and 8, Waligorsky in view of Krupka discloses the image sensor of claim 1 (See claim 1 above).
Waligorsky in view of Krupka is silent about wherein the calibration circuit is configured to perform the calibration operation based on calibration information, and the calibration information comprises at least one selected from the group consisting of intrinsic characteristic parameters related to physical characteristics of the image sensor, a wiggling lookup table related to a wiggling effect, a fixed phase pattern noise (FPPN) lookup table related to FPPN, and temperature parameters related to external environment temperatures; and a memory is configured to store the calibration information.
Kang from the same or similar field of endeavor discloses wherein the calibration circuit is configured to perform the calibration operation based on calibration information, and the calibration information comprises a wiggling lookup table related to a wiggling effect (pg. 2, para. 11, a wiggling error based on a phase difference is determined and provided in the form of a table); and wherein the memory is configured to store the calibration information (pg. 13, para. 2, a memory stores a mapping table for the wiggle error).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Kang into the teachings of Waligorsky in view of Krupka for higher resolution range finding.
Claim(s) 9, 15, and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Waligorsky et al. (Waligorsky) (US 2016/0198147) in view of Krupka et al. (Krupka) (US 2014/0368613), and further in view of Bever et al. (Bever) (US 2014/0145281).
Regarding claim 9, Waligorksy in view of Krupka discloses a camera module comprising:
a light source unit configured to transmit an optical transmission signal to an object (Walligorsky: FIG. 1, [0046], light source 114 emits light to an object 101); and
an image sensor configured to receive an optical reception signal reflected from the object ([0012] time of flight sensor 100), wherein the image sensor comprises:
a pixel array comprising a plurality of unit pixels (see claim 1 above);
a control circuit configured to transmit a modulation signal to the light source unit and a plurality of demodulation signals to the pixel array (Walligorksy: [0006], light source modulation and pixel demodulation signals are sent);
a readout circuit configured to read out pixel signals from the pixel array in units of sub-frames and generate raw data (see claim 1 above);
a preprocessing circuit configured to preprocess the raw data to generate phase data (see claim 1 above) based on at least two sub-frames having different phase shifts in response to different photogate signals (FIG. 18, [0069], discrete raw frames having phase shifts of 0, 90, 180, and 270 degrees are processed to determine phase data);
a frame memory configured to store the phase data (see claim 1 above);
an image signal processor configured to generate depth information based on the phase data (see claim 1 above); and
an output interface circuit configured to output depth data comprising the depth information in units of depth frames (see claim 1 above).
Waligorksy in view of Krupka is silent about the modulation signal and the demodulation signals having an identical modulation frequency.
Bever from the same or similar field of endeavor the modulation signal and the demodulation signals having an identical modulation frequency ([0029], light modulation and pixel demodulation occur at the same modulation frequency).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Bever into the teachings of Waligorsky in view of Krupka for more accurate imaging.
Regarding claim 15, Waligorsky further discloses wherein the image sensor further comprises a calibration circuit configured to generate correction data by performing a calibration operation on the phase data ([0049], phase correction (calibration) is performed, [0099], a correction may be applied to the depth values according to embodiments, to compensate for the error introduced by the phase transition), and
the image signal processor is configured to generate the depth information using the correction data ([0079], [0099], depth data is generated using correction data).
Regarding claim 19, the limitations of claim 19 are rejected in the analysis of claims 1 and 9 above.
Claim(s) 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Waligorsky et al. (Waligorsky) (US 2016/0198147) in view of Krupka et al. (Krupka) (US 2014/0368613), and further in view of Bever et al. (Bever) (US 2014/0145281), and further in view of Xu (CN 110488240 A).
Regarding claim 10, Waligorsky in view of Krupka in view of Bever discloses the camera module of claim 9 (see claim 9 above). Walligorsky further discloses based on the first photogate signal, the second photogate signal has a phase difference of 90°, the third photogate signal has a phase difference of 180°, and the fourth photogate signal has a phase difference of 270° ([0076], images are captured at 0°, 90°, 180°, and 270°).
Waligorsky in view of Krupka in view of Bever is silent about wherein the plurality of unit pixels have a 4-tap structure comprising first to fourth taps configured to receive first to fourth photogate signals included in the plurality of demodulation signals.
Xu from the same or similar field of endeavor discloses wherein the plurality of unit pixels have a 4-tap structure comprising first to fourth taps configured to receive first to fourth photogate signals included in the plurality of demodulation signals (pg 6. Para. 4, each pixel has 4 taps for collecting 4 respective images).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Xu into the teachings of Waligorsky in view of Krupka in view of Bever for faster image collection and processing by allowing parallel capture and processing of 4 image signals.
Claim(s) 13 and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Waligorsky et al. (Waligorsky) (US 2016/0198147) in view of Krupka et al. (Krupka) (US 2014/0368613), and further in view of Bever et al. (Bever) (US 2014/0145281), and further in view of Takahashi et al. (Takahashi) (US 2017/0261611).
Regarding claim 13, the limitations of claim 13 are rejected in the analysis of claim 6.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Takahashi into the teachings of Waligorsky in view of Krupka in view of Bever for higher resolution range finding (Takahashi: [0006]).
Regarding claim 14, Waligorsky further discloses wherein the image signal processor is configured to generate the depth information corresponding to one depth frame based on a first piece of the phase data generated in the first sub-frame and a second piece of the phase data generated in the second sub-frame ([0006], [0014], [0094], sub frames for each phase of the image light modulation is readout to produce raw image data during intervals IF1-IF4; [0087], [0094], [0142], 4 sub images are captured are different phases; [0079], a depth frame is generated from the captured sub frames).
Claim(s) 16-18 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Waligorsky et al. (Waligorsky) (US 2016/0198147) in view of Krupka et al. (Krupka) (US 2014/0368613), in view of Bever et al. (Bever) (US 2014/0145281), and further in view of Kang (KR 20210032669 A).
Regarding claim 16, the limitations of claim 16 are rejected in the analysis of claim 7 (see claim 7 above).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Kang into the teachings of Waligorsky in view of Krupka in view of Bever for higher resolution range finding.
Regarding claim 17, the limitations of claim 17 are rejected in the analysis of claim 8 (see claim 8 above).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Kang into the teachings of Waligorsky in view of Krupka in view of Bever for higher resolution range finding.
Regarding claim 18, Waligorsky in view of Krupka in view of Bever discloses the camera module of claim 16 (see claim 16 above).
Waligorsky in view of Krupka in view of Bever is silent about a memory is configured to store the calibration information, and the image sensor is configured to receive the calibration information from the memory.
Kang from the same or similar field of endeavor discloses a memory is configured to store the calibration information (pg. 13, para. 2, a memory stores a mapping table for the wiggle error), and the image sensor is configured to receive the calibration information from the memory (pg. 2, para. 11, a wiggling error based on a phase difference is determined and provided in the form of a table).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Kang into the teachings of Waligorsky in view of Krupka in view of Bever for higher resolution range finding.
Regarding claim 20, the limitations of claim 20 are rejected in the analysis of claim 8 (see claim 8 above).
Allowable Subject Matter
Claims 3, 4, 11, and 12 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Tadmor (US 2019/0181171) ([0156] a 4 tap pixel is used for time of flight measurement, [0206], images are captured in sub-frames).
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JEFFERY A WILLIAMS whose telephone number is (571)270-7579. The examiner can normally be reached M-F 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sath Perungavoor can be reached at 571-272-7455. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JEFFERY A WILLIAMS/Primary Examiner, Art Unit 2488
Prosecution Insights