DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Drawings
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(4) because:
reference character “19” has been used to designate two different parts. One is in Fig. 1, page 5, line 9, an illumination device 19. The other one is in page 5, line 12, the reflected light 19.
reference character “45” has been used to designate two different parts. One is in Fig. 4, page 7, line 29, optical filter 45. The other one is in page 7, line 29, microlenses 45.
reference characters “19” and “RL” have been used to designate “the reflected light” in Fig. 1, Spec. page 5, line 12 and Spec. page 5, line 15.
reference characters “45” and “46” have been used to designate “microlenses” in Fig. 4 Spec. page 8, line 13 and Spec. page 7, line 29.
reference characters “2101” and “1201” have been used to designate “CPU” in Fig. 8 and Spec. page 12, line 17.
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-2, 4, 7, 14 and 17 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Tan et al. (US 20070181810 A1, hereinafter “Tan”).
Regarding claim 1, Tan teaches a multi-sensing pixel array that comprises, within a multi-layer stacked die, sensing pixels and an active illumination (Tan; Fig. 7A, [0039]-[0040], one or more photodetector devices 702 are integrated with semiconductor laser 701 of laser array 700. Thus photodetectors 702 can be fabricated along with lasers 701, e.g., part of the same semiconductor die, or can be integrated with the various lasers into a single composite device; [0022], VCSELs are particularly useful for the disclosed lidar application. VCSELs emits orthogonal to the plane of the active layers. Mirrors are formed above and below the optical cavity. Further, VCSEL incorporate the mirrors monolithically in their design, they allow for fabrication of integrated 1D/2D laser arrays with close device spacing. The VCSEL is known as multi-layer stacked device. Therefore, VCSEL light source 701 (with stacked layer) and photodetector devices 702 are within a multi-layer stacked die).
Regarding claim 2, Tan teaches the multi-sensing pixel array of claim 1, in which the sensing pixels comprise depth sensing pixels (Tan; [0019]-[0020], disclosed time of flight analysis of laser pulses to detect the maximum range of analyzed objects for distance measurement. The photodetectors used in Fig. 7A is depth sensing pixels).
Regarding claim 4, Tan teaches the multi-sensing pixel array of claim 1, in which the sensing pixels and the active illumination are arranged in a single pixel array (Tan; Fig. 7A, [0040], one of photodetector devices 702 are integrated with semiconductor lasers 701 of laser array 700. Thus, photodetectors 702 can be fabricated along with lasers 701, e.g., part of the same semiconductor die, or can be integrated with the various lasers into a single composite device).
Regarding claim 7, Tan teaches the multi-sensing pixel array of claim 1, in which the depth sensing pixels comprise ToF pixels (Tan; [0019]-[0020], disclosed time of flight analysis of laser pulses to detect the maximum range of analyzed objects for distance measurement. The photodetectors used in Fig. 7A is depth sensing pixels using TOF measurement).
Regarding claim 14, Tan teaches the multi-sensing pixel array of claim 1, wherein, for each active illuminator, a respective microlens is arranged to create a field of illumination of the illuminator (Tan; Fig. 4a-4b, [0032], laser and optical system 400 includes laser array 410 and optical element 420. Optical element 420 includes an integrated collimating lens array 430 (equivalent to microlens array) which are designed and located to work in conjunction with each of the lasers in laser array 410).
Regarding claim 17, Tan teaches the multi-sensing pixel array of claim 1, wherein the multi-sensing pixel array is implemented in a single IC (Tan; Fig. 7A, [0040], one of photodetector devices 702 are integrated with semiconductor lasers 701 of laser array 700. Thus, photodetectors 702 can be fabricated along with lasers 701, e.g., part of the same semiconductor die, or can be integrated with the various lasers into a single composite device).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 3 and 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tan, modified in view of Malinovich (WO 0249367 A2, hereinafter “Malinovich”).
Regarding claim 3, Tan teaches the multi-sensing pixel array as recited in claim 1.
Tan does not teach,
in which the sensing pixels comprise imaging pixels and depth sensing pixels.
Malinovich teaches in Fig. 1, page 9, line 16, IR-RBG photosurface 20 used in a 3D picture camera having a tiling configuration of IR pixels 21 (distance sensing pixels are IR pixels (page 4, line 16)), R pixels 22, G pixels 23 and B pixels 24 (RGB pixels are imaging pixels).
It would have been obvious to one of ordinary skill in the art prior to the effective filling date of this invention to modify the multi-sensing pixel array taught by Tan to include the sensing pixels comprise imaging pixels and depth sensing pixels taught by Malinovich with a reasonable expectation of success. The reasoning for this is integrating both RGB imaging pixel and IR depth pixel to detect both image and depth information of the field of view simultaneously predictably to reduce device size.
Regarding claim 6, Tan teaches the multi-sensing pixel array as recited in claim 1.
Tan does not teach,
in which the imaging pixels comprise RGB pixels or IR pixels, or combinations thereof.
Malinovich teaches in Fig. 1, page 9, line 16, IR-RBG photosurface 20 used in a 3D picture camera having a tiling configuration of IR pixels 21 (distance sensing pixels are IR pixels (page 4, line 16)), R pixels 22, G pixels 23 and B pixels 24 (RGB pixels are imaging pixels).
It would have been obvious to one of ordinary skill in the art prior to the effective filling date of this invention to modify the multi-sensing pixel array taught by Tan to include in which the imaging pixels comprise RGB pixels or IR pixels, or combinations thereof taught by Malinovich with a reasonable expectation of success. The reasoning for this is comprising RGB pixels to detect the scene in a color picture. Furthermore, the RGB pixels integrates with IR pixel allows a single image sensor to capture both visible color and infrared light simultaneously with both image and depth information.
Claim(s) 5 and 8-11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tan, modified in view of Chen (US 20230048279 A1, hereinafter “Chen”).
Regarding claim 5, Tan teaches the multi-sensing pixel array as recited in claim 1.
Tan does not teach,
in which the sensing pixels and the active illuminators share a main optical stack.
Chen disclosed in Fig. 8, [0058], the optical device 3 includes transmitter 11, receivers 12 and 13 and the optical element 24 with encapsulant layer 16 (equivalent to main optical stack). the optical element 24 includes an optical element 241, 242 and 243 formed in one piece (equivalent to the sensing pixels and the active illuminators share a main optical stack). Portion 241 contacts the receiver 13, portion 242 contacts the receiver 12 and portion 243 contacts the transmitter 11. The portions 241 and 242 or the optical element may be similar to the optical element 23 [0055], [0056]. The optical element 24 may include a diffractive optical element , such as a beam shaper, splitter, diffuser, diffractive focusing Lense, and/or a grating. The portion 243 of the optical element 24 may be configured to modify light from the transmitter 11. In some arrangements, the portion 243 of the optical element 24 may be configured to shape, split, diffuse, or diffract light from the transmitter 11.
It would have been obvious to one of ordinary skill in the art prior to the effective filling date of this invention to modify the multi-sensing pixel array taught by Tan to include in which the sensing pixels and the active illuminators share a main optical stack taught by Chen with a reasonable expectation of success. The reasoning for this is sharing the same main optical stack with sensing pixels and the active illuminator predictably to reduce the size of the optical device.
Regarding claim 8, Tan teaches the multi-sensing pixel array as recited in claim 1.
Tan does not teach,
comprising a main optical stack, an imaging stack, and an illumination stack.
Chen disclosed in Fig. 8, [0058], the optical device 3 includes transmitter 11 (equivalent to an illumination stack), receiver 12 (detect infrared light [0028]) and 13 (detector visible light [0029]) (equivalent to imaging stack) and the optical element 24 with an encapsulant 16 (equivalent to main optical stack).
It would have been obvious to one of ordinary skill in the art prior to the effective filling date of this invention to modify the multi-sensing pixel array taught by Tan to include comprising a main optical stack, an imaging stack, and an illumination stack taught by Chen with a reasonable expectation of success. The reasoning for this is integrating main optical stack, an sensing pixels and the active illuminator predictably to reduce the size of the optical device.
Regarding claim 9, Tan as modified above teaches the multi-sensing pixel array as recited in claim 8, wherein the main optical stack comprises a main lens (Tan; Fig. 1, [0016], optical system includes semiconductor laser array 110 and a suitable lens or other optics system 130; Fig. 7A, [0040], one or more photodetector devices 702 are integrated with semiconductor laser 701 of laser array 700. Thus, photodetectors 702 can be fabricated along with lasers 701, e.g., part of the same semiconductor die, or can be integrated with the various lasers into a single composite device).
Regarding claim 10, Tan as modified above teaches the multi-sensing pixel array as recited in claim 8,
Tan does not teach,
wherein the imaging stack comprises microlenses, a pixel array comprising RGB pixels, and ToF pixels.
Malinovich teaches in Fig. 1, page 9, line 16, IR-RBG photo surface 20 used in a 3D picture camera having a tiling configuration of IR pixels 21 (distance sensing pixels are IR pixels (page 4, line 16)), R pixels 22, G pixels 23 and B pixels 24 (RGB pixels are imaging pixels); Fig. 2, page 10, line 11, IR-RBG photosurface 30 having a tiling pattern of pixels 21-24 identical to the tiling pattern of pixels 21-24 in photosurface 20 which comprising in addition, an array of circular microlenses 32. Each microlens 32 is coupled to a different one of pixels 21-24 and collects light and directs the collected light onto a photosensitive region 28 of the pixel to which it is coupled.
It would have been obvious to one of ordinary skill in the art prior to the effective filling date of this invention to modify the multi-sensing pixel array taught by Tan to include comprising a main optical stack, an imaging stack, and an illumination stack taught by Chen, include wherein the imaging stack comprises microlenses, a pixel array comprising RGB pixels, and ToF pixels taught by Malinovich with a reasonable expectation of success. The reasoning for this is including microlens arrays in each of the pixels 21-24 for collecting light onto a photosensitive region of the related pixels predictably to focus incoming light onto the active light-sensitive area for increasing sensitivity.
Regarding claim 11, Tan as modified above teaches the multi-sensing pixel array as recited in claim 8, wherein the illumination stack comprises active illuminators (Tan; Fig. 7A, [0040], laser array 700 includes semiconductor lasers 701 and photodetectors 702).
Claim(s) 12 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tan, modified in view of Jenkins (US 20190369213 A1, hereinafter “Jenkins”).
Regarding claim 12, Tan teaches the multi-sensing pixel array as recited in claim 1.
Tan does not teach,
wherein the illumination stack comprises a respective light channel that is arranged for each respective illumination source and that is configured to guide out the illumination.
Jenkins disclosed in Fig. 3-Fig. 4, [0030], the optical device comprises a respective 3D array of waveguides 37 terminating at multiple points along the hemisphere defined by the curved surface 62 which is arranged for each respective illumination source and that is configured to guide out the illumination.
It would have been obvious to one of ordinary skill in the art prior to the effective filling date of this invention to modify the multi-sensing pixel array taught by Tan to include wherein the illumination stack comprises a respective light channel that is arranged for each respective illumination source and that is configured to guide out the illumination taught by Jenkins with a reasonable expectation of success. The reasoning for this is using waveguide aligned along an imaginary curved surface to guide the respective light source for enhancing beam steering (Jenkins; [0029], [0030]).
Regarding claim 19, Tan teaches the multi-sensing pixel array as recited in claim 1.
Tan does not teach,
in which the coupling of the active illumination with the light channels is performed using a ball lens coupling technology.
Jenkins disclosed in Fig. 3-Fig. 4, [0008]-[0009], an optical system includes an optical body, a plurality of optical waveguides, an imaginary curved surface and an optical element 65 (may comprise a portion of a spherical lens, e.g., ball lens, half ball lens) which couples the optical body and be optically aligned with the plurality of optical waveguides; [0030], the optical device comprises a respective 3D array of waveguides 37 terminating at multiple points along the hemisphere defined by the curved surface 62 which is arranged for each respective illumination source and that is configured to guide out the illumination.
It would have been obvious to one of ordinary skill in the art prior to the effective filling date of this invention to modify the multi-sensing pixel array taught by Tan to include in which the coupling of the active illumination with the light channels is performed using a ball lens coupling technology taught by Jenkins with a reasonable expectation of success. The reasoning for this is using waveguide aligned along an curved surface (such as ball lens) to guide the respective light source for enhancing beam steering (Jenkins; [0008]-[0009], [0029]-[0030]).
Claim(s) 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tan, modified in view of Rezk et al. (US 20200135802 A1, hereinafter “Rezk”).
Regarding claim 13, Tan teaches the multi-sensing pixel array as recited in claim 1.
Tan does not teach,
in which a light channel is configured as a fiber optical light guide with a step-index profile or with a graded-index profile, or as a single-mode optical fiber.
Rezk disclosed in Fig. 1A, [0002], using single mode fiber as guiding channel that guide the emitted/reflected light signal.
It would have been obvious to one of ordinary skill in the art prior to the effective filling date of this invention to modify the multi-sensing pixel array taught by Tan to include in which a light channel is configured as a fiber optical light guide with a step-index profile or with a graded-index profile, or as a single-mode optical fiber taught by Rezk with a reasonable expectation of success. The reasoning for this is using single mode optical fiber as guiding channel for laser signal with advantage of high beam quality, minimal divergence and zero modal dispersion.
Claim(s) 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tan, modified in view of Mishra et al. (US 20160373722 A1, hereinafter “Mishra”).
Regarding claim 15, Tan teaches the multi-sensing pixel array as recited in claim 1.
Tan does not teach,
configured to provide the same field of view for the imaging pixels and the depth sensing pixels.
Mishra disclosed in Fig. 1A, [0016], an imaging device 110 configured to capture both visual imaging data and depth imaging data (e.g., an RGBz camera, or RGBD camera) is aligned to capture imaging data from a scene 120. The imaging data captured by the imaging device 110 includes both RGB imaging data 10A and depth imaging data 10B of the scene 120 from within the field of view of the imaging device 110; [0037], additionally, because the visual imaging data and the depth imaging data from which a steganographic depth image is formed may be captured from the same scene, and from the same field of view, a segmenting algorithm or technique may segment either the visual imaging data or the depth imaging data individually, or both the visual imaging data and the depth imaging data collectively, in order to identify such regions or sectors.
It would have been obvious to one of ordinary skill in the art prior to the effective filling date of this invention to modify the multi-sensing pixel array taught by Tan to include configured to provide the same field of view for the imaging pixels and the depth sensing pixels taught by Mishra with a reasonable expectation of success. The reasoning for this is because the visual imaging data and the depth imaging data from which a steganographic depth image is formed may be captured from the same scene, and from the same field of view, a segmenting algorithm or technique may segment either the visual imaging data or the depth imaging data individually, or both the visual imaging data and the depth imaging data collectively, in order to identify such regions or sectors (Mishra; [0016], [0037]).
Claim(s) 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tan, modified in view of Finkelstein (US 20200135776 A1, hereinafter “Finkelstein”).
Regarding claim 16, Tan teaches the multi-sensing pixel array as recited in claim 1.
Tan does not teach,
in which a pixels and cavities control is provided that is configured to activate only those ToF pixels that are actually needed.
Finkelstein disclosed in paragraph [0063], this multiple photodiodes may be activated together, or only partially such that a subset of the photodiodes are activated. For instance, in response to detecting a relatively strong signal from a target, only one of the four SPADs defining a pixel may be activated so that power consumption of the pixel is reduced by 75%. These multiple photodiodes may be read separately, i.e., their outputs can be sampled using respective electronic devices and/or circuit paths that are not shared by some or all of the other photodiodes between the isolation region.
It would have been obvious to one of ordinary skill in the art prior to the effective filling date of this invention to modify the multi-sensing pixel array taught by Tan to include in which a pixels and cavities control is provided that is configured to activate only those ToF pixels that are actually needed taught by Finkelstein with a reasonable expectation of success. The reasoning for this is in response to detecting a relatively strong signal from a target, only one of the four SPADs defining a pixel may be activated so that power consumption of the pixel is reduced by 75% (Finkelstein; [0063]).
Claim(s) 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tan, modified in view of Yamaguchi et al. (US 20210043687 A1, hereinafter “Yamaguchi”).
Regarding claim 18, Tan teaches the multi-sensing pixel array as recited in claim 1.
Tan does not teach,
wherein the multi-sensing pixel array is implemented according to an organic vertical stacking technology.
Yamaguchi disclosed in Fig. 1, Fig. 2, [0066], a pixel 100 is configured of an organic photoelectric conversion layer (OPC) as photoelectric conversion region and a photodiode (PD) as well as a plurality of pixel transistors; [0080], the organic photoelectric conversion layer 112 absorbs only lights in the visible light region, and generates signal charges corresponding to lights of the respective color components or R, G, B component; [0083], a signal charge corresponding to the IR component light is generated in the photodiode 115.
It would have been obvious to one of ordinary skill in the art prior to the effective filling date of this invention to modify the multi-sensing pixel array taught by Tan to include wherein the multi-sensing pixel array is implemented according to an organic vertical stacking technology taught by Yamaguchi with a reasonable expectation of success. The reasoning for this is using an organic photoelectric conversion layer (OPC) as photoelectric conversion region and a photodiode (PD) as well as a plurality of pixel transistors wherein the organic photoelectric conversion layer 112 absorbs only lights in the visible light region, and generates signal charges corresponding to lights of the respective color components or R, G, B component. Therefore, both visible light and infrared light signals can be detected at the same time capable of generating a high-resolution IR image while keeping high quality of a visible light image (Yamaguchi; [0002]-[0005], [0066], [0080], [0083]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Tan et al. (US 20070205285 A1) disclosed in Fig. 4, [0026], emitter-detector module 37 includes an LED 32 and a photodetector 33 (includes one or more photodiodes) that are mounted on a substrate 31. These components are encapsulated in a clear epoxy layer 35 having a top surface 36 that is molded to provide a lens that images the light from LED 32 that is reflected from code wheel 34 onto photodetector 33.
Luo (US 20030010904 A1) disclosed in Fig. 11, [0062], [0063], ball lens 506 focus light exiting from the end face of a single mode optical fiber 504 onto a reflecting surface 508 then goes to detector.
Finkelstein et al. (US 11598857 B2) disclosed in Fig. 3, column 12, line 22, a detector array 310 including imaging pixels 310i interspersed between ToF pixels 310d, such as indirect ToF (iToF) pixels. Multiple image sensors (e.g., 110i, 210i) and ToF sensors (e.g., 110d, 210d) may define a respective imaging pixels 310i and ToF pixel 310d, respectiviely.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHIA-LING CHEN whose telephone number is (571)272-1047. The examiner can normally be reached Monday thru Friday 8-5 ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Yuqing Xiao can be reached at (571)270-3630. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of publishe`d or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHIA-LING CHEN/Examiner, Art Unit 3645
/YUQING XIAO/Supervisory Patent Examiner, Art Unit 3645
Prosecution Insights