DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Wang (US2022/0078318) in view of Venkataraman (US2011/0080487) and further in view of Hori (US2002/0060743).
Regarding claim 1, Wang discloses an apparatus comprising:
at least one image sensor including a plurality of pixel arrays (Fig. 2A-3: 204a-b; Fig. 4A-4B: 401a-c or 402a-402c), each of the pixel arrays being associated, respectively, with a different one of a plurality of optical channels configured for detection of incoming light rays of a particular wavelength or a particular range of wavelengths centered on the particular wavelength (Abstract; [0004-006; 0010-0012; 0026; 0030-0033]: metalens 202 may split, diffract and/or focus the different spectrums of light onto the each of SWIR, LWIR, R,G, B pixel array 204a-204b, sub pixel array 401a-401c and 402a-402c);
a plurality of metalenses, each of which is disposed, respectively, in a different one of the plurality of optical channels and is configured, respectively, to focus incoming light rays onto a different one of the pixel arrays ([0004-0012]: Fig. 2A-4B: metalens 202 and pixel array 204a-204b and sub pixel array 401a-401c; [0033]); and
readout and processing circuitry operable to read out signals from the plurality of pixel arrays and to generate a respective image for each of the optical channels (Fig. 5: See readout 504a-504b; [0003; 0005; 0025; 0034-0036]: see readout circuitry 504a-b, I/O 507 and image processing and fusion function) and process the images ([0003-0005; 0028; 0037]; claims 5, 11 and 20).
However, Wang fails to explicitly disclose to “to readout lower resolution images and generate a respective lower-resolution image for each of the optical channels and to obtain a higher-resolution image.
In an analogous of art, Venkataraman teaches a super-resolution module 526 to obtain a high resolution synthesized image 422 from low resolution images ([0146]). In light of the teaching from Venkataraman, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include the super-resolution module Venkataraman. The modification thus provide a means to obtain high resolution synthesized image from low resolution images (Venkataraman: [0146]).
However, Wang in view of Venkataraman fails to teach: to process images to “obtain a higher-resolution monochromatic image”.
In an analogous of art, Hori teaches an image processing apparatus that generates a single monochromatic image of an object of a higher resolution by interpolating adjacent pixel data from a plurality of image data having different relative positions. The image is synthesized by interpolation, by filling, in a single image data, the pixels having no pixel data with corresponding pixel data of a single image data at a different position. This interpolation requires no special calculation which takes each of the colors RGB into consideration ([0023-0032]). Hori further teaches the image processing apparatus that acquires a monochromatic image of high resolution within a short time by using the pixel shift method for a color imaging device (abstract; [0012-0013]). In light of the teaching from Hori, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include image processing apparatus of Hori. The modification thus allows an image processing apparatus to acquire a monochromatic image of high resolution within a short time by using the pixel shift method for a color imaging device (Hori: abstract; [0023-0032]).
Regarding claim 2, Wang discloses the apparatus of claim 1 wherein each of the plurality of metalenses is configured to focus incoming light rays of the particular wavelength, or falling within the particular range of wavelengths, onto a respective one of the pixel arrays ([0004-0006; 0010-0012; 0031-0033]).
Regarding claim 3, Wang discloses the apparatus of claim 1 wherein each of the plurality of optical channels includes a respective optical filter ([0010; 0031-0033]; See ).
Regarding claim 4, Wang discloses the apparatus of claim 3 wherein each optical filter is configured to pass light having the particular wavelength or falling within the particular range of wavelengths ([0010; 0031-0033]: metalens 202 separates visible light of 400nm-700nm and NIR of 700nm-900nm; See also RGB pixel array 204a, separate R/G/B subarrays 401a-401c or multiple pixels arrays 402a-402c that may each sense R,G,B, NIR, SWIR or LWIR).
Regarding claim 5, Wang discloses the apparatus of claim 3, wherein each of the optical filters is disposed between the image sensor and a different respective one of the metalenses (Fig. 2A-4B: See locations of RGB pixel array 204a, R/G/B subarrays 401a-401c or multiple pixels arrays 402a-402c with R,G,B, NIR, SWIR or LWIR light and the metalens 202 above the pixel/sub-pixel arrays).
Regarding claim 6, Wang discloses the apparatus of claim 3, wherein each of the optical filters is disposed over a different respective one of the metalenses ([0027; 0030-0032]: metalens 202 separates visible light and NIR light by focusing visible light on to an RGB pixel array 204a and NIR onto a SPAD/APD pixel array 204b. Therefore, each of the optical filters is inherently disposed over a different respective one of the metalenses).
Regarding claim 7, Wang in view of Venkataraman and further in view of Hori discloses the apparatus of claim 1, wherein each of the pixel arrays is operable to acquire an image of a scene, and wherein there is a sub-pixel shift in the image acquired by a first one of the pixel arrays relative to the image acquired by a second one of the pixel arrays (Wang teaches image sensor with pixel and sub-pixels arrays. Abstract; [0012-0013]: Hori teaches method for shifting pixels to obtain higher resolution monochromatic image).
Regarding claim 8, Wang discloses the apparatus of claim 1, wherein the at least one image sensor includes a plurality of image sensors, each of which includes a different respective one of the pixel arrays ([0011-0014]: single chip with multiple camera or pixel arrays).
Regarding claim 9, Wang discloses the apparatus of claim 1, wherein the at least one image sensor is a single image sensor that includes each of the pixel arrays (Fig. 3-4B: See image sensor or multiple camera with pixel arrays 204a-204b and 401a-402c).
Regarding claim 10, Wang in view of Venkataraman and further in view of Hori discloses the apparatus of claim 1, wherein the readout and processing circuitry is operable to process the lower-resolution images (Venkataraman: [0146]) to obtain a higher- resolution monochromatic image using a super-resolution protocol (Hori: abstract; [0012-0013]).
Regarding claim 11, Wang discloses a method comprising:
acquiring, by each of two or more pixel arrays associated with different respective optical channels of an imaging device (Fig. 2A-3: See pixels arrays 204a-b; Fig. 4A-4B: 401a-c or 402a-c with visible/ NIR optical channels of metalens 202),
a respective image of a scene, where each of the images is based on light rays passing through a respective metalens in a respective one of the optical channels ([0004-0012]: Fig. 2A-4B: metalens 202 and pixel array 204a-204b and sub pixel array 401a-401c; [0033]);
reading out, from the pixel arrays, signals representing the acquired images (Fig. 5: See readout 504a-504b; [0003; 0005; 0025; 0034-0036]: see readout circuitry 504a-b) and process the lower-resolution images to obtain higher resolution image ([0003-0005; 0028; 0037]; claims 5, 11 and 20).
However, Wang fails to disclose: “acquired lower-resolution image and using a super-resolution protocol to obtain a higher-resolution image of the scene based on the lower-resolution images”.
In an analogous of art, Venkataraman teaches a super-resolution module 526 to obtain a high resolution synthesized image 422 from low resolution images ([0146]). In light of the teaching from Venkataraman, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include the super-resolution module Venkataraman. The modification thus provide a means to obtain high resolution synthesized image from low resolution images (Venkataraman: [0146]).
However, Wang in view of Venkataraman fails to disclose: “higher-resolution monochromatic image”.
In an analogous of art, Hori teaches an image processing apparatus that generates a single monochromatic image of an object of a higher resolution by interpolating adjacent pixel data from a plurality of image data having different relative positions. The image is synthesized by interpolation, by filling, in a single image data, the pixels having no pixel data with corresponding pixel data of a single image data at a different position. This interpolation requires no special calculation which takes each of the colors RGB into consideration. ([0023-0032]). Hori further teaches the image processing apparatus that acquires a monochromatic image of high resolution within a short time by using the pixel shift method for a color imaging device (abstract; [0012-0013]). In light of the teaching from Hori, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include image processing apparatus of Hori. The modification thus allows an image processing apparatus to acquire a monochromatic image of high resolution within a short time by using the pixel shift method for a color imaging device (Hori: abstract; [0023-0032]).
Regarding claim 12, Wang in view of Venkataraman and further in view of Hori discloses the method of claim 11 including displaying the higher-resolution image on a display screen of a computing device (Abstract; [0003; 0024; 0039]: Wang teaches an on chip image fusion unit with fusing image function and an electronic device 600, computing, PDAs, wireless phone with a touchscreen display; [0146]: Venkataraman teaches a camera system to obtain high resolution image from synthesized low resolution images; Abstract; [0012-0013]: Hori discloses method and apparatus to obtain high resolution monochromatic image. In light of the teaching from Wang, Venkataraman and/or Hori, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to display the fusing image or high resolution monochromatic image on a display of computer, wireless phone or PDA. The modification thus provide a more versatile screen display of a computer, wireless phone or PDA device).
Regarding claim 13, Wang in view of Venkataraman and further in view of Hori discloses the method of claim 11 including displaying the higher-resolution image on a display screen of a smartphone (Abstract; [0003; 0024; 0039]: Wang teaches an on chip image fusion unit with fusing image function and an electronic device 600, computing, PDAs, wireless phone with a touchscreen display; [0146]: Venkataraman teaches a camera system to obtain high resolution image from synthesized low resolution images; Abstract; [0012-0013]: Hori discloses method and apparatus to obtain high resolution monochromatic image. In light of the teaching from Wang, Venkataraman and/or Hori, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to display the fusing image or high resolution monochromatic image on a display of computer, wireless phone or PDA. The modification thus provide a more versatile screen display of a computer, wireless phone or PDA device).
Regarding claim 14, the claim is a method of the apparatus claim 2. Therefore, claim 14 is analyzed and rejected as claim 2.
Regarding claim 15, Wang discloses the method of claim 14 wherein each of the metalenses comprises meta-atoms arranged to resonate at a fixed frequency corresponding to the particular wavelength (Wang: Abstract; [0030]: See nanostructures 207, scatterers and/or nano antennas that focus/split or diffract light or NIR wavelength light into pixels arrays. It is also commonly known in the art that wavelength and frequency are inversely related: v = λf; v = wave velocity; λ = wavelength distance over which a wave shape repeats; f = wave frequency. Thus, the meta-lens, meta-atoms or nanostructures 207 of Wang resonate at a particular NIR or visible light wavelength that is corresponding to a predetermine frequency).
Regarding claim 16, Wang in view of Venkataraman and further in view of Hori discloses the method of claim 11, wherein there is a sub-pixel shift in the lower-resolution image ([0146]: Venkataraman teaches a camera system to obtain high resolution image from synthesized low resolution images; Hori: Abstract; [0012-0013]: Hori discloses method and apparatus to obtain high resolution monochromatic image base on pixel shift and interpolation methods) acquired by a first one of the pixel arrays relative to the lower-resolution image acquired by a second one of the pixel arrays (Wang: Abstract; [003-005; 0024]: Fig. 2A-3: See images output from pixels arrays 204a-b; Fig. 4A-4B: 401a-c or 402a-c for fusion functions/ processes; [0061; 0146]: Venkataraman teaches imagers or pixel arrays to capture low resolution images).
Regarding claim 17, the claim is a method of the apparatus claim 2. Therefore, claim 17 is analyzed and rejected as claim 2.
Regarding claim 18, the claim is a method of the apparatus claim 15. Therefore, claim 18 is analyzed and rejected as claim 15.
Regarding claim 19, the claim is a method of the apparatus claim 2. Therefore, claim 19 is analyzed and rejected as claim 2.
Regarding claim 20, the claim is a method of the apparatus claim 15. Therefore, claim 20 is analyzed and rejected as claim 15.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HUNG H LAM whose telephone number is (571)272-7367. The examiner can normally be reached 9AM-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, TWYLER HASKINS can be reached at (571) 272-7406. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HUNG H LAM/Primary Examiner, Art Unit 2639 09/27/25