DETAILED ACTION
Claims 1-20 are pending in the application.
Response to Arguments
Applicant’s arguments, see page 8, filed 12/30/2025, with respect to the rejection of pending claims under 35 U.S.C. 102 & 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, new grounds of rejection are made in view of Cho et al.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-9 and 12-17 are rejected under 35 U.S.C. 103 as being unpatentable over Yang et al (US 2023/0048991 A1) in view of Cho et al (US 2021/0352209 A1).
As per claim 1, Yang discloses an image capture device (figs. 2A and 2B, head-mounted device 20 (will be referred to as HMD hereinafter), comprising:
a lens array including a plurality of lenses supported by a lens support structure, wherein the plurality of lenses are arranged to capture light rays from multiple view-points (figs. 2A and 2B, HMD 20, lens group 215);
a sensor to capture light and convert the captured light into data that is used to form an image (fig. 2B, HMD 20, image sensor 214); and
a plurality of apertures positioned between the lenses and the sensor, wherein the plurality of apertures are positioned with respect to the lenses and the sensor (fig. 2B, HMD 20, collimator 213 is comprised of a plurality of holes (i.e. apertures) which allow light to pass through to image sensor 214 with respect to the lens group 215, see para 0055-0058)
Yang fails to teach to only allow rays of light that would have reached a certain reference location spaced from the sensor to pass through the plurality of apertures. However, Cho discloses an image sensor comprising mask patterns 820 wherein patterned regions corresponding to sensing elements having an “open space” and “closed space” which may allow “only certain” light to pass through (Cho, fig. 8, masking pattern 820, para 0115 and 0116).
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the teachings of Yang in view of Cho, as a whole, by incorporating the mask patterns as taught by Cho, into the image sensor as taught by Yang, because doing so would provide a more efficient way of blocking/allowing certain portion(s) of incoming light, thus being able to control incoming light more effectively.
As per claim 2, the combined teachings of Yang in view of Cho, as a whole, further discloses the image capture device of claim 1, wherein sections between the plurality of apertures are positioned to physically block rays of light that would not have reached the certain reference location through the plurality of lenses (Yang, fig. 2B, HMD 20, collimator 213, contains areas that do not have holes which are used to block light from reaching image sensor 214, and Cho, fig. 8, masking pattern 820, para 0115 and 0116).
As per claim 3, the combined teachings of Yang in view of Cho, as a whole, further discloses the image capture device of claim 1, wherein the plurality of lenses spatially multiplex a spatio-angular light field impinging on the sensor onto different regions of the sensor (Yang, figs. 2A and 2B, HMD 20, is a light field camera wherein light is impinging on the image sensor 214 in different regions of the sensor, and Cho, fig. 8, masking pattern 820, para 0115 and 0116).
As per claim 4, the combined teachings of Yang in view of Cho, as a whole, further discloses the image capture device of claim 1, wherein the plurality of lenses selectively pick off incoming light rays that converge to the certain reference location (Yang, figs. 2A and 2B, HMD 20, lens group 215, picks up incoming light rays, collimator 213 converges the light rays onto image sensor 214, para 0055-0057, and Cho, fig. 8, masking pattern 820, para 0115 and 0116).
As per claim 5, the combined teachings of Yang in view of Cho, as a whole, further discloses the image capture device of claim 1, wherein each of the plurality of lenses has a respective pupil that only accepts rays within a predefined angular range for a corresponding position on the sensor (fig. 1C, HMD 10, lens group 115 has a pupil, which corresponds to image sensor 114).
As per claim 6, the combined teachings of Yang in view of Cho, as a whole, further discloses the image capture device of claim 5, wherein prescriptions, sizes, and locations of the pupils are jointly optimized for a geometry of the sensor and the certain reference location (Yang, fig. 1C, HMD 10, lens group 115 has a pupil, which corresponds to image sensor 114 and a specific location, i.e. reference location).
As per claim 7, the combined teachings of Yang in view of Cho, as a whole, further discloses the image capture device of claim 1, wherein the certain reference location comprises a target center of perspective of a user of the image capture device (Yang, fig. 1A and 1C, HMD 10, lens group 115, is positioned center to the user of HMD 10).
As per claim 8, the combined teachings of Yang in view of Cho, as a whole, further discloses the image capture device of claim 1, wherein the image capture device is to be mounted on a front side of a head-mountable display to capture images of an environment in front of the head-mountable display and wherein the certain reference location comprises a virtual eye position behind the head-mountable display (Yang, fig. 2A, HMD 20, first light field camera 210, second light field camera 220).
As per claim 9, the combined teachings of Yang in view of Cho, as a whole, further discloses a head-mounted display (Yang, fig. 2A, HMD 20), comprising:
a chassis having a front side and a back side (Yang, fig. 2A, HMD 20, supporting structure 250 (i.e. chassis) has a front side and a back side); and
an image capture device mounted to the front side of the chassis (Yang, fig. 2A, HMD 20, first light field camera 210, second light field camera 220), the image capture device including:
a lens array including a plurality of lenses supported by a lens support structure, wherein the plurality of lenses are arranged to capture light rays from multiple view-points;
a sensor to capture light and convert the captured light into data that is used to form an image; and
a plurality of apertures positioned between the lenses and the sensor, wherein the plurality of apertures are positioned with respect to the lenses and the sensor to allow only rays of light that would have reached a virtual eye position spaced from the sensor to pass through the plurality of apertures, wherein the virtual eye position is positioned behind the back side of the chassis (claim limitations have been discussed and rejected, see claim 1 above, also virtual eye would be located behind the supporting structure 250 as taught by Yang).
As per claim 12, the combined teachings of Yang in view of Cho, as a whole, further discloses the head-mounted display of claim 9, wherein the lens support structure is to physically block rays of light that would not have reached the virtual eye position through the plurality of lenses (claim limitations have been discussed and rejected, see claim 2 above).
As per claim 13, the combined teachings of Yang in view of Cho, as a whole, further discloses the head-mounted display of claim 9, wherein the plurality of lenses spatially multiplex a spatio-angular light field impinging on the sensor onto different regions of the sensor (claim limitations have been discussed and rejected, see claim 3 above).
As per claim 14, the combined teachings of Yang in view of Cho, as a whole, further discloses the head-mounted display of claim 9, wherein the plurality of lenses selectively pick off incoming light rays that converge to the virtual eye position (claim limitations have been discussed and rejected, see claim 4 above).
As per claim 15, the combined teachings of Yang in view of Cho, as a whole, further discloses the head-mounted display of claim 9, wherein each of the plurality of lenses has a respective pupil that only accepts rays within a predefined angular range for a corresponding position on the sensor (claim limitations have been discussed and rejected, see claim 5 above).
As per claim 16, the combined teachings of Yang in view of Cho, as a whole, further discloses the head-mounted display of claim 15, wherein prescriptions, sizes, and locations of the pupils are jointly optimized for a geometry of the sensor and the virtual eye position (claim limitations have been discussed and rejected, see claim 6 above).
As per claim 17, the combined teachings of Yang in view of Cho, as a whole, further discloses the head-mounted display of claim 9, wherein the virtual eye position comprises a target center of perspective of a user of the head-mounted display (claim limitations have been discussed and rejected, see claim 7 above).
Claims 10, 11, and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Yang et al (US 2023/0048991 A1) and Cho et al (US 2021/0352209 A1), in further view of Zhang et al (US 2023/0252687 A1).
As per claim 10, Yang in view of Cho, as a whole, further discloses the head-mounted display of claim 9, further comprising:
a processor (Yang, fig. 1G, HMD 10, central processing unit 190); and
cause the processor to:
having a plurality of sub-aperture views of a scene, wherein the plurality of sub-aperture views comprise views of light that would have reached the virtual eye position that is spaced from a sensor (Yang, figs. 1A, 1G, and 8, light field camera 810 shows the ability for HMD 10 to be comprised of multiple lens subgroups including a first lines subgroup 816, second lens subgroup 817, and third lens subgroup 818 which are spaced from the image sensor 114 wherein light would reach the “virtual eye” behind HMD 10, para 0069);
apply a reconstruction algorithm on the plurality of sub-aperture views (Yang, fig. 1G, HMD 10, image processing unit 180, para 0054).
Yang in view of Cho, as a whole, fails to teach a memory on which is stored machine-readable instructions that when executed by the processor, access raw sensor data and capture the raw sensor data, apply gradient domain image stitching to the plurality of sub-apertures following the application of the reconstruction algorithm to generate a stitched image, wherein the stitched image is to accurately reproduce the image at a perspective that matches the virtual eye position. However, Zhang discloses an image capture and processing system 100/200 comprising ROM 145 and RAM 140 to store instructions to be executed by a processor, image sensor 130 having the ability to access raw pixel data, and image processor 150 in which a gradient can be applied to images being captured, and the ability to combine images to produce an image on the display (should be noted that image capture and processing system 100/200 may be an HMD device, figs. 1 and 2, image capture and processing system 100/200, image sensor 130, image processor 150, ROM 145, RAM 140, I/O 160, para 0022, 0053, 0063 and 0064).
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the teachings of Yang and Cho, in further view of Zhang, as a whole, by incorporating the functionality of processing images as taught by Zhang, into the HMD device as taught by Yang and Cho, because doing so would provide a more efficient way of capturing and processing images, thus enhancing the output of the images captured.
As per claim 11, the combined teachings of Yang and Cho, in further view of Zhang, as a whole, further discloses the head-mounted display of claim 10, wherein the raw sensor data was captured by the image capture device (Zhang, figs 1 and 2, image capture and processing system 100/200, image sensor 130, para 0164), the head-mounted display having a display positioned on the back side of the chassis, the instructions further causing the processor to:
cause the stitched image to be displayed on the display of the head-mounted display (Yang, fig. 1A, HMD 10, images captured are combined by image processing unit 180 and displayed on light field display 130/140).
As per claim 18, the combined teachings of Yang and Cho, in further view of Zhang, as a whole, further discloses a method comprising:
accessing, by a processor, raw sensor data having a plurality of sub-aperture views of a scene, wherein the plurality of sub-aperture views comprise only views of light that would have reached a virtual eye position that is spaced from a sensor that captured the raw sensor data;
applying, by the processor, a reconstruction algorithm on the plurality of sub-aperture views; and
applying, by the processor, gradient domain image stitching to the plurality of sub-apertures following application of the reconstruction algorithm to generate a stitched image, wherein the stitched image is to accurately reproduce the image at a perspective that matches the virtual eye position (claim limitations have been discussed and rejected, see claim 10 above).
As per claim 19, the combined teachings of Yang and Cho, in further view of Zhang, as a whole, further discloses the method of claim 18, wherein the raw sensor data was captured by an image capture device positioned on a front side of a head-mounted display, the head-mounted display having a display positioned on a back side of the head-mounted display, the method further comprising:
causing the stitched image to be displayed on the display of the head-mounted display (claim limitations have been discussed and rejected, see claim 11 above).
As per claim 20, the combined teachings of Yang and Cho, in further view of Zhang, as a whole, further discloses the method of claim 19, wherein the image capture device comprises:
a lens array including a plurality of lenses supported by a lens support structure, wherein the plurality of lenses are arranged to capture light rays from multiple view-points;
a sensor to capture light and convert the captured light into data that is used to form an image; and
a plurality of apertures positioned between the lenses and the sensor, wherein the plurality of apertures are positioned with respect to the lenses and the sensor to allow only rays of light that would have reached a virtual eye position spaced from the sensor to pass through the plurality of apertures, wherein the virtual eye position is positioned behind the back side of the image capture device (claim limitations have been discussed and rejected, see claims 1 and 9 above).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOHN H MOREHEAD III whose telephone number is (571)270-3845. The examiner can normally be reached M - F 0930-1800 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Twyler Haskins can be reached at (571) 272-7406. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JOHN H MOREHEAD III/Examiner, Art Unit 2639
/TWYLER L HASKINS/Supervisory Patent Examiner, Art Unit 2639
Prosecution Insights