DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
2. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Information Disclosure Statement
3. The Information Disclosure Statement filed 23 March 2026 has been fully considered by Examiner. An annotated copy is included herewith.
Response to Arguments
4. Applicant’s arguments, see Remarks, filed 4 June 2026, with respect to the rejections of claims 24 and 25 under 35 U.S.C. § 112(b) have been fully considered and are persuasive. The rejections of claims 24 and 25 under 35 U.S.C. § 112(b) have been withdrawn.
5. Applicant’s arguments, see Remarks, filed 4 June 2026, with respect to the rejections of claims 1-25 under 35 U.S.C. §§ 102 & 103 have been fully considered and are persuasive. Therefore, the rejections have been withdrawn. However, upon further consideration, new grounds of rejection are made in view of newly applied prior art. The new grounds of rejection are necessitated by Applicant’s amendments to the claims. Accordingly, the present Office Action is made final.
Claim Rejections - 35 USC § 103
6. The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
7. Claims 1-6, 11-15, 18, 19, 21-24 and 26 are rejected under 35 U.S.C. 103 as being unpatentable over Price (US-2021/0400185) in view of Gardner (US-2024/0037879).
Regarding claim 1: Price discloses a method (fig 3 of Price) comprising: at an electronic device (fig 2(202) and [0018] of Price – HMD device) having a processor (fig 2(208) and [0023] of Price) and one or more sensors (fig 2(204,224) and [0026] of Price), the one or more sensors comprising an image sensor (fig 2(224) and [0026] of Price): capturing sensor data (fig 3(310) and [0030]-[0032] of Price) corresponding to a physical environment via the one or more sensors ([0031] of Price – ‘the ambient light condition may be determined based on a signal received by an optical sensor, such as optical sensor 204’); determining a characteristic of a light source in the physical environment based on modeling the physical environment based on sensor data captured via the one or more sensors (fig 3 and [0030]-[0031] of Price – ‘responsive to an indication that an ambient light condition in an environment is below a lighting threshold determine an amount of motion’; ‘the ambient light condition may be determined based on a signal received by an optical sensor’); based on the characteristic of the light source in the physical environment, determining a camera parameter for an image captured via the image sensor (fig 3(320) and [0032]-[0033] of Price – ‘automatically select exposure time based on determined motion’ (based on ambient light condition)); and providing passthrough video of the physical environment including the image based on the determined camera parameter (fig 3(340), [0011], and [0049] of Price – ‘external cameras may provide a video passthrough mode’ ([0011]); ‘display captured imagery at the head-mounted display’ ([0049])).
Price does not disclose a 3D pose of a light source in the physical environment.
Gardner discloses a 3D pose of a light source in the physical environment (figs 6A-6B, and [0043]-[0045] of Gardner).
Price and Gardner are analogous art because they are from the same field of endeavor, namely head-mounted virtual reality devices. Before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to determine a 3D pose of a light source in the physical environment, as taught by Gardner. The motivation for doing so would have been to more realistically render the virtual reality space. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Price according to the relied-upon teachings of Gardner to obtain the invention as specified in claim 1.
Regarding claim 2: Price in view of Gardner discloses the method of claim 1 (as rejected above), wherein determining the environment characteristic comprises detection of an environment light source optical characteristic comprising brightness ([0028] of Price – determines amount of signal required based on brightness level), color ([0018] of Price – ‘cameras included in the computing system 200 may be sensitive to various ranges of electromagnetic radiation as preferred by designers of the computing system 104, such as UV light, visible light, near infrared light, or other suitable frequencies’), temporal brightness ([0028] of Price – determines amount of signal required based on changes in brightness level), flicker profile (list recited in the alternative, and other items in list are taught), physical technology ([0018] of Price – different types of cameras and device configurations), spatial emission profile (list recited in the alternative, and other items in list are taught), or light source classification ([0018] and [0022] of Price – different light sources considered).
Regarding claim 3: Price in view of Gardner discloses the method of claim 2 (as rejected above), wherein the environment light source optical characteristics is determined by: analyzing ambient light sensor (ALS) data received over time ([0028] of Price – determines amount of signal required based on changing brightness level); analyzing flicker sensor data received over time (list recited in the alternative, and other items in list are taught); analyzing spatial resolution data ([0047] of Price); analyzing spatial ALS data ([0028] of Price – determines amount of signal required based on brightness level); analyzing ray tracing data ([0022] of Price – time-of-flight methods are ray-based analysis methods); accessing a database of known light sources (list recited in the alternative, and other items in list are taught); accessing smart home light data (list recited in the alternative, and other items in list are taught); display or monitor detection (list recited in the alternative, and other items in list are taught); window detection (list recited in the alternative, and other items in list are taught); accessing weather data (list recited in the alternative, and other items in list are taught); or assessing glare or flare data (list recited in the alternative, and other items in list are taught).
Regarding claim 4: Price in view of Gardner discloses the method of claim 1 (as rejected above), wherein determining the environment characteristic comprises detection of an environment surface optical characteristic comprising color ([0018] of Price – ‘cameras included in the computing system 200 may be sensitive to various ranges of electromagnetic radiation as preferred by designers of the computing system 104, such as UV light, visible light, near infrared light, or other suitable frequencies’), specular characteristic (list recited in the alternative, and other items in list are taught), diffuse characteristic (list recited in the alternative, and other items in list are taught), reflectance characteristic ([0043] of Price – reflectivity of the environment), or transparency characteristic ([0026] of Price – transparency of near eye displays and visor).
Regarding claim 5: Price in view of Gardner discloses the method of claim 4 (as rejected above), wherein the environment surface optical characteristics is determined by: analyzing image data of the physical environment ([0014]-[0015] of Price); analyzing ambient light sensor (ALS) data ([0028] of Price – determines amount of signal required based on brightness level); or analyzing ray tracing data (list recited in the alternative, and other items in list are taught).
Regarding claim 6: Price in view of Gardner discloses the method of claim 1 (as rejected above), wherein modeling the physical environment comprises generating a 3D mapping of the physical environment based on: inertial measurement unit (IMU) data ([0017] of Price); camera image data ([0013]-[0014] of Price); visual inertial odometry (VIO) data (list recited in the alternative, and other items in list are taught); a scene understanding process (list recited in the alternative, and other items in list are taught); or simultaneous localization and mapping (SLAM) (list recited in the alternative, and other items in list are taught).
Regarding claim 11: Price in view of Gardner discloses the method of claim 1 (as rejected above), wherein the passthrough is provided based on a user behavior model, the user behavior model based on: detecting if a user is approximately static, moving, transitioning to a different location, or on a moving platform ([0016]-[0017] of Price – low or high motion conditions).
Regarding claim 12: Price in view of Gardner discloses the method of claim 11 (as rejected above), wherein the user behavior model is based on: historical user pose data; live user pose data; eye tracking; or identifying one or more apps currently executing ([0023]-[0024] of Price – eye tracking, relative movement of HMD (live user pose data); list cited in alternative).
Regarding claim 13: Price in view of Gardner discloses the method of claim 1 (as rejected above), wherein determining the environment characteristic based on modeling the physical environment comprises predicting an optical characteristics of light entering the image sensor at one or more plausible poses in the physical environment ([0018] of Price).
Regarding claim 14: Price in view of Gardner discloses the method of claim 13 (as rejected above), wherein the one or more plausible poses are determined based on a 3D mapping of the physical environment, a user behavior model, or a time horizon ([0016]-[0017] of Price).
Regarding claim 15: Price in view of Gardner discloses the method of claim 13 (as rejected above), wherein predicting the optical characteristics of the light entering the image sensor comprises: ray tracing based on image sensor physical design, focal distance, aperture size, field of view ([0013] of Price), vignetting, responsivity ([0029]-[0030] of Price), spectral quantum efficiency of different color channels, transmittance, sensor timing ([0020] of Price), sensor readout time, or far-field light map modeling; image sensor calibration; occlusion mitigation or hallucination; far field light map; flicker profile blending; image sensor shutter simulation ([0020] of Price); or color spectrum blending using spectral responsivity ([0018]-[0019] of Price – also, list is in the alternative, and at least one item is taught).
Regarding claim 18: Price in view of Gardner discloses the method of claim 1 (as rejected above), wherein determining the camera parameter is based on a mitigation that accounts for unmapped environments (fig 1, [0013], and [0017] of Price – optimizes for areas based on motion, and thus mitigates errors even for unmapped environments).
Regarding claim 19: Price in view of Gardner discloses the method of claim 1 (as rejected above), wherein determining the camera parameters comprises determining one or more zones in which the camera parameter is to be stabilized (fig 1(110-112,110a-112a) and [0013]-[0015] of Price – stabilized for motion, lighting levels, etc., according to zones in which real-world objects and their video representatives can be overlaid and viewed simultaneously).
Regarding claim 21: Price in view of Gardner discloses the method of claim 1 (as rejected above), wherein the camera parameter is an exposure, gain, tone mapping, or color balance ([0015], [0033], and [0043] of Price).
Regarding claim 22: Price in view of Gardner discloses the method of claim 1 (as rejected above), wherein the camera parameter is a noise reduction parameter or a sharpness enhancement parameter ([0016] of Price).
Regarding claim 23: Price in view of Gardner discloses the method of claim 1 (as rejected above), wherein providing the passthrough video is based on further based on a camera characteristic ([0011] of Price).
Regarding claim 24: Price discloses a head-mounted device (HMD) (fig 2(202) and [0018] of Price) comprising: a non-transitory computer-readable storage medium (fig 2(220,222) and [0023] of Price); one or more sensors comprising an image sensor (fig 2(204) and [0018]-[0019] of Price); and one or more processors coupled to the non-transitory computer-readable storage medium (fig 2(208) and [0023] of Price), wherein the non-transitory computer-readable storage medium comprises program instructions that, when executed on the one or more processors, cause the system to perform operations ([0023] of Price – ‘processor 208 is configured to execute instructions stored in the storage, using volatile memory 220 while executing instructions belonging to various programs and non-volatile memory 222 for storage of the programs’) comprising: capturing sensor data (fig 3(310) and [0030]-[0032] of Price) corresponding to a physical environment via the one or more sensors ([0031] of Price – ‘the ambient light condition may be determined based on a signal received by an optical sensor, such as optical sensor 204’); determining a characteristic of a light source in the physical environment based on modeling the physical environment based on sensor data captured via the one or more sensors (fig 3 and [0030] of Price – ‘responsive to an indication that an ambient light condition in an environment is below a lighting threshold determine an amount of motion’); based on the a characteristic of the light source in the physical environment, determining a camera parameter for an image captured via the image sensor (fig 3(320) and [0032]-[0033] of Price – ‘automatically select exposure time based on determined motion’ (based on ambient light condition)); and providing passthrough video of the physical environment including the image based on the determined camera parameter (fig 3(340), [0011], and [0049] of Price – ‘external cameras may provide a video passthrough mode’ ([0011]); ‘display captured imagery at the head-mounted display’ ([0049])).
Price does not disclose a 3D pose of a light source in the physical environment.
Gardner discloses a 3D pose of a light source in the physical environment (figs 6A-6B, and [0043]-[0045] of Gardner).
Price and Gardner are analogous art because they are from the same field of endeavor, namely head-mounted virtual reality devices. Before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to determine a 3D pose of a light source in the physical environment, as taught by Gardner. The motivation for doing so would have been to more realistically render the virtual reality space. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Price according to the relied-upon teachings of Gardner to obtain the invention as specified in claim 24.
Regarding claim 26: Price discloses a method comprising: at an electronic device (fig 2(202) and [0018] of Price) having a processor (fig 2(208) and [0023] of Price) and one or more sensors, the one or more sensors comprising an image sensor (fig 2(204) and [0018]-[0019] of Price): capturing sensor data (fig 3(310) and [0030]-[0032] of Price) corresponding to a physical environment via the one or more sensors ([0031] of Price – ‘the ambient light condition may be determined based on a signal received by an optical sensor, such as optical sensor 204’); determining a mapping of surfaces in the physical environment based on modeling the physical environment based on the sensor data captured via the one or more sensors (fig 1, [0013]-[0014], [0029]-[0030], and [0047] of Price – mapped according to detected objects, movement, lighting, etc.); based on the mapping of the surfaces in the physical environment, determining a camera parameter for an image captured via the image sensor (fig 3(320) and [0032]-[0033] of Price – ‘automatically select exposure time based on determined motion’ (based on ambient light condition)); and providing passthrough video of the physical environment including the image based on the determined camera parameter (fig 3(340), [0011], and [0049] of Price – ‘external cameras may provide a video passthrough mode’ ([0011]); ‘display captured imagery at the head-mounted display’ ([0049])).
Price does not disclose 3D mapping of surfaces in the physical environment.
Gardner discloses a 3D mapping of surfaces in the physical environment (figs 6A-6B, [0045], and [0056] of Gardner).
Price and Gardner are analogous art because they are from the same field of endeavor, namely head-mounted virtual reality devices. Before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to map 3D surfaces in the physical environment, as taught by Gardner. The motivation for doing so would have been to more realistically render the virtual reality space. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Price according to the relied-upon teachings of Gardner to obtain the invention as specified in claim 26.
8. Claims 7-10 are rejected under 35 U.S.C. 103 as being unpatentable over Price (US-2021/0400185) in view of Gardner (US-2024/0037879), and in further view of Knorr (US-2019/0230297).
Regarding claim 7: Price in view of Gardner discloses the method of claim 1 (as rejected above). Price in view of Gardner does not disclose wherein modeling the physical environment comprises, using motion tracking data, to: generate 3D poses of light sources in the physical environment; and generate 3D poses of surfaces in the physical environment.
Knorr discloses wherein modeling the physical environment comprises, using motion tracking data, to: generate 3D poses of light sources in the physical environment; and generate 3D poses of surfaces in the physical environment (figs 4-8, [0082]-[0085], [0193], and [0201] of Knorr – poses of various light source and surfaces in the 3D physical environment).
Price and Knorr are analogous art because they are from the same field of endeavor, namely augmented reality displays. Before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to model the physical environment by using motion tracking data, to: generate 3D poses of light sources in the physical environment; and generate 3D poses of surfaces in the physical environment, as taught by Knorr. The motivation for doing so would have been to provide a more realistically rendered visual experience for the user of the augmented reality display device. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Price further according to the relied-upon teachings of Knorr to obtain the invention as specified in claim 7.
Regarding claim 8: Price in view of Gardner, and in further view of Knorr, disclose the method of claim 7 (as rejected above), wherein the modeling is based on: analyzing ambient light sensor (ALS) data received over time ([0028] of Price – determines amount of signal required based on brightness level); analyzing flicker sensor data received over time (list recited in the alternative, and other items in list are taught); analyzing spatial resolution data ([0047] of Price); analyzing spatial ALS data ([0028] of Price – determines amount of signal required based on brightness level); or analyzing ray tracing data (list recited in the alternative, and other items in list are taught).
Regarding claim 9: Price in view of Gardner, and in further view of Knorr, discloses the method of claim 1 (as rejected above). Price in view of Gardner does not disclose wherein modeling the physical environment comprises: room-based classification of the physical environment or spatial separation classification of the physical environment, or far-field light map modeling.
Knorr discloses wherein modeling the physical environment comprises: room-based classification of the physical environment (figs 6-8, [0119], and [0217] of Knorr) or spatial separation classification of the physical environment (figs 4-5, [0047], [0056], [0191], and [0203] of Price – different distances in the physical environment are classified using a checkered planar surface), or far-field light map modeling (list recited in the alternative, and other items in list are taught).
Price and Knorr are analogous art because they are from the same field of endeavor, namely augmented reality displays. Before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to model the physical environment by: room-based classification of the physical environment or spatial separation classification of the physical environment, or far-field light map modeling, as taught by Knorr. The motivation for doing so would have been to provide a more realistically rendered visual experience for the user of the augmented reality display device. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Price further according to the relied-upon teachings of Knorr to obtain the invention as specified in claim 9.
Regarding claim 10: Price in view of Gardner discloses the method of claim 1 (as rejected above). Price in view of Gardner does not disclose wherein modeling the physical environment is based on: previously-obtained image statistic data corresponding to one or more viewpoints within the physical environment; and scaling historical data based on proportion light in the physical environment corresponding to natural light.
Knorr discloses wherein modeling the physical environment is based on: previously-obtained image statistic data corresponding to one or more viewpoints within the physical environment (figs 4-8, [0131], [0176]-[0177], and [0221]-[0222] of Price – various statistics corresponding to the viewpoints); and scaling historical data based on proportion light in the physical environment corresponding to natural light ([0114], and [0175] of Price – historical data relating to pose determination results, which includes light proportions).
Price and Knorr are analogous art because they are from the same field of endeavor, namely augmented reality displays. Before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to model the physical environment based on: previously-obtained image statistic data corresponding to one or more viewpoints within the physical environment; and scaling historical data based on proportion light in the physical environment corresponding to natural light, as taught by Knorr. The motivation for doing so would have been to provide a more realistically rendered visual experience for the user of the augmented reality display device. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Price further according to the relied-upon teachings of Knorr to obtain the invention as specified in claim 10.
9. Claims 16 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Price (US-2021/0400185) in view of Gardner (US-2024/0037879), and in further view of Feng (US-2023/0388658).
Regarding claim 16: Price in view of Gardner discloses the method of claim 1 (as rejected above), wherein providing the passthrough video comprises accounting for motion blur ([0011] and [0033]-[0034] of Price). Price in view of Gardner does not disclose wherein providing the passthrough video comprises accounting for flicker visibility, noise visibility, dynamic range, and brightness stability.
Feng discloses accounting for flicker visibility ([0063], [0072]-[0074], and [0233] of Feng), noise visibility ([0099], and [0208] of Feng), dynamic range ([0054], [0072]-[0073], and [0099] of Feng), and brightness stability ([0070], and [0099] of Feng).
Price and Feng are analogous art because they are from similar problem solving areas, namely determination of light source data in images. Before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to provide the passthrough video, as taught by Price, by accounting for flicker visibility, noise visibility, dynamic range, and brightness stability, as taught by Feng. The motivation for doing so would have been to provide a more consistent imaging result. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Price further according to the relied-upon teachings of Feng to obtain the invention as specified in claim 16.
Regarding claim 20: Price in view of Gardner discloses the method of claim 1 (as rejected above). Price in view of Gardner does not disclose wherein determining the camera parameters is based on room color temperature, room transition handling, or skin color stabilization.
Feng discloses wherein determining the camera parameters is based on room color temperature (fig 6, [0077], and [0099] of Feng), room transition handling (list recited in the alternative, and other items in list are taught), or skin color stabilization (list recited in the alternative, and other items in list are taught).
Price and Feng are analogous art because they are from similar problem solving areas, namely determination of light source data in images. Before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to determine the camera parameters based on room color temperature, room transition handling, or skin color stabilization, as taught by Feng. The motivation for doing so would have been to provide a more consistent imaging result. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Price further according to the relied-upon teachings of Feng to obtain the invention as specified in claim 20.
10. Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over Price (US-2021/0400185) in view of Gardner (US-2024/0037879), and in further view of Feng (US-2023/0388658) and well-known prior art.
Regarding claim 17: Price in view of Gardner discloses the method of claim 1 (as rejected above). Price in view of Gardner does not disclose wherein determining the camera parameters is based on a cost function-based optimization.
Feng discloses wherein determining the camera parameters is based on an optimization ([0099] of Feng).
Price and Feng are analogous art because they are from similar problem solving areas, namely determination of light source data in images. Before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to determine the camera parameters based on an optimization, as taught by Feng. The motivation for doing so would have been to provide a more efficient imaging result. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Price further according to the relied-upon teachings of Feng.
Price in view of Gardner, and in further view of Feng, does not disclose that the optimization is a cost function-based optimization. However, OFFICIAL NOTICE was taken that a cost function-based optimization is old, well-known and expected in the art in the Office Action of 4 March 2026, and is considered admitted prior art since the official notice was not timely traversed. Before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to have the optimization taught by Feng be specifically a cost function-based optimization, since doing so would further improve the efficiency of the overall imaging and display system. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Price further according to well-known prior art to obtain the invention as specified in claim 17.
Conclusion
11. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to James A Thompson whose telephone number is (571)272-7441. The examiner can normally be reached M-F 8am-6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alicia Harrington can be reached at 571-272-2330. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JAMES A THOMPSON/Primary Examiner, Art Unit 2615
Prosecution Insights