DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The Amendment filed July 30th 2026 has been entered. Claims 1-6, 8-13, and 15-22 are pending in the application. Claims 21 and 22 are newly added. Applicant’s amendments to the Claims 1, 5-6, 8, 12-13, and 15-20 have overcome the rejections previously set forth in the Non-Final Office Action mailed April 30th 2026. A second search has been performed to address the material amended in the aforementioned claims. Newly found reference Baumheinrich (WO 2020157149 A1) was used for the amended limitations and newly added claims.
Response to Arguments
The Examiner appreciates the Applicant’s thorough review of the previous action. Applicant’s arguments with respect to claims 1-6, 8-13, and 15-22 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3, 4, 5, 8, 10, 11, 12, 15, 17, 18, 19, 21, and 22 are rejected under 35 U.S.C 103 as being unpatentable over Ninan (US 20210329316 A1) in view of Yeoh (US 20180275410 A1), and Baumheinrich (WO 2020157149 A1).
Regarding claim 1:
Ninan teaches:
A machine-implemented method, comprising:
capturing image data using a camera (Ninan: HDR source images as described herein can be captured/generated with one or more camera systems deployed in one or more spatial environments [0074]) of an eXtended Reality (XR) system (Ninan: mechanisms as described herein form a part of a media processing system, including but not limited to any of: […], virtual reality system, augmented reality system [0046]);
estimating a dark adaptation level of at least one of a user (Ninan: Techniques as described herein can be implemented to predict the viewer's light adaptation level (or a light level to which the viewer is adapted) and emulate the natural vision process in the process of rendering display mapped video content display mapped from foviated video content [0106]) based on the image data (Ninan: Image metadata such as DM metadata can be used to specify or influence how remapping curves used in DM operations [0106]);
selecting a rendering configuration based on the estimated dark adaptation level (Ninan: the display mapped image is to be remapped from a foviated image based at least in part on the viewer's light adaptation level predicated for the time point based on the light adaptation curve [0116]; see Note 1A);
generating an XR display using the selected rendering configuration (Ninan: The image rendering system (324-1) may be used to support […] virtual reality, augmented reality [0158]); and
displaying the XR display to the user (Ninan: A display mapped image adapted from a foviated image based on spatially differentiated DM metadata can be rendered on any target display in a wide variety of target displays [0044]).
Note 1A: Ninan teaches that: “Techniques as described herein can be implemented to predict the viewer's light adaptation level (or a light level to which the viewer is adapted) and emulate the natural vision process in the process of rendering display mapped video content display mapped from foviated video content.” [0106]. In other words, Ninan adjusts how the display renders images based on the user’s light adaptation level.
Ninan fails to explicitly teach:
selecting a night mode rendering configuration based on the estimated dark adaptation level, the night mode rendering configuration including
RGB color for content displayed in a foveal gaze region identified from eye tracking data and
light corresponding to a peak light sensitivity of rod photoreceptors of the at least one eye for content displayed in a peripheral region outside the foveal gaze region;
Yeoh teaches:
selecting a night mode rendering configuration (Yeoh: the display system may be configured to simply associate particular measured pupil areas with particular resolution levels or adjustments [0246]; see also Note 1B) based on the estimated dark adaptation level (Yeoh: the display system may be configured to determine whether the user's eyes are in a photopic, mesotopic, or scotopic vision mode based upon the determined pupil area [0246]), the night mode rendering configuration including
RGB color for content displayed in a foveal gaze region identified from eye tracking data (Yeoh: The image source 23002 can be configured to provide the high-FOV low-resolution image stream in the first set of wavelength ranges, […] For example, the first set of wavelength ranges can correspond to the red, green, and blue (RGB) colors. [0456]) and
light corresponding to another color displayed in a peripheral region outside the foveal gaze region (Yeoh: The image source 23002 can be configured to provide the […] low-FOV high-resolution image stream in the second set of wavelength ranges. […] the second set of wavelength ranges can correspond to the RGB colors in a different hue than that of the first set of wavelength ranges. [0456])
Note 1B: The specification of the present application describes the night mode rendering configuration: “In a night mode, rendering settings are adjusted such as spatial resolution, temporal resolution, color mode, and brightness based on ambient light levels to align with the user's level of dark adaptation” [0031].
Similarly, Yeoh teaches that: “adjustments to spatial resolution, color depth, and/or light intensity resolution are preferably tied to the mode of vision (photopic, mesotopic, or scotopic vision) active at a given time” [0257]. Note that photopic, mesotopic, and scotopic vision are known in the art to be phases the eye changes between based on light conditions, which is acknowledged in the specification of the present application in paragraphs [0073 – 0076].
The specification of the present application further teaches: “In some examples, an eye-tracking-based adaptation estimation method is used to estimate the user's eye dark adaptation level” [0034].
Yeoh also teaches tracking the user’s eye to determine the pupil’s adaptation to light: “the display system may be configured to determine whether the user's eyes are in a photopic, mesotopic, or scotopic vision mode based upon the determined pupil area” [0246].
Therefore, when the teachings of Yeoh are combined with the teachings of Ninan, it would be obvious to one of ordinary skill in the art to select a night mode rendering configuration based on the estimated dark adaptation level.
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Yeoh with Ninan. Selecting a night mode rendering configuration based on the estimated dark adaptation level, as in Yeoh, would benefit the Ninan teachings by improving the user’s vision while improving realism: “These elements 50, 40 are “virtual” in that they do not exist in the real world. Because the human visual perception system is complex, it is challenging to produce AR technology that facilitates a comfortable, natural-feeling, rich presentation of virtual image elements amongst other virtual or real-world imagery elements.” (Yeoh, [0005]).
Ninan in view of Yeoh fails to explicitly teach:
selecting a night mode rendering configuration based on the estimated dark adaptation level, the night mode rendering configuration including
RGB color for content displayed in a foveal gaze region identified from eye tracking data and
light corresponding to a peak light sensitivity of rod photoreceptors of the at least one eye for content displayed in a peripheral region outside the foveal gaze region;
Baumheinrich teaches:
a rendering configuration including
RGB color for content displayed in a foveal gaze region identified from eye tracking data (Baumheinrich: A high resolution for all primary colors is therefore particularly necessary in the center, paragraph spanning Pg. 80-81) and
light corresponding to a peak light sensitivity of rod photoreceptors of the at least one eye for content displayed in a peripheral region outside the foveal gaze region (Baumheinrich: At the edge it may be sufficient to work with an emitter adapted to the spectral sensitivity of the rods (max. Sensitivity at 498 nm, see Figure ID and the sensitivity of the eye), paragraph spanning Pg. 80-81);
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Baumheinrich with Ninan in view of Yeoh. Rendering light corresponding to the sensitivity of the rods in a peripheral region versus a foveal region, as in Baumheinrich, would benefit the Ninan and Yeoh teachings by enabling the system to reduce an amount of power used to display the content on the electronic display without being noticeable to the person viewing the electronic display: “the color rendering can also be greatly reduced and, in extreme cases, only limited to green light, since the color perception of the eye is also severely restricted in the peripheral areas”, (Baumheinrich: Pg. 55, par. 6)
Regarding claim 3:
Ninan in view of Yeoh and Baumheinrich teaches:
The machine-implemented method of claim 1 (as shown above), wherein estimating the dark adaptation level comprises analyzing eye tracking data (Yeoh: In yet other embodiments, the amount of light reaching the retina may be determined by imaging the eye of the user to determine pupil size. Because pupil size is related to the amount of light reaching the retina, determining pupil size allows the amount of light reaching the retina to be extrapolated. [0123]).
Regarding claim 4:
Ninan in view of Yeoh and Baumheinrich teaches:
The machine-implemented method of claim 1 (as shown above), wherein selecting the rendering configuration comprises selecting peripheral rendering settings (Ninan: A peripheral-vision DM metadata portion may comprise maximum, minimum, and/or average luminance levels or luma values in a peripheral-vision region in the viewer's vision field [0100]) and foveal rendering settings (Ninan: a focal-vision DM metadata portion may comprise maximum, minimum, and/or average luminance levels or luma values in a focal-vision region in the viewer's vision field [0100]).
Regarding claim 5:
Ninan in view of Yeoh and Baumheinrich teaches:
The machine-implemented method of claim 1 (as shown above), wherein the rendering configuration includes a lower spatial resolution in the peripheral region versus the foveal gaze region (Yeoh: Preferably, as discussed herein, the high-resolution portion maps to the foveal vision region of the user's eyes while the low resolution portion maps to the peripheral vision region of the user's eyes. [0267]).
Regarding claim 8:
Claim 8 is substantially similar to claim 1, and is therefore rejected for similar reasons. Claim 8 contains the following notable differences:
Claim 8 claims a machine instead of a machine-implemented method. Ninan teaches a machine: “Computer system 500 may implement the techniques described herein using customized hard-wired logic, one or more ASICs or FPGAs, firmware and/or program logic which in combination with the computer system causes or programs computer system 500 to be a special-purpose machine.” (Ninan, [0210])
Regarding claim 10:
Claim 10 is substantially similar to claim 3, and is therefore rejected for similar reasons. Claim 10 contains the following notable differences:
Claim 10 claims a machine instead of a machine-implemented method. In the rejection of claim 8, it was shown that Ninan teaches a machine.
Regarding claim 11:
Claim 11 is substantially similar to claim 4, and is therefore rejected for similar reasons. Claim 11 contains the following notable differences:
Claim 11 claims a machine instead of a machine-implemented method. In the rejection of claim 8, it was shown that Ninan teaches a machine.
Regarding claim 12:
Claim 12 is substantially similar to claim 5, and is therefore rejected for similar reasons. Claim 12 contains the following notable differences:
Claim 12 claims a machine instead of a machine-implemented method. In the rejection of claim 8, it was shown that Ninan teaches a machine.
Regarding claim 15:
Claim 15 is substantially similar to claim 1, and is therefore rejected for similar reasons. Claim 15 contains the following notable differences:
Claim 15 claims a machine-storage medium instead of a machine-implemented method. Ninan teaches a machine-storage medium: “In an embodiment, a non-transitory computer readable storage medium stores software instructions, which when executed by one or more processors cause performance of a method as described herein.” (Ninan, [0202])
Regarding claim 17:
Claim 17 is substantially similar to claim 3, and is therefore rejected for similar reasons. Claim 17 contains the following notable differences:
Claim 17 claims a machine-storage medium instead of a machine-implemented method. In the rejection of claim 15, it was shown that Ninan teaches a machine-storage medium.
Regarding claim 18:
Claim 18 is substantially similar to claim 4, and is therefore rejected for similar reasons. Claim 18 contains the following notable differences:
Claim 18 claims a machine-storage medium instead of a machine-implemented method. In the rejection of claim 15, it was shown that Ninan teaches a machine-storage medium.
Regarding claim 19:
Claim 19 is substantially similar to claim 5, and is therefore rejected for similar reasons. Claim 19 contains the following notable differences:
Claim 19 claims a machine-storage medium instead of a machine-implemented method. In the rejection of claim 15, it was shown that Ninan teaches a machine-storage medium.
Regarding claim 21:
Ninan in view of Yeoh and Baumheinrich teaches:
The machine-implemented method of claim 1 (as shown above), wherein the light corresponding to a peak light sensitivity of rod photoreceptors of the at least one eye comprises green light (Baumheinrich: Here, the color rendering can also be greatly reduced and, in extreme cases, only limited to green light, since the color perception of the eye is also severely restricted in the peripheral areas, Pg. 55, par. 6) centered at a wavelength of 498nm (Baumheinrich: At the edge it may be sufficient to work with an emitter adapted to the spectral sensitivity of the rods (max. Sensitivity at 498 nm, see Figure ID and the sensitivity of the eye), paragraph spanning Pg. 80-81).
Regarding claim 22:
Claim 22 is substantially similar to claim 21, and is therefore rejected for similar reasons. Claim 22 contains the following notable differences:
Claim 22 claims a machine instead of a machine-implemented method. In the rejection of claim 8, it was shown that Ninan teaches a machine.
Claims 2, 9, and 16 are rejected under 35 U.S.C 103 as being unpatentable over Ninan (US 20210329316 A1) in view of Yeoh (US 20180275410 A1), Baumheinrich (WO 2020157149 A1), and Luidolt (NPL: Gaze-Dependent Simulation of Light Perception in Virtual Reality).
Regarding claim 2:
Ninan in view of Yeoh and Baumheinrich teaches:
The machine-implemented method of claim 1 (as shown above), wherein estimating the dark adaptation level comprises estimating an eye response (Yeoh: the display system may be configured to determine whether the user's eyes are in a photopic, mesotopic, or scotopic vision mode based upon the determined pupil area [0246]).
Ninan in view of Yeoh and Baumheinrich fails to explicitly teach:
estimating a rod photoreceptor response.
Luidolt teaches:
wherein estimating the dark adaptation level comprises estimating a rod photoreceptor response (Luidolt: The function σ(Y) denotes the sensitivity of rods with regard to the luminance, Pg. 4, Section 4.1: Temporal Eye Adaptation, par. 4).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Luidolt with Ninan in view of Yeoh and Baumheinrich. Estimating a rod photoreceptor response, as in Luidolt, would benefit the Ninan and Yeoh and Baumheinrich teachings by improving realism of the vision simulation: “The majority of the algorithms aim to compress HDR images as well as possible; however, they might neglect perceptual effects, like rod or cone vision.” (Luidolt, Pg. 3, Section 3.4: Perceptual Tonemappers); “To simulate human vision as accurately as possible, perceptual algorithms need to respond to different lighting conditions in a similar way as the actual human eye” Luidolt, Pg. 4, Section 4: Simulating Human Vision).
Regarding claim 9:
Claim 9 is substantially similar to claim 2, and is therefore rejected for similar reasons. Claim 9 contains the following notable differences:
Claim 9 claims a machine instead of a machine-implemented method. In the rejection of claim 8, it was shown that Ninan teaches a machine.
Regarding claim 16:
Claim 16 is substantially similar to claim 2, and is therefore rejected for similar reasons. Claim 16 contains the following notable differences:
Claim 16 claims a machine-storage medium instead of a machine-implemented method. In the rejection of claim 15, it was shown that Ninan teaches a machine-storage medium.
Claims 6, 13, and 20 are rejected under 35 U.S.C 103 as being unpatentable over Ninan (US 20210329316 A1) in view of Yeoh (US 20180275410 A1), Baumheinrich (WO 2020157149 A1), and Blackmon (US20170169602A1).
Regarding claim 6:
Ninan in view of Yeoh and Baumheinrich teaches:
The machine-implemented method of claim 1 (as shown above),
Ninan in view of Yeoh and Baumheinrich fails to explicitly teach:
wherein the rendering configuration includes a lower temporal resolution in a peripheral region versus a foveal region
Blackmon teaches:
wherein the rendering configuration includes a lower temporal resolution in a peripheral region versus the foveal gaze region (Blackmon: in other examples, the ray tracing logic may render the region of interest of the images at a first frame rate, and the rasterisation logic may render the rasterisation region of the images at a second frame rate, where the first frame rate is different to the second frame rate. The first frame rate could be higher or lower than the second frame rate. [0108]; see Note 6A).
Note 6A: Blackmon teaches: “A region of interest may correspond to a foveal region of the image” [0010]. In other words, Blackmon teaches that a foveal region may be rendered with a higher frame rate (i.e., temporal resolution) than the rest of the image (the peripheral region).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Blackmon with Ninan in view of Yeoh and Baumheinrich. Including a lower temporal resolution in a peripheral region versus a foveal region, as in Blackmon, would benefit the Ninan in view of Yeoh and Baumheinrich teachings by enabling higher resolutions on regions the user is looking at: “since human visual acuity decreases rapidly away from the foveal region, high image quality is maintained for the image as perceived by a user who directs their gaze at the centre of the foveal region 104.” (Blackmon, [0006]).
Regarding claim 13:
Claim 13 is substantially similar to claim 6, and is therefore rejected for similar reasons. Claim 13 contains the following notable differences:
Claim 13 claims a machine instead of a machine-implemented method. In the rejection of claim 8, it was shown that Ninan teaches a machine.
Regarding claim 20:
Claim 20 is substantially similar to claim 6, and is therefore rejected for similar reasons. Claim 20 contains the following notable differences:
Claim 20 claims a machine-storage medium instead of a machine-implemented method. In the rejection of claim 15, it was shown that Ninan teaches a machine-storage medium.
Conclusion
The Examiner identified potential limitation(s) in the specification that would overcome the prior art rejections under 103 if amended into the claims. Note that in such a situation, further search and consideration would be required:
The specification of the present application recites: “In operation 1402, the night mode component 728 provides at least one rendering mode such as, but not limited to: 1) mode using wavelengths above 640nm; 2) mode using 498nm wavelength light; and 3) dim RGB color mode. For example, the night mode component 728 makes these three rendering modes available to select based on the application and whether disrupting rods' night vision is acceptable” [0125]. While the cited references render obvious a 498nm mode, the cited prior art in this action does not teach a mode using wavelengths above 640nm or a dim RGB mode that is available based on the application or “whether disrupting rods' night vision is acceptable”.
“In operation 1008, the night mode component 728 estimates a dark adaptation level based on at least one eye 724 of the user 704 based on whether the flickering is noticed. For example, the night mode component 728 makes this estimation based on the analysis done in operation 1006 of whether the user's eye 724 noticed the low-light flickering stimulus. If the analysis determines that the flickering was noticed by the eye 724, then the night mode component 728 estimates that the eye 724 has achieved a higher level of dark adaptation. This is because the photoreceptors of the eye were able to detect the low-light signal. Conversely, if the analysis in 1006 found that the flickering was not noticed, then the night mode component 728 estimates that the eye 724 has a lower level of dark adaptation” as in [0104] of the specification of the present application.
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Li (US 20220011858 A1) teaches: a rendering configuration including light corresponding to a peak light sensitivity of rod photoreceptors of the at least one eye for content displayed in a peripheral region outside the foveal gaze region (Li: For example, the color of the peripheral area may be green-shifted to reduce a power usage of the electronic display [0067]);
The Examiner notes that while Li does not disclose 498 nm light, 498 nm light is visually blue-green, as disclosed by Wikipedia (NPL: Scotopic vision).
Applicant's amendment necessitated the new grounds of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to VINCENT ALEXANDER PROVIDENCE whose telephone number is (571)270-5765. The examiner can normally be reached Monday-Thursday 8:30-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, King Poon can be reached at (571)270-0728. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/VINCENT ALEXANDER PROVIDENCE/Examiner, Art Unit 2617 /KING Y POON/Supervisory Patent Examiner, Art Unit 2617
