DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the inventor to file provisions of the AIA .
Response to Amendment / Arguments
Applicant’s arguments with respect to the amended claims have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-6, 8, 9, 11-16, 18, 19 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Choi (U.S. Patent No. 11,209,656) in view of Yeoh (U.S. Patent App. Pub. No. 2018/0275410) and Coleman (U.S. Patent App. Pub. No. 2021/0122647 A1).
Regarding claim 1:
Choi teaches: a mixed-reality device (Fig. 1: 120, head mounted display for augmented reality) comprising: an eye tracker configured to determine a position of an eye (Fig. 1: 130, eye tracking unit); a near-eye display (Fig. 1:120, the HMD has a near-eye display); a logic subsystem (Fig. 6: 630, image processor/ Fig. 6 is an image source assembly in a near-eye display system); and a storage subsystem holding instructions executable by the logic subsystem (Fig. 6: 630 and related description, the processor can execute instructions stored in memory) to: …
render a plurality of luminance-adjusted image frames over a window of time by: for each luminance-adjusted frame (see below mapping of the “image frame”, repeat for the next frame(-s)):
generate an image frame including a plurality of pixels, wherein each pixel has a native luminance level (Choi, C6, last paragraph, near eye display can present images; of pixels (C5, third full paragraph), with luminance (see C5, last paragraph, luminance images. Without performing the invention of Choi, there is what applicant terms “native” luminance);
map the position of the eye to the image frame (see Fig. 9: 920, 940, 950, 970 and related descriptions, Fig. 9 being a simplified flowchart for tracking a user’s eye. To quote Choi (regarding Fig. 9: 970, specifically):
“At block 970, the gaze direction of the user's eye may be determined based on, for example, the location of the center of the pupil and the position of the center of the cornea. In some embodiments, the pupillary axis of the use's eye may be determined and may then be used to determine the foveal axis (or line of sight, gaze direction, or visual axis) of the user's eye, for example, based on an angle between the pupillary axis and the foveal axis. Based on the gaze direction (and thus the center of the field of view) of the user's eye and the sensitivity of human eyes at different regions of the retina, the display system may determine the luminance levels for the individual light sources in different display zones of the display system that correspond to different zones on the retina of a user's eye. The individual light sources in the different display zones may then be controlled to emit at the different luminance levels.”
The individual light sources in different display zones corresponds to the image frame (since the image is being generated/displayed by projected light)),
adjust luminance levels of pixels of the image frame as a function of angle relative to the position of the eye mapped to the image frame … to generate the luminance-adjusted image frame (see C4, beginning at L40, whereby Choi describes the relationship between viewing angle of a human eye and luminance levels (i.e., For example, the number of receptors may decrease from approximately 155,000 receptors per square millimeter near the fovea to fewer than 10,000 receptors per square millimeter at a region corresponding to a view angle of 10° or greater with respect to the foveal axis. Therefore, portions of a display panel that are farther away from the center of the field of view of a user may not be very noticeable to the user's eye even if these portions of the display panel have high luminance levels or high light intensities.”. See also definition of “luminance, at C5, last paragraph, which further described relationship between luminance level as a function of angle of view (angle relative to a position of the eye mapped, as mapped above. Luminance levels of pixels (display regions, pixels mapped above) are adjusted as a function of angle (see Fig. 13, claim 4, and above mapping) (*see below discussion re: Yeoh reference, for “rod and cone anatomy”), and
render, via the near-eye display, the luminance-adjusted image frame (Fig. 9: 970 and related description, some of which copied and pasted above. A luminance-adjusted image frame is rendered. Another example: claim 4),
Re: adjust luminance … and based on at least on a rod and cone anatomy of the eye, consider the following. Yeoh teaches that it is known that rods and cones (of a human’s eye) are active at different levels of incident light. “For example, cones are active under relatively bright conditions, while rods are active under relatively low light conditions” (quoting para. 121).
Yeoh also teaches that vision behavior of an eye may be divided into three modes, based on the light conditions (para. 241), and that attenuation (control) of light and luminance levels can be made, based on rod/cone anatomy. See paras. 243-248.
Re: in a transition operating mode where the mixed reality device is preparing for deactivation…
adjust luminance…further based on luminance levels of pixels of a previously-rendered image frame of a plurality of image frames corresponding to the plurality of luminance-adjusted image frames…
wherein the luminance levels of the pixels of the plurality of luminance-adjusted image frames are reduced to a greater degree over the window of time, such that the luminance levels of the pixels of the plurality of luminance-adjusted image frames become dimmer over the window of time, consider the following.
First, *claim interpretation*. Paragraph 68 of Applicant’s specification as filed is what supports this amended subject matter, shown above, which is also the same paragraph that Applicant indicates in its 01/02/2026 Remarks that support can be found. In terms of “transition operating mode where the …device is preparing for deactivation”, it is actually a mode where a user is preparing to deactivate or “or take it off in the near future and desires to maintain dark adaptation in order to see in low-light or dark ambient lighting conditions” (quoting spec, para. 68). The examiner’s claim interpretation is thus that the “transition operating mode” is a mode that the device is in, that a user could be in the process of wanting to deactivate or take it off, and the mode operates on image frames as claimed. Which moves into the rest of the above claim language, regarding luminance levels of previously-rendered image frames, and reduction of luminance levels to a greater degree over a window of time. Applicant’s specification does not have the words “previously-rendered” or even “previously” or “previous”. However, paragraph 68 (which Applicant cites to as support for the amendments), describes frames that “become dimmer over a window of time” as part of the ”transition operating mode”. Dimming frames over a number of frames is how the examiner is interpreting the above claim language, in terms of previously-rendered frames, and reduction of luminance over a window of time. The Examiner’s claim interpretation is a broad, reasonable interpretation consistent with Applicant’s specification as filed.
Now, to the prior art.
Re: in a transition operating mode where the mixed reality device is preparing for deactivation: See Yeoh, paras. 241-45, which teach at least three different operating modes. Yeoh is not limited to how many modes or how the modes are determined, up to and including arbitrary assignment of illumination levels.
See also, e.g. paras. 257 and 258, reproduced in part below:
“the display system may be configured to simply associate particular ranges of ambient illumination levels or pupil size with particular resolutions, whether spatial resolution, color depth, or light intensity resolution. In addition, while the resolution adjustments are preferably tied to three levels of light conditions (corresponding to three modes of vision) as discussed herein, in some embodiments, the resolution adjustments may be tied to two levels of light conditions, or more than three levels of light conditions.” (para. 257). …and “Without being limited by theory, it is believed that the human visual system requires a period of time to adapt to different illumination levels, with that period of time increasing as illumination levels decrease.” (para. 258, emphasis added).
Likewise, Coleman teaches that a light emitting system can also operate in any one of several operational modes. See “Modes of Illumination and/or Irradiation” in Coleman, beginning at para. 155 and extending to para. 156. Here is para. 155 reproduced – which teaches/suggests a “transition operating mode”.
[0155]…. one or modes of illumination and/or irradiation selected from the group: standard occupant; standard non-occupant; predictive; user configurable; socially adaptive; reflective adaptive (adapts to reflectivity of object); ambient light adaptive (adapts to ambient light conditions (sunlight, etc.); energy saving mode-which AVLED (or which bins from a single AVLED best illuminates the space taking into account sunlight and reflections illuminating the environment; multi-fixture network adaptive (which fixture is most efficient at illuminating space) or contribute more/less to light up dark spots; Spatial zone mode, Open loop mode, Color enhancement mode, Luminance/illuminance contrast enhancement mode, High efficiency mode, Sunlight mimicry mode, Specification maintaining mode, Illuminance specification mode, Luminance specification mode, Irradiance specification mode, Radiance specification mode, Relative light output specification mode, Luminance uniformity, mode, Illuminance uniformity mode, Irradiance uniformity mode, Radiance uniformity mode, Color uniformity mode, Spectral uniformity mode, Uniformity mode, Shadow reduction mode, Light reflecting and light emitting object differentiation mode, Predictive illumination mode, Safety and security mode, Environmental monitoring mode, Smoke, heat, or CO detection mode, Tracking and/or Identification mode, Reduced or glare free illumination mode, Reduced light trespass or light trespass free mode, Reduced light pollution or light pollution free mode, Selective warming mode, Socially adaptive mode, Health monitoring mode, Environmental monitoring mode, Entertainment mode, Variable illumination for camera mode, Light field display mode, Light communication mode
Modifying the applied references, in view of Yeoh and Coleman, such to include a transition operating mode, for when a user is preparing to deactivate or take-off the device, such to give a user “a period of time to adapt to different illumination levels” per Yeoh, or any of the non-limiting examples of modes per Coleman, is obvious over the prior art. See MPEP 2143(A). Likewise, a user deciding to operate in any mode before taking off a device is also an intended use of the device, and respectfully does not create a structural difference in the invention.
Re: adjust luminance…further based on luminance levels of pixels of a previously-rendered image frame of a plurality of image frames corresponding to the plurality of luminance-adjusted image frames…
wherein the luminance levels of the pixels of the plurality of luminance-adjusted image frames are reduced to a greater degree over the window of time, such that the luminance levels of the pixels of the plurality of luminance-adjusted image frames become dimmer over the window of time, see Coleman, e.g. paras. 155-56. In terms of a dimming mode, this is taught/suggested by Coleman’s teachings on operating modes of its light emitting device, any one of the above modes, such as: predictive, user configurable, Specification maintaining mode, Illuminance specification mode, Luminance specification mode, Irradiance specification mode, Radiance specification mode, Relative light output specification mode, Selective warming mode, etc. Coleman also teaches that dimming is known (para. 238, 252, 289). Accordingly, a mode that operates to dim image frames over a window of time, such as the luminance-adjusted image frames, as mapped above, is taught/suggested and obvious and predictable over Coleman.
Moreover, modifying Choi, in view of Yeoh, to add the above mapped and rationale mode, and also take into consideration rod and cone anatomy of an eye, per the teachings of Yeoh, when determining luminance values of an image, per Choi and Yeoh (the latter teaching the relationship between rod/cone and luminance), is all of taught and suggested by the prior art, and would have been obvious and predictable to one of ordinary skill in the art as of the effective filing date of the claimed invention. See MPEP §2143(A).
The prior art included each element recited in claim 1, although not necessarily in a single embodiment, with the only difference being between the claimed element and the prior art being the lack of actual combination of certain elements in a single prior art embodiment, as described above.
One of ordinary skill in the art could have combined the elements as claimed by known methods, and in that combination, each element merely performs the same function as it does separately. One of ordinary skill in the art would have also recognized that the results of the combination were predictable as of the effective filing date of the claimed invention. Additional motivation for one of ordinary skill in the art would be to fine tune image properties in consideration of user eye behavior and known responses of regions of an eye to luminance, light and other image properties.
Regarding claim 2:
Choi teaches: the mixed-reality device of claim 1, wherein the storage subsystem holds instructions executable by the logic subsystem to: determine, via the eye tracker, a position of a fovea of the eye (C9, last full paragraph, determine foveal axes of the user’s eyes; alternatively, Fig 8: 862, determine user fovea using eye tracking; another example; C26, full paragraph), and
adjust the luminance levels of the pixels of the image frame as a function of angle relative to the position of the fovea of the eye mapped to the image frame (Fig. 9: 970 and related description, fovea position is also used to determine adjusted luminance; alternatively, see C27, second full paragraph through C28, full paragraph, maximum luminance may be on foveal axis of user’s eye) and based at least on the rod and cone anatomy of the eye to generate the luminance-adjusted image frame (rod and cone was mapped in claim 1).
It would have been obvious for one of ordinary skill in the art, as of the effective filing date of Applicant’s claims, to have further modified the applied references, in view of Choi, to have obtained the above, motivated to use known features of a user’s eye to adjust/display/render images appropriately.
Regarding claim 3:
It would have been obvious for one of ordinary skill in the art to have further modified the applied reference(-s), in view of same, to have obtained: the mixed-reality device of claim 2, wherein the storage subsystem holds instructions executable by the logic subsystem to: determine a photopic luminance region of pixels of the luminance-adjusted image frame and a luminance roll-off region of pixels of the luminance-adjusted image frame (Yeoh, paras. 241-46, the system can determine whether the user’s eyes are in a photopic, mesotopic or scotopic vision mode based on determined pupil area; para. 246: different pupil areas can correspond to one of the 3 modes (photopic, mesotopic or scotopic), and luminance is correlated with pupil size (i.e. size of pupil area. See para. 245). Mesotopic (occurring under lower light and lamination levels, per para. 424); or scotopic levels (also lower illumination), correspond to a luminance roll-off region)
based at least on the position of the fovea of the eye (Choi, Fig. 9: 970 and related description, likewise teaches foveal axis and foveal zone as relevant to luminance/light regions) (also and alternatively, Yeoh, para. 216, the system can determine user gaze, which in turn is based on fovea position; alternatively, see para. 226, display of image pixels is based at least on position of the fovea, this in turn is related to the luminance determination mapped in claim 1; and second alternatively, see para. 269, Yeoh teaches the relationship of rod/cones and fovea, the rod/cone concentration relevant to photopic and other luminance regions),
set luminance levels of pixels in the photopic luminance region to the native luminance levels of corresponding pixels in the image frame, and
adjust luminance levels of pixels in the luminance roll-off region lower than the native luminance levels of corresponding pixels of the image frame (see para. 243, “ In some embodiments, the illumination levels associated with each of type of vision may be assigned arbitrarily, based on user preferences, and/or customization for a group to which the user belongs (e.g., based on gender, age, ethnicity, the presence of visual abnormalities, etc.”. This teaches setting and adjusting the luminance levels as claimed) , and the results of the modification would have been obvious and predictable to one of ordinary skill in the art as of the effective filing date of the claimed invention. See MPEP §2143(A).
The prior art included each element recited in claim 3, although not necessarily in a single embodiment, with the only difference being between the claimed element and the prior art being the lack of actual combination of certain elements in a single prior art embodiment, as described above.
One of ordinary skill in the art could have combined the elements as claimed by known methods, and in that combination, each element merely performs the same function as it does separately. One of ordinary skill in the art would have also recognized that the results of the combination were predictable as of the effective filing date of the claimed invention. Additional motivation for one of ordinary skill in the art would be to fine tune image properties in consideration of user eye behavior and known responses of regions of an eye to luminance, light and other image properties.
Regarding claim 4:
It would have been obvious for one of ordinary skill in the art to have further modified the applied reference(-s), in view of same, to have obtained: the mixed-reality device of claim 3, wherein the luminance levels of the pixels in the luminance roll-off region are adjusted as a function of angle and cone density relative to the position of the fovea, and the results of the modification would have been obvious and predictable to one of ordinary skill in the art as of the effective filing date of the claimed invention. See MPEP §2143(A).
Yeoh teaches rod and cone densities (Fig. 16: illustrates density of rods and cones), which is also the density of these photoreceptors (rods, cones), across the retina (this corresponds to a position relative to the fovea, based on known anatomy of the eye, fovea being at center of retina). In terms of using these densities to adjust/determine luminance levels, see mapping to claim 3 re: photopic, mesotopic or scotopic regions. While not copied/posted here, these paragraphs of Yeoh (241-46), mapped above and also herein, describe these three regions in terms of rods and cones, and their dominance respectively. Modifying the applied references, such that one of ordinary skill uses science of the eye, per both references, to adjust for luminance, as per both references, as a function of photoreceptor rod and cone densities, per Yeoh, and fovea position, also per the prior art and eye anatomy, is taught by the prior art, and would have been obvious and predictable to one of ordinary skill, also motivated to use known knowledge about eyes and their response to visual conditions, to modify visual properties of images.
One of ordinary skill in the art could have combined the elements as claimed by known methods, and in that combination, each element merely performs the same function as it does separately. One of ordinary skill in the art would have also recognized that the results of the combination were predictable as of the effective filing date of the claimed invention.
Regarding claim 5:
Yeoh teaches: the mixed-reality device of claim 1, wherein the storage subsystem holds instructions executable by the logic subsystem to: determine, via the eye tracker, an eye relief distance between the position of the eye and the near-eye display, and map the position of the eye to the image frame based at least on the eye relief distance (Yeoh, para. 142 and Fig. 4B. in combination with para. 19, eye-gaze tracker) (alternatively for eye tracker, Choi, Fig. 1: 130, eye tracking unit).
It would have been obvious for one of ordinary skill in the art, as of the effective filing date of Applicant’s claims, to have further modified the applied references, in view of same, to have obtained the above, and the results of the modification would have been obvious and predictable to one of ordinary skill in the art as of the effective filing date of the claimed invention. See MPEP §2143(A).
One of ordinary skill in the art could have combined the elements as claimed by known methods, and in that combination, each element merely performs the same function as it does separately. One of ordinary skill in the art would have also recognized that the results of the combination were predictable as of the effective filing date of the claimed invention. Additional motivation for one of ordinary skill in the art would be to fine tune image properties in consideration of user eye behavior and known responses of regions of an eye to luminance, light and other image properties.
Regarding claim 6:
Yeoh teaches: the mixed-reality device of claim 1, further comprising: an ambient light sensor configured to determine an ambient luminance level of a surrounding environment (para. 244, sensor to determine ambient light or illumination. See also para. 245, which specifically uses “ambient luminance”); and
wherein the storage subsystem holds instructions executable by the logic subsystem to: adjust the luminance levels of the pixels of the image frame further based at least on the ambient luminance level of the surrounding environment to generate the luminance-adjusted image frame (para. 244. See also paras. 240, 241, 242, 244, 245 (ambient luminance), for more detailed information on ambient light/luminance level and luminance).
It would have been obvious for one of ordinary skill in the art, as of the effective filing date of Applicant’s claims, to have further modified the applied references, in view of same, to have obtained the above, motivated to use known features of a user’s eye and surroundings to adjust/display/render images appropriately.
Regarding claim 8:
Yeoh teaches: the mixed-reality device of claim 1, wherein the mixed-reality device is configured to operate in a plurality of different luminance operating modes in which luminance levels of pixels of the image frame are adjusted differently based at least on a luminance operation mode selected from the plurality of different luminance operating modes to generate the luminance-adjusted image frame (Yeoh teaches at least three difference luminance eye vision behavior modes (photopic, mesotopic and scotopic), as mapped in claim 4 (e.g. Yeoh, paras. 241-45). These eye vision behaviors can correspond to illumination levels/modes (para. 243). Yeoh also teaches flexibility with modes/illumination levels (para. 243). This teaches Applicant’s claim 8).
It would have been obvious for one of ordinary skill in the art, as of the effective filing date of Applicant’s claims, to have further modified the applied references, in view of same, to have obtained the above, motivated to use known features of a user’s eye and surroundings to adjust/display/render images appropriately and with user control.
Regarding claim 9:
Yeoh teaches: the mixed-reality device of claim 8, further comprising: an input subsystem (Yeoh, para. 246, user input, as part of the I/O interface of Fig. 1: 140 in Choi) configured to receive user input (Yeoh, para. 246) indicating a luminance operating mode selected from the plurality of luminance operating modes of the mixed-reality device (select a mode as mapped in claim 8 in Yeoh); and
wherein the storage subsystem holds instructions executable by the logic subsystem to: adjust luminance levels of pixels of the image frame further based at least on the selected luminance operating mode to generate the luminance-adjusted image frame (see mapping to claim 8 re: luminance levels and modes).
It would have been obvious for one of ordinary skill in the art, as of the effective filing date of Applicant’s claims, to have further modified the applied references, in view of same, to have obtained the above, motivated to use known features of a user’s eye and surroundings to adjust/display/render images appropriately and with user control.
Regarding claim 11:
It would have been obvious for one of ordinary skill in the art to have further modified the applied reference(-s), in view of same, to have obtained: the mixed-reality device of claim 1, wherein the near-eye display includes a left-eye display and a right-eye display (Choi, C7, L35-37, display can have a left and right display) (likewise, Yeoh, Fig. 25A: 4052, also teaches a L and R eye display) wherein the plurality of different luminance operating modes includes a monocular operating mode (see mapping to claim 10 re: modes, having a “monocular” mode is taught by Yeoh, the same reference also teaching monocular fields of view (para. 276-77 and Fig. 24)),
and wherein the storage subsystem holds instructions executable by the logic subsystem to (Yeoh para. 344): in the monocular operating mode, render the luminance-adjusted image frame for display in one of the left-eye display and the right-eye display (see Yeoh, paras. 266, 280 (“provide slightly different images to the right and left eyes”), 411, paragraphs which teach that rendering different images for L and R eyes is known. Accordingly, the monocular operating mode can be one where different images are rendered for each eye, all of which taught by the prior art. See also Yeoh, Figs. 24Aand 25A),
generate a third luminance-adjusted image based at least on the image frame, wherein pixels in the third luminance-adjust image are reduced to a greater degree than corresponding pixels in the luminance-adjusted image (see mapping to claim 1, the adjusted luminance taught by the prior art isn’t limited to any specific end result. Having a second luminance adjusted frame having less luminance than a one (another frame), is obvious embodiment over the prior art, as is different rendering for a L and R eye display. See Yeoh, paras. 266, 280, 411, as also mapped above); and
render the third luminance-adjusted image (see “render” step of claim 1), in the other of the left-eye display and the right-eye display (see above mapping, Yeoh teaches different rendering for L and R eye displays),
and the results of the modification would have been obvious and predictable to one of ordinary skill in the art as of the effective filing date of the claimed invention. See MPEP §2143(A).
The prior art included each element recited in claim 11, although not necessarily in a single embodiment, with the only difference being between the claimed element and the prior art being the lack of actual combination of certain elements in a single prior art embodiment, as mapped and described above.
One of ordinary skill in the art could have combined the elements as claimed by known methods, and in that combination, each element merely performs the same function as it does separately. One of ordinary skill in the art would have also recognized that the results of the combination were predictable as of the effective filing date of the claimed invention. Additional motivation for one of ordinary skill in the art would be to fine tune image properties in consideration of user eye behavior and known responses of regions of an eye to luminance, light and other image properties.
Regarding claim 12: see also claim 1.
The method for controlling a mixed-reality device of claim 12 corresponds to the functions performed by the device of claim 1; the same rationale for rejection applies.
Regarding claim 13: see claim 2.
These claims are similar; the same rationale for rejection applies.
Regarding claim 14: see claim 3.
These claims are similar; the same rationale for rejection applies.
Regarding claim 15: see claim 5.
These claims are similar; the same rationale for rejection applies.
Regarding claim 16: see claim 6.
These claims are similar; the same rationale for rejection applies.
Regarding claim 18: see claim 8.
These claims are similar; the same rationale for rejection applies.
Regarding claim 19: see claim 9.
These claims are similar; the same rationale for rejection applies.
Claim(s) 7 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Choi in view of Yeoh and Coleman, and further in view of Lewis (U.S. Patent No. 11,428,955).
Regarding claim 7:
Choi, Yeoh and Lewis teach: the mixed-reality device of claim 1, further comprising: a communication subsystem (Choi, one aspect of Fig. 1: 120, such to communicate via the I/O interface of Fig. 1: 140) configured to receive a predicted ambient luminance level of the surrounding environment (Lewis, C16, full paragraph, IFF (identify-friend-or-foe) signals can be received, which can be predictive information. One example is in C16, under “Electromagnetic Signals and Predictive Actions”, the subsection “(Dark Tunnel)”, and also subsection (Sudden Lighting Changes) in C17, reproduced in part below:
PNG
media_image1.png
440
482
media_image1.png
Greyscale
See above, at C16 of Lewis reference / the signal that wearer is about to enter or exit a dark tunnel is “predicted ambient luminance level of the surrounding environment”. The system of Lewis adjusts luminance accordingly.
PNG
media_image2.png
624
456
media_image2.png
Greyscale
See above, other examples of predictive, and even ones that use predictive techniques via artificial intelligence of machine learning.
And, another example, from C24 of Lewis, L41-55:
PNG
media_image3.png
278
468
media_image3.png
Greyscale
from a remote computing system (see “Sudden Lighting Changes’ of Lewis above, the information or signal can come from a transmitter near the display. This is remote computing system) (alternatively, see Choi, Fig. 1: 110 console, also remove computing device that can give these signals, particularly with respect to the artificial intelligence described by Lewis (Choi, Fig. 1: 116 artificial reality engine) (second alternatively, Yeoh, para. 509, remote processing modules are known, modifying the processing of Lewis, to be handled via a remote processing module, is taught and obvious over the prior art, motivated to more evenly distribute processing tasks in a system); and
wherein the storage subsystem holds instructions executable by the logic subsystem to: adjust the luminance levels of the pixels of the image frame further based at least on the predicted ambient luminance level of the surrounding environment to generate the luminance-adjusted image frame (see above mapping and copied/pasted sections of Lewis).
It would have been obvious for one of ordinary skill in the art, as of the effective filing date of Applicant’s claims, to have further modified the applied references, in view of same, to have obtained the above, motivated to use known features of a user’s eye and surroundings to adjust/display/render images appropriately and in a timely fashion.
Regarding claim 17: see claim 7.
These claims are similar; the same rationale for rejection applies.
Claim(s) 20 is re rejected under 35 U.S.C. 103 as being unpatentable over Choi in view of Yeoh and Coleman and further in view of RC (“Rods & Cones” (https://www.cis.rit.edu/people/faculty/montag/vandplite/pages/chap_9/ch9p1.html), pp 1-6, January 30, 2003) and Mathur (U.S. Patent App. Pub. No. 20190287495).
Regarding claim 20: see also claim 4.
A portion of claim 20 is a combination of the features of claim 4 (which depends variously from claims 3, 2 and 1). Claim 20 is mapped to each claim below. The same rationale for claim 4 applies to claim 20. Regarding the remaining features of claim 20, see mapping to “RC” below.
a mixed-reality device comprising: an eye tracker configured to determine a position of a fovea of an eye (claim 2); and a near-eye display (claim 2) configured to:
in a transition operating mode where the mixed-reality device is preparing for deactivation, render a plurality of luminance-adjusted image frames over a window of time by: for each luminance-adjusted image frame (claim1)¸
generate an image frame including a plurality of pixels, wherein each pixel has a native luminance level (claim 1); map the position of the fovea of the eye to the image frame (claim 2),
determine a photopic luminance region of pixels and a luminance roll-off region of pixels of the image frame based at least on the position of the fovea of the eye (claim 3),
wherein the photopic luminance region is centered at the position of the fovea and extends outwardly angularly to an angular position where cone density of the eye becomes constant (RC, page 1, and description of Figure: “The density of cones falls of [sic] rapidly to a constant level at about 10-15 degrees from the fovea.”) , and wherein the luminance roll-off rejection begins at an angular boundary of the photopic luminance region and extends outward angularly such that the luminance roll-off region surrounds the photopic luminance region (RC, Id. and Mathur, paras. 184, 255, 300-05, 318, 331),
set luminance levels of pixels in the photopic luminance region to native luminance levels of the image frame (claim 3), and adjust luminance levels of pixels in the luminance roll-off region as a function of angle and cone density relative to the position of the fovea (claim 4) such that as an angular distance from the position of the fovea increases (claim 4), and further based on luminance levels of pixels of a previously-rendered image frame of a plurality of image frames corresponding to the plurality of luminance-adjusted image frames (claim 1) to generate the luminance-adjusted image frame (claim 1); and render the luminance-adjusted image frame for display to the eye (claim 4),
wherein the luminance levels of the pixels of the plurality of luminance-adjusted image frames are reduced to a greater degree over the window of time, such that the luminance levels of the pixels of the plurality of luminance-adjusted image frames become dimmer over the window of time (claim 1).
Modifying the applied references, in view of same, to have obtained the above, would have been obvious and predictable to one of ordinary skill in the art as of the effective filing date of the claimed invention. See MPEP §2143(A).
The prior art included each element recited in claim 20, although not necessarily in a single embodiment, with the only difference being between the claimed element and the prior art being the lack of actual combination of certain elements in a single prior art embodiment, as mapped and described above.
One of ordinary skill in the art could have combined the elements as claimed by known methods, and in that combination, each element merely performs the same function as it does separately. One of ordinary skill in the art would have also recognized that the results of the combination were predictable as of the effective filing date of the claimed invention. Additional motivation for one of ordinary skill in the art would be to fine tune image properties in consideration of user eye behavior and known responses of regions of an eye to luminance, light and other image properties.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Sarah Lhymn whose telephone number is (571)270-0632. The examiner can normally be reached M-F, 9:00 AM to 6:00 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao Wu can be reached at 571-272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Sarah Lhymn
Primary Examiner
Art Unit 2613
/Sarah Lhymn/Primary Examiner, Art Unit 2613
Prosecution Insights