DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1, 3-5, 8, 10-12, 15-16, and 20 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Mallinson (US 11703947 B2).
Regarding claim 1, Mallinson teaches an augmented reality / virtual reality (AR/VR) display device, comprising: a display to render content; an eye tracking system to capture a user's gaze;
"The saccade detection system 100 includes a processor 101, a memory 102, a display 130, and a sensor 110. The saccade detection system 100 may comprise, or be included with, one or more of a head mounted display (“HMD”) device, an augmented reality device, a virtual reality device, a wearable device, a portable user device, a smartphone, a personal computer, a television, a tablet computer, a game console, etc.” - Col 3, Lines 40-47
and a controller to manage the display and the eye tracking system, wherein the controller is to: detect a saccade based on capturing a shift in the user's gaze;
"The saccade detection system 100 may monitor a user's eye movement as the user views the computer generated graphics shown by the display 130 and/or real-world scenes." - Col 5, Lines 1-3
reduce a rendering quality of the rendered content for a duration of a post-saccade window;
" In some embodiments, the system may take in to effect that saccadic masking may persist for some time after the end of saccadic movement of the eye and allow the reduction period to exceed the expected duration of saccadic eye movement." - Col 7 Lines 52-56
NOTE: Mallinson teaches configuring the processor to reduce the resources for rendering frames of the content (reduction period, see col 3, lines 4-9) which functionally corresponds to reducing rendering quality.
and increase the rendering quality upon completion of the post-saccade window.
"The system returns the rendering method back to the normal baseline just prior to the predicted end of a saccade period. The predicted end of a saccade period may correspond to the predicted end of saccadic movement or the predicted end of the saccadic masking perceptual effect, which could last for some time after the end of the saccadic movement." - Col 9, Lines 35-39
NOTE: Mallinson teaches returning to the initial rendering method just prior to the end of a saccade period. However, the end of a saccade period is determined by a saccade duration prediction module 425. This predicted end may last some time after the actual end of the saccadic movement which shows that returning the rendering quality back from a reduced state to the initial state may occur after the completion of the saccade movement (post saccade window).
Regarding claim 8, the claim recites similar limitations to claim 1. Therefore, method claim 8 corresponds to the device disclosed in claim 1 and is rejected for the same reasons of anticipation as used above.
Regarding claim 15, the claim recites similar limitations to claim 1. Therefore, non-transitory computer readable memory claim 15 corresponds to the device disclosed in claim 1 and is rejected for the same reasons of anticipation as used above.
Regarding claim 3, Mallinson teaches the device of claim 1. Mallinson teaches wherein the duration of the post-saccade window is customized for the user.
"Since each individual's physiology varies, in some embodiments, prior to the process shown in FIG. 2, the system may lead a user through a calibration sequence to generate or configure an eye movement model individualized for the user." - Col 8, lines 36-40
NOTE: Mallinson discloses calibrating the user's eyes to configure an individualized eye movement model. Mallinson further discloses that a saccade detection module uses the eye movement model 450 from the calibration sequence to determine when a saccade occurs, see par 41. After the onset of a saccade is detected, the saccade duration prediction module 425 monitors the end of the saccade period and calculates the duration of the saccade. The end of the saccade period may be tracked by sensor 420. The measurements are compared with a calibration module 420 to determine the accuracy of the prediction module's prediction of the saccade duration. This will then update the eye movement model which was obtained from the user after a calibration sequence, see par 44. Mallinson teaches that the prediction may also involve the duration of time that occurs after the saccade movement ends which can be understood as the post-saccade window.
Regarding claim 10, the claim recites similar limitations to claim 3. Therefore, method claim 10 corresponds to the device disclosed in claim 3 and is rejected for the same reasons of anticipation as used above.
Regarding claim 4, Mallinson teaches the device of claim 1. Mallinson further teaches wherein the rendering quality includes one or more of a resolution or a frame rate.
"In a system with variable frame rate rendering, such as a system with a variable frame rate GPU, the system may pause rendering until the end of the saccadic masking period or render at a lower frame rate during the reduction period." - Col 7, Lines 24-27
NOTE: Mallinson discloses pausing rendering or rendering at a lower frame rate during saccade which functionally corresponds to the rendering quality including a frame rate.
Regarding claim 11, the claim recites similar limitations to claim 4. Therefore, method claim 11 corresponds to the device disclosed in claim 4 and is rejected for the same reasons of anticipation as used above.
Regarding claim 5, Mallinson teaches the device of claim 1. Mallinson teaches wherein the controller is to increase the rendering quality to full quality upon completion of the post-saccade window in a binary manner
"The system returns the rendering method back to the normal baseline just prior to the predicted end of a saccade period. The predicted end of a saccade period may correspond to the predicted end of saccadic movement or the predicted end of the saccadic masking perceptual effect, which could last for some time after the end of the saccadic movement." - Col 9, Lines 35-39
NOTE: Mallinson teaches returning the rendering method back to the normal base line after the predicted end of the saccade period which may take place after the actual end of the saccadic movement (post saccade window). Returning from a reduced rendering state to normal baseline (which can be understood as switching from a low quality state to a full quality state) is functionally the same behavior as increasing rendering quality in a binary manner.
Regarding claim 12, the claim recites similar limitations to claim 5. Therefore, method claim 12 corresponds to the device disclosed in claim 5 and is rejected for the same reasons of anticipation as used above.
Regarding claim 16, the claim recites similar limitations to claim 5. Therefore, non-transitory computer readable memory claim 16 corresponds to the device disclosed in claim 5 and is rejected for the same reasons of anticipation as used above.
Regarding claim 20, Mallinson teaches the non-transitory computer readable medium of claim 15. Mallinson further teaches wherein the instructions, when executed by the processor, cause the processor to receive input associated with customizing a duration of the post-saccade window.
"Since each individual's physiology varies, in some embodiments, prior to the process shown in FIG. 2, the system may lead a user through a calibration sequence to generate or configure an eye movement model individualized for the user." - Col 8, lines 36-40
NOTE: Mallinson discloses having the user go through a calibration sequence involving sensors to obtain information is used to configure an eye movement model individualized for the user. The data received by the sensors can be understood as inputs which are then used by a saccade detection module to predict the duration of the saccade window. The received sensor inputs directly correlate to determining saccade duration which one of ordinary skill can then use to customize the duration to align with the user’s saccade period. Mallinson teaches that the prediction may also involve the duration of time that occurs after the saccade movement ends which can be understood as the post-saccade window.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 2, 9, and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Mallison and Ibbotson et al (“Visual perception and saccadic eye movements”), hereinafter Ibbotson.
Regarding claim 2, Mallinson teaches the device of claim 1. Mallinson does not teach wherein duration of the post- saccade window is in a range from one hundred (100) milliseconds to four hundred (400) milliseconds. However, Ibbotson teaches wherein duration of the post- saccade window is in a range from one hundred (100) milliseconds to four hundred (400) milliseconds.
“After each saccade, many visual areas show a response increase that could underlie postsaccadic behavioral enhancements (Figure1). The onset of the postsaccadic increase coincides with saccade-end ,persists for 200-400ms," - Pg 3, Saccading enhancement, Par 3, lines 1-5
NOTE: Ibbotson discloses a post-saccadic response that begins when the saccade ends and persists for 200-400ms. This corresponds to the post-saccade window being in a range from 100ms to 400ms
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to modify Mallinson by incorporating the teachings of Ibbotson to have the duration of the post-saccade window be in a range from 100ms to 400 s. One would be motivated to make this combination to obtain the predicted result of consistently displaying VR content. When aligning the decreasing and increasing of rendering quality with the post-saccade range, the user will be able seamlessly experience the VR content without experiencing the decreased rendering quality that wouldn't have been perceived anyway. This combination would also allow the system to reduce consumption of system resources.
Regarding claim 9, the claim recites similar limitations to claim 2. Therefore, method claim 9 corresponds to the device disclosed in claim 2 and is rejected for the same reasons of obviousness as used above.
Regarding claim 19, the claim recites similar limitations to claim 2. Therefore, non-transitory computer readable memory claim 19 corresponds to the device disclosed in claim 2 and is rejected for the same reasons of obviousness as used above.
Claim(s) 6-7, 13-14, and 17-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Mallison and Connor et al (US 11507184 B2), hereinafter Connor.
Regarding claim 6, Mallinson teaches the device of claim 1. Mallinson does not teach wherein the controller is to increase the rendering quality to full quality upon completion of the post-saccade window following a ramp-up profile. However Connor teaches wherein the controller is to increase the rendering quality to full quality upon completion of the post-saccade window following a ramp-up profile
"FIG. 16b schematically illustrates an example in which one or more image frames having a first image resolution Q1 are generated for display, then one or more image frames having a second image resolution Q2 higher than the first image resolution are generated for display just before the time T2 at which the saccade is predicted to end, and at the time T2 image frames having a third image resolution Q3 higher than the second image resolution are generated for display" - Col 24, Lines 53-61
NOTE: Connor discloses displaying images of varying resolutions during a saccade period denotes as T1-T2. During this time period, image resolution Q1 to Q3 are displayed wherein resolution Q2 is higher than Q1 and Q3 is higher than Q2. This functionally corresponds to increasing rendering quality to full quality following a ramp up profile. After the combination, the image resolution ramp up profile as taught by Conners can be incorporated into Mallinson's system for returning to the baseline rendering method which occurs at a predicted end of the saccade. This combination will then have Mallinson' system gradually increase the image resolution (which is the behavior of a ramp-up profile) of the VR content after the saccade movements end.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to modify Mallinson by incorporating the teachings of Connor to have the controller increase the rendering quality to full quality upon completion of the post-saccade window following a ramp up profile. One would have been motivated to make this combination to align the increase in resolution to the user's gradual return of vision. The gradual increase would have a predicted result of reducing perception of visual transitions while restoring the full rendering quality of the VR content.
Regarding claim 13, the claim recites similar limitations to claim 6. Therefore, method claim 13 corresponds to the device disclosed in claim 6 and is rejected for the same reasons of obviousness as used above.
Regarding claim 17, the claim recites similar limitations to claim 6. Therefore, non-transitory computer readable memory claim 17 corresponds to the device disclosed in claim 6 and is rejected for the same reasons of obviousness as used above.
Regarding claim 7, Mallinson in view of Connor teaches the device of claim 6. Mallinson does not teach wherein the ramp-up profile is customized for the user. However, Connor teaches wherein the ramp-up profile is customized for the user.
"FIG. 16b schematically illustrates an example in which one or more image frames having a first image resolution Q1 are generated for display, then one or more image frames having a second image resolution Q2 higher than the first image resolution are generated for display just before the time T2 at which the saccade is predicted to end, and at the time T2 image frames having a third image resolution Q3 higher than the second image resolution are generated for display" - Col 24, Lines 53-61
NOTE: Connor discloses the ramp up profile, please see rejection of claim 6. After the combination, the ramp up profile as taught by Connor can modify Mallinson's system for obtaining an eye movement model that is individualized/customized to the user, see rejection of claim 3. Connor discloses that increasing the rendering quality with the ramp up profile is done after the end of the saccade window. This concept can be applied to Mallinson's teachings of the post-saccade window so that the ramp up profile will increase the resolution of the VR content when the saccade detection module predicts the end of the saccade. Since Mallinson already obtains unique user information regarding the saccade duration, one of ordinary skill in the art could then use the information to customize the ramp up profile to align with the user's post-saccade window.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to modify Mallinson by incorporating the teachings of Connor to have the ramp up profile be customized for the user. One would be motivated to make this combination because customizing the ramp-up profile according to the user's measured eye movement characteristics would lead to a predicted result of allowing restoration of the rendering quality to closely align with the user's visual recovery which would improve perceived image quality and avoid unnecessary rendering workload.
Regarding claim 14, the claim recites similar limitations to claim 7. Therefore, method claim 14 corresponds to the device disclosed in claim 7 and is rejected for the same reasons of obviousness as used above.
Regarding claim 18, the claim recites similar limitations to claim 7. Therefore, non-transitory computer readable memory claim 18 corresponds to the device disclosed in claim 7 and is rejected for the same reasons of obviousness as used above.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID V. NGUYEN whose telephone number is (571)272-6111. The examiner can normally be reached M-F 9:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, King Y Poon can be reached at 571-270-0728. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAVID VAN NGUYEN/Examiner, Art Unit 2617 /KING Y POON/Supervisory Patent Examiner, Art Unit 2617
Prosecution Insights