DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-8 and 16-20 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention. In particular, claims 1, 2, and 16 recite a method of performing texture sampling for a further region at a lower rate than a region of interest. However, this feature is not supported by the specification. On para. [0052 and 0193], the specification merely discloses "For the regions or objects of more interest, the sampling rate may be increased". However, the specification does not specifically disclose "texture sampling rate for the region of more interest may be increased".
Dependent claims 3-8 and 17-20 are rejected as being depending upon base claims 1 and 16.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-3, 6-7, 9-10, 13-14, and 16-19 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Patney et al. (US Pub. 2017/0263046 A1).
Regarding claim 1; Patney teaches a head mounted display system (Fig.8, para. [0088], a head mounted display (HMD)) comprising:
a graphics processing unit (a parallel processing unit (PPU) 200, Fig.2); and
a render engine (a graphics processing pipeline 600, Fig.6) implemented on the graphics processing unit (para. [0064]),
the render engine to: segment graphics into a region of interest and a further region of further interest based on eye gaze detection, the interest being more than the further interest (para. [0021 and 0088-0090], the system is configured to detect a fixation point by performing a gaze tracking. The fixation point is transmitted to the processor for performing a foveated rendering. In particular, an image will be perceived by the viewer such that pixels rendered at a higher shading rate are perceived on the fovea of the viewer's eye, and pixels rendered at a lower shading rate are perceived on the periphery of the retina. Since the fovea is associated with higher visual acuity, more rendering time is spent focused on the portions of the image that will be perceived on the fovea. Therefore, the region of fixation would be a region of interest. A peripheral region would be a region of further interest);
perform texture sampling for the region of interest at a rate; and perform texture sampling for the further region of further interest at a further rate that is lower than the rate (para. [0024], the foveated rendering algorithm is implemented as a coarse pixel shading shader which is configured to sample a mip-mapped texture map at a level of detail (LOD) calculated based on a variable shading rate corresponding to a sample location. Para. [0093-0095], a base LOD is calculated for sampling the texture map based on a texel size corresponding with a pixel. The base LOD is modified based on the variable shading rate for the pixel or pixel tile utilized during coarse pixel shading. For example, the base LOD may be used if the variable shading rate is one color sample per pixel; the base LOD may be incremented by one if the variable shading rate is one color sample per 2×2 pixel tile; the base LOD may be incremented by two if the variable shading rate is one color sample per 4×4 pixel tile; and the base LOD may be incremented by three if the variable shading rate is one color sample per 8×8 pixel tile, and so on. The color sample for the pixel or pixel tile is then generated by sampling the texture map 904 using texture coordinates for the pixel (or interpolated from multiple pixels over the pixel tile) at a particular level of detail in the mip-mapped texture specified by the modified LOD. Para. [0104], the shading rate at the fovea may be set to a resolution of one color sample per pixel (i.e., texture sampling rate at the fovea would be 1:1). As pixel tiles move into the periphery of the image, the shading rate may be changed to one color sample per 2×2 pixel tile (i.e., texture sampling rate would be 1:4). As pixel tiles move even further into the periphery of the image, the shading rate may be changed to one color sample per 4×4 pixel tile (i.e., texture sampling rate would be 1:16). Therefore, Patney further discloses that texture sampling rate for the periphery of the image is lower than that at the fovea).
Regarding claim 2; Patney teaches the head mounted display system of claim 1 as discussed above. Patney further teaches the render engine supports foveation (para. [0023-0024], Patney discloses a foveated rendering algorithm. In particular, pixels rendered at a higher shading rate are perceived on the fovea of the viewer's eye, and pixels rendered at a lower shading rate are perceived on the periphery of the retina).
Regarding claim 3; Patney teaches the head mounted display system of claim 1 as discussed above. Patney further teaches the region of interest corresponds to a location of focus (Para. [0088], a fixation point is detected by gaze tracking. In other words, the fixation point would be corresponding to the region of interest).
Regarding claim 6; Patney teaches the head mounted display system of claim 1 as discussed above. Patney further teaches the head mounted display comprising a strap (para. [0089], the head mounted display comprises a strap that surrounds the user’s head).
Regarding claim 7; Patney teaches the head mounted display system of claim 1 as discussed above. Patney further teaches the head mounted display comprising a left eye display and a right eye display (Fig.8, a left eye display 822 and a right eye display 824).
Regarding claim 9; Patney teaches a head mounted display system (Fig.8, para. [0088], a head mounted display (HMD)) comprising:
a graphics processing unit (a parallel processing unit (PPU) 200, Fig.2);
a render engine implemented on the graphics processing unit (a graphics processing pipeline 600, Fig.6), the render engine to:
receive, via an application program interface (para. [0036], a host processor executes a driver kernel that implements an application programming interface (API)), a rate for a region of interest (para. [0021 and 0088-0090], the system is configured to detect a fixation point by performing a gaze tracking. The fixation point is transmitted to the processor for performing a foveated rendering. In particular, an image will be perceived by the viewer such that pixels rendered at a higher shading rate are perceived on the fovea of the viewer's eye, and pixels rendered at a lower shading rate are perceived on the periphery of the retina. Since the fovea is associated with higher visual acuity, more rendering time is spent focused on the portions of the image that will be perceived on the fovea. Therefore, the region of fixation would be a region of interest. A peripheral region would be a region of further interest);
receive, via the application program interface, a further rate for a further region of interest, the further rate being lower than the rate; perform texture sampling for the region of interest at the rate; and perform texture sampling for the further region of interest at the further rate (para. [0024], the foveated rendering algorithm is implemented as a coarse pixel shading shader which is configured to sample a mip-mapped texture map at a level of detail (LOD) calculated based on a variable shading rate corresponding to a sample location. Para. [0093-0095], a base LOD is calculated for sampling the texture map based on a texel size corresponding with a pixel. The base LOD is modified based on the variable shading rate for the pixel or pixel tile utilized during coarse pixel shading. For example, the base LOD may be used if the variable shading rate is one color sample per pixel; the base LOD may be incremented by one if the variable shading rate is one color sample per 2×2 pixel tile; the base LOD may be incremented by two if the variable shading rate is one color sample per 4×4 pixel tile; and the base LOD may be incremented by three if the variable shading rate is one color sample per 8×8 pixel tile, and so on. The color sample for the pixel or pixel tile is then generated by sampling the texture map 904 using texture coordinates for the pixel (or interpolated from multiple pixels over the pixel tile) at a particular level of detail in the mip-mapped texture specified by the modified LOD. Para. [0104], the shading rate at the fovea may be set to a resolution of one color sample per pixel (i.e., texture sampling rate at the fovea would be 1:1). As pixel tiles move into the periphery of the image, the shading rate may be changed to one color sample per 2×2 pixel tile (i.e., texture sampling rate would be 1:4). As pixel tiles move even further into the periphery of the image, the shading rate may be changed to one color sample per 4×4 pixel tile (i.e., texture sampling rate would be 1:16). Therefore, Patney further discloses that texture sampling rate for the periphery of the image is lower than that at the fovea).
Regarding claim 10; Patney teaches the head mounted display system of claim 9 as discussed above. The limitation of claim 10 is substantially similar to claim 3. Therefore, claim 10 is rejected under the same rationale as claim 3.
Regarding claim 13; Patney teaches the head mounted display system of claim 9 as discussed above. The limitation of claim 13 is substantially similar to claim 6. Therefore, claim 13 is rejected under the same rationale as claim 6.
Regarding claim 14; Patney teaches the head mounted display system of claim 9 as discussed above. The limitation of claim 14 is substantially similar to claim 7. Therefore, claim 14 is rejected under the same rationale as claim 7.
Regarding claim 16; Patney teaches a method comprising: determining a region of interest and a further region of further interest; performing texture sampling for the region of interest at a rate; and performing texture sampling for the further region of further interest at a further rate that is lower than the rate (similar to the analysis of claim 1).
Regarding claim 17; Patney teaches the method of claim 16 as discussed above. Patney further teaches the interest is more than the further interest (see the analysis of claim 1, Patney discloses a method of detecting a fixation point corresponding to a fovea (i.e., a region of interest) in a foveated image. In other words, the interest at the fixation point (or the fovea) is more than in the periphery of the image).
Regarding claim 18; Patney teaches the method of claim 16 as discussed above. Patney further teaches the region of interest corresponds to a location of focus determined based on eye gaze detection (para. [0088], detecting a fixation point by gaze tracking).
Regarding claim 19; Patney teaches the method of claim 16 as discussed above. Patney further teaches receiving, via an application program interface (para. [0036], application programming interface (API)), the rate for the region of interest and the further rate for the further region of interest (see the analysis of claim 1).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 4-5, 11-12, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Patney et al. (US Pub. 2017/0263046 A1) in view of Navraev et al. (US Pub. 2018/0232936 A1).
Regarding claim 4; Patney teaches the head mounted display system of claim 1 as discussed above. Patney does not teach the rate is twice the further rate (In particular, Patney discloses that a base LOD is modified based on the variable shading rate which may be set to one color sample per pixel, one color sample per 2×2 pixel tile, one color sample per 4×4 pixel tile, or one color sample per 8×8 pixel tile, and so on. A shading rate of one color sample per pixel is assigned to the fovea. A shading rate is decreased as pixel tiles moves into the periphery of the foveated image. However, Patney does not teach that pixel tile may include 1x2 pixel tile and 2x1 pixel tile).
Navraev teaches that pixel tile may include 1x2 pixel tile and 2x1 pixel tile (para. [0068], Navarev discloses a method for rendering graphics using variable shading rates to generate multiple outputs. For example, the shading rate may include a 1×1 pixel shading rate, a 2×1 pixel shading rate, a 1×2 pixel shading rate, a 2×2 pixel shading rate, a 2×4 pixel shading rate, etc.).
Accordingly, the method of Patney of performing texture sampling based on variable shading rates as modified by the method of Navarev of providing a method for rendering graphics using variable shading rates including a 1×1 pixel shading rate, a 2×1 pixel shading rate, and a 1×2 pixel shading rate would render a method of performing texture sampling based on variable shading rates including a 1×1 pixel shading rate, a 2×1 pixel shading rate, and a 1×2 pixel shading rate. In particular, the 1×1 pixel shading rate would be assigned to the fovea. The 2×1 pixel shading rate and 1×2 pixel shading rate would be assigned to the periphery of the foveated image. As such, Patney as modified by Navarev further teaches that the rate is twice the further rate (i.e., the shading rate at the fovea is twice the shading rates in the periphery of the foveated image).
At the time of invention was effectively filed, it would have been obvious to one of ordinary skill in the art to modify the method of Patney of performing foveated rendering using variable shading rates to include the teaching of Navraev of rendering graphics using variable shading rates including a 1×1 pixel shading rate, a 2×1 pixel shading rate, a 1×2 pixel shading rate, a 2×2 pixel shading rate, a 2×4 pixel shading rate, etc. The motivation would have been in order to smoothly transition from a fine shaded portion (e.g., the fovea) to coarsely shaded portions (e.g., the periphery of the image).
Regarding claim 5; Patney teaches the head mounted display system of claim 1 as discussed above. The limitation of claim 5 is substantially similar to claim 4. Therefore, claim 5 is rejected under the same rationale as claim 4.
Regarding claim 11; Patney teaches the head mounted display system of claim 9 as discussed above. The limitation of claim 11 is substantially similar to claim 4. Therefore, claim 11 is rejected under the same rationale as claim 4.
Regarding claim 12; Patney teaches the head mounted display system of claim 9 as discussed above. The limitation of claim 12 is substantially similar to claim 5. Therefore, claim 12 is rejected under the same rationale as claim 5.
Regarding claim 20; Patney teaches the method of claim 16 as discussed above. The limitation of claim 20 is substantially similar to claim 4. Therefore, claim 20 is rejected under the same rationale as claim 4.
Claims 8 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Patney et al. (US Pub. 2017/0263046 A1) in view of Sendai et al. (US Pub. 2016/0209916 A1).
Regarding claim 8; Patney teaches the head mounted display system of claim 1 as discussed above. Patney does not teach a left look-front camera and a right look-front camera.
Sendai teaches the head mounted display system comprises a left look-front camera and a right look-front camera (Fig.1, para. [0077], a head mounted display device comprises stereo cameras 61 including a left camera and a right camera).
At the time of invention was effectively filed, it would have been obvious to one of ordinary skill in the art to modify the head mounted display system of Patney to include the stereo cameras as taught by Sendai. The motivation would have been in order to improve high-fidelity mixed reality passthrough by mimicking human vision.
Regarding claim 15; Patney teaches the head mounted display system of claim 9 as discussed above. The limitation of claim 15 is substantially similar to claim 8. Therefore, claim 15 is rejected under the same rationale as claim 8.
Inquiries
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NGUYEN H TRUONG whose telephone number is (571)270-1630. The examiner can normally be reached M-F: 10-6.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chanh Nguyen can be reached at 571-272-7772. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NGUYEN H TRUONG/Examiner, Art Unit 2623
/CHANH D NGUYEN/Supervisory Patent Examiner, Art Unit 2623
Prosecution Insights