Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 6 and 17 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 6 and 17 recite “a pixel block of N*M pixels in the first reprojected image, wherein the second pixel block is a pixel block of N*M pixels in the first temporary image, wherein a position of the first pixel block in the first reprojected image corresponds to a position of the second pixel block in the first temporary image, and wherein at least one of N and M is an integer greater than 1”. N and M are undefined.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 12 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Cheng et al. (US 2022/0101504) in view of Bronder (US 2018/0307304).
Regarding claim 1, Cheng et al. (hereinafter Cheng) discloses an image rendering method (Cheng, [0004], “a method comprises calculating motion between a preceding image frame and a target image frame to be rendered”), comprising:
rendering, by an electronic device (Cheng, [0015], “The image rendering system 100 comprises an image processor 120 configured to receive output of one or more input source(s) 110, render or generate images based on the received output of the one or more input source(s) 110”), a first data frame at a first resolution to obtain a first image frame (Cheng, [0023], “the motion analyzer module 205 commands the image/MV render module 210 to render a full-size image frame, or an original (O) frame, for the current frame”), wherein the first data frame is a data frame of a scene (Cheng, [0023], “render a full-size image frame, or an original (O) frame, for the current frame”) of an application that is running on the electronic device (Cheng, [0016], “The input source(s) 110 may comprise one or more components related to controlling the rendering of a game display”);
rendering, by the electronic device, a plurality of second data frames at a second resolution to obtain a plurality of second image frames (Cheng, [0038], “renders a small image for the current frame based on the determined motion. For example, method 300 may render a small image comprising a portion of the target frame or the current frame wherein substantial motion occurs, and may not render the remaining regions of the target frame or current frame wherein substantial motion does not occur. As another example, method 300 may render a small image comprising the entire target frame but at a lower resolution. As yet another example, method 300 may render a small image comprising the portion of the current frame wherein substantial motion occurs, but at a lower resolution than a desired resolution of the current frame”. Repeating the same process reads on a plurality of second data frames at a second resolution to obtain a plurality of second image frames), wherein the second resolution is less than the first resolution (Cheng, [0038], “method 300 may render a small image comprising the entire target frame but at a lower resolution”), wherein the plurality of second data frames are data frames of the scene that follow the data frame (Cheng, [0034], “At 305, method 300 determines motion between a previous frame to a current frame. For example, method 300 analyzes the motion between a previous frame and the current frame or target frame by analyzing or evaluating the rendering commands for the target frame”), and wherein the plurality of second data frames have a one-to-one correspondence with the plurality of second image frames (Cheng, [0038], “At 335, method 300 renders a small image for the current frame based on the determined motion”); and
performing, by the electronic device, a compensation process on rendering data of the plurality of second image frames based on using the first image frame as a reference frame (Cheng, [0040], “At 350, method 300 performs motion-compensated image enhancement based on the small image, the reference frame, and the determined motion to generate the current frame”).
Bronder discloses a first data frame of a first scene (Bronder, [0010], “generate a first scene of the virtual reality simulation based on the virtual motion information and the head motion information, generate a set of visual cues based on the head motion information, render a frame including the set of visual cues and the first scene of the virtual reality simulation”).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the rendering method of Cheng which renders a first frame a first resolution and renders subsequent frames at a second resolution with the scene generation function, as taught by Bronder which teaches generating a first scene of a virtual environment. The motivation for doing so would have been reducing rendering workload while maintaining visual quality.
Regarding claim 12, Cheng discloses an electronic device (Cheng, [0013], “image rendering. In particular, systems and methods are provided for accelerated image rendering with motion compensation. An image rendering system, such as the system depicted in FIG. 1, may include an image processor that renders image frames for display based on rendering commands received from one or more input sources”), comprising:
a processor (Cheng, [0016], “a central processing unit (CPU)”); and
a memory (Cheng, [0016], “a system memory”);
wherein the processor is coupled to the memory (Cheng, Fig. 1);
wherein the memory is configured to store a program or instructions (Cheng, [0018], “a memory 124 comprising non-transitory and transitory memory for storing instructions to be executed by the image processor 120 to perform the methods described further herein”); and wherein the processor is configured to execute the program or instructions to facilitate the following being performed by the electronic device (Cheng, [0018], “a memory 124 comprising non-transitory and transitory memory for storing instructions to be executed by the image processor 120 to perform the methods described further herein”).
Cheng as modified by Bronder with the same motivation from claim 1 discloses a transceiver (Bronder, [0088], “communications component 40 may include one or more buses, and may further include transmit chain components and receive chain components associated with a transmitter and receiver, respectively, operable for interfacing with external devices”).
The remaining limitations recite in claim 12 are similar in scope to the method recited in claim 1 and therefore are rejected under the same rationale.
Regarding claim 20, Cheng discloses a non-transitory computer-readable storage medium having instructions stored thereon, wherein the instructions, when executed by an electronic device, facilitate performance of the following by the electronic device (Cheng, [0018], “a memory 124 comprising non-transitory and transitory memory for storing instructions to be executed by the image processor 120 to perform the methods described further herein. The memory 124 may further provide a buffer and/or cache for storing data relating to the processing and/or rendering of image frames from the input source(s) 110, in order to support the functions of the image processor 120”).
The limitations recite in claim 20 are similar in scope to the method recited in claim 1 and therefore are rejected under the same rationale.
Claims 2-5 and 13-16 are rejected under 35 U.S.C. 103 as being unpatentable over Cheng et al. (US 2022/0101504) in view of Bronder (US 2018/0307304), as applied to claims 1 and 12, in view of Narayanan et al. (US 2014/0184626) in further view of Crocker et al. (US 5,915,265).
Regarding claim 2, Cheng discloses a buffer (Cheng, [0018], “provide a buffer and/or cache for storing data relating to the processing and/or rendering of image frames from the input source(s) 110, in order to support the functions of the image processor 120”);
the first resolution and the second resolution (Cheng, [0023], “the motion analyzer module 205 commands the image/MV render module 210 to render a full-size image frame, or an original (O) frame, for the current frame”. In addition, in paragraph [0038], “renders a small image for the current frame based on the determined motion. For example, method 300 may render a small image comprising a portion of the target frame or the current frame wherein substantial motion occurs, and may not render the remaining regions of the target frame or current frame wherein substantial motion does not occur. As another example, method 300 may render a small image comprising the entire target frame but at a lower resolution. As yet another example, method 300 may render a small image comprising the portion of the current frame wherein substantial motion occurs, but at a lower resolution than a desired resolution of the current frame”);
buffer the reference frame (Cheng, [0018], “provide a buffer and/or cache for storing data relating to the processing and/or rendering of image frames from the input source(s) 110, in order to support the functions of the image processor 120”. In addition, in paragraph [0040], “At 350, method 300 performs motion-compensated image enhancement based on the small image, the reference frame, and the determined motion to generate the current frame”);
buffer a respective second image frame of the plurality of second image frames (Cheng, [0018], “provide a buffer and/or cache for storing data relating to the processing and/or rendering of image frames from the input source(s) 110, in order to support the functions of the image processor 120”. In addition, in paragraph [0038], “renders a small image for the current frame based on the determined motion. For example, method 300 may render a small image comprising a portion of the target frame or the current frame wherein substantial motion occurs, and may not render the remaining regions of the target frame or current frame wherein substantial motion does not occur. As another example, method 300 may render a small image comprising the entire target frame but at a lower resolution. As yet another example, method 300 may render a small image comprising the portion of the current frame wherein substantial motion occurs, but at a lower resolution than a desired resolution of the current frame”);
performing the compensation process on rendering data of the respective second image frame based on rendering data of the reference frame (Cheng, [0040], “At 350, method 300 performs motion-compensated image enhancement based on the small image, the reference frame, and the determined motion to generate the current frame”);
Cheng as modified by Bronder does not expressly disclose “a first buffer and a second buffer”;
Narayanan et al. (hereinafter Narayanan) discloses a first buffer and a second buffer (Narayanan, [0021], “a frame buffer 220 that comprises front/back buffer 222a and front/back buffer 222b”).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to use the multiple buffers of Narayanan to store the image data corresponding to different resolutions, as taught by Cheng. The motivation for doing so would have been enabling efficient processing at different resolutions.
In addition, Cheng as modified by Bronder and Narayanan does not expressly disclose “a size of a buffer corresponds to resolution”;
Crocker et al. (hereinafter Crocker) discloses a size of a buffer corresponds to resolution (Crocker, col 2. 65-67, “The size of frame buffer allocated is dependent on the resolution and number of colors desired”).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the buffer of Cheng as modified by Bronder and Narayanan by allocating a buffer size corresponding to image resolution as taught by Crocker. The motivation for doing so would have been providing ability to accommodate image data of different resolutions.
Regarding claim 3, Cheng discloses before rendering the first data frame to obtain the first image frame (Cheng, [0033], “FIG. 3 shows a high-level flow chart illustrating an example method 300 for image rendering based on estimated motion between image frames”), the method further comprises:
obtaining scene indication information corresponding to the first data frame (Cheng, [0034], “At 305, method 300 determines motion between a previous frame to a current frame. For example, method 300 analyzes the motion between a previous frame and the current frame or target frame by analyzing or evaluating the rendering commands for the target frame”); and determining that a change in application scene has occurred based on the scene indication information (Cheng, [0035], “At 310, method 300 determines whether the current frame includes a scene change relative to the previous frame. For example, if the rendering commands indicate that the entire scene is changing in the current frame relative to the previous frame…If the current frame includes a scene change (“YES”), method 300 continues to 320”).
Regarding claim 4, Cheng discloses before rendering the plurality of second data frames to obtain the plurality of second image frames (Cheng, [0033], “FIG. 3 shows a high-level flow chart illustrating an example method 300 for image rendering based on estimated motion between image frames”), the method further comprises:
for a respective second data frame, obtaining scene indication information corresponding to the respective second data frame (Cheng, [0034], “At 305, method 300 determines motion between a previous frame to a current frame. For example, method 300 analyzes the motion between a previous frame and the current frame or target frame by analyzing or evaluating the rendering commands for the target frame”); and
determining that a change in application scene has not occurred based on the scene indication information (Cheng, [0035], “if the current frame does not include a scene change (“NO”), method 300 continues to 315”).
Regarding claim 5, Cheng discloses upsampling the respective second image frame to obtain an upsampled first temporary image (Cheng, [0038], “method 300 optionally upscales the small image to the desired resolution of the current frame”);
reprojecting the reference frame relative to the respective second image frame to obtain a first reprojected image (Cheng, [0039], “retrieves a reference frame from memory, such as the memory 250. The reference frame may comprise a preceding enhanced partial render (e.g., an enhanced R frame) or a preceding full-size frame (e.g., O frame)”. In addition, in paragraph [0040], “method 300 performs motion compensation on the reference frame based on the determined motion to obtain a motion-compensated reference frame”); and
performing compensation on rendering data of a pixel at a position in the first temporary image using rendering data of a pixel at a corresponding position in the first reprojected image (Cheng, [0040], “method 300 fills empty pixels in the R frame containing the small image, for example, with corresponding pixels from the motion-compensated reference frame. Additionally or alternatively, method 300 selectively blends low-quality pixels in the R frame with the relatively higher quality, motion-compensated pixels of the motion-compensated reference frame”).
Cheng as modified by Bronder and Narayanan with the same motivation from claim 2 discloses the first buffer and the second buffer (Narayanan, [0021], “a frame buffer 220 that comprises front/back buffer 222a and front/back buffer 222b”).
Regarding claims 13-16, claims 13-16 recite functions that are similar in scope to the method steps recited in claims 2-5 and therefore are rejected under the same rationale.
Claims 9 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Cheng et al. (US 2022/0101504) in view of Bronder (US 2018/0307304) in view of Narayanan et al. (US 2014/0184626) in view of Crocker et al. (US 5,915,265), as applied to claims 2 and 13, in view of Ansorregui et al. (US 2020/0380739) in further view of Shin et al. (US 2020/0359000).
Regarding claim 9, Cheng discloses the reference frame (Cheng, [0040], “At 350, method 300 performs motion-compensated image enhancement based on the small image, the reference frame, and the determined motion to generate the current frame”), performing compensation on rendering data of the respective second image frame (Cheng, [0040], “method 300 fills empty pixels in the R frame containing the small image, for example, with corresponding pixels from the motion-compensated reference frame. Additionally or alternatively, method 300 selectively blends low-quality pixels in the R frame with the relatively higher quality, motion-compensated pixels of the motion-compensated reference frame”);
Cheng as modified by Bronder and Narayanan with the same motivation from claim 2 discloses the first buffer and the second buffer (Narayanan, [0021], “a frame buffer 220 that comprises front/back buffer 222a and front/back buffer 222b”);
Cheng as modified by Bronder, Narayanan and Crocker does not expressly disclose “determining a residual image”;
Ansorregui et al. (hereinafter Ansorregui) discloses determining a residual image (Ansorregui, [0026], “residual pixel values calculated as the difference between values of corresponding pixels in the first image and the modified second image”).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to compensate the image data of Cheng using the concept of residual image as taught by Ansorregui in order to improving image quality.
Cheng as modified by Bronder, Narayanan, Crocker and Ansorregui does not expressly disclose “determining a second reprojected image of the residual image”;
Shin et al. (hereinafter Shin) discloses determine a reprojected image of the residual image (Shin, [0166], “the generated residual videos may be 3D-warped to a base view (S1102). Here, residual videos may be 3D-warped to a base view, or pixels corresponding to a residual video within an additional view video may be 3D-warped”. The 3D wrap operation transforms the residual image from one view coordinate system to another view coordinate system. The transformations reads on reproject).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to apply the concept of warping technique of Shin to the residual image as taught by Cheng as modified by Bronder, Narayanan, Crocker and Ansorregui in order to improving accuracy of the reconstructed image.
Regarding claim 19, claim 19 recites function that is similar in scope to the method recited in claim 9 and therefor is rejected under the same rationale.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Cheng et al. (US 2022/0101504) in view of Bronder (US 2018/0307304) in view of Narayanan et al. (US 2014/0184626) in view of Crocker et al. (US 5,915,265) in view of Ansorregui et al. (US 2020/0380739) in view of Shin et al. (US 2020/0359000), as applied to claim 9, in view of Burt et al. (The Laplacian Pyramid as a Compact Image Code, IEEE, 1983).
Regarding claim 10, Cheng teaches the reference frame; Cheng as modified by Bronder, Narayanan teaches the first buffer;
Cheng as modified by Bronder, Narayanan, Crocker and Ansorregui with the same motivation from claim 9 discloses determining the residual image (Ansorregui, [0026], “residual pixel values calculated as the difference between values of corresponding pixels in the first image and the modified second image”);
Cheng as modified by Bronder, Narayanan, Crocker, Ansorregui and Shin does not expressly disclose “downsampling the reference frame, and then performing upsampling, to obtain a second temporary image”;
Burt et al. (hereinafter Burt) discloses downsampling image to a resolution and then upsampling to obtain a temporary image, determine an image based on the original image and the temporary image (Burt, The Gaussian Pyramid, [0001], “We say that g1 is a “reduced” version of g0 in that both resolution and sample density are decreased”. In addition, in The Laplacian Pyramid, [0001], “we encode the error image which remains when an expanded g1, is subtracted from g0. This image becomes the bottom level of the Laplacian pyramid”. Reduce version reads on downsampling and expanded version reads on upsampling. The error image represents the image based on the original image and the temporary image).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to generate the residual image of Cheng as modified by Bronder, Narayanan, Crocker, Ansorregui using the concept of downsampling and upsampling technique as taught by Burt in order to improving image reconstruction quality.
Allowable Subject Matter
Claim 11 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claims 6-8 and 17-18 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KYLE ZHAI whose telephone number is (571)270-3740. The examiner can normally be reached 9AM-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ke Xiao can be reached at (571) 272 - 7776. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KYLE ZHAI/Primary Examiner, Art Unit 2611
Prosecution Insights