DETAILED ACTION
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 12, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Shen et al., US 2024/0196033, in view of Amer et al., US 2019/0104311.
In Reference to Claims 1, 12, and 17
Shen et al. teaches an apparatus with one or more processors and memory storing computer readable instructions, A non-transitory computer readable medium storing instructions for execution by one or more processors (Fig. 2-3 and Par. 16- 17) and method comprising obtaining, by a cloud server (Abstract and Par. 1 “cloud gaming system”, Fig. 3 and Par. 17), a synchronization object of game texture data for a target game running on the cloud server (Fig. 5 and Par. 20, 32, and 35 which teaches a “present call” 527 or 557), and the synchronization object indicating a rendering completion status for the game picture by the game texture data (Fig. 5 and Par. 32 and 35 “With completion of rendering of frame N+1, the video game instance issues a present call 527.” See also Par. 20 “the video game instance 108 renders a video frame, makes a present call to the API, and then proceeds to the rendering of the next video frame. As such, the present call issued by the video game instance 108 signaling completion of rendering of the previous frame thus also can signal the start of rendering of the next frame, and thus serve as a frame start trigger.” ); obtaining a reading result of an image encoder for the synchronization object; obtaining, using the reading result, a rendering completion status for the game picture from the synchronization object (Fig. 5 and Par. 32 and 35 “This present call 527 signals both the frame end 528 for frame N+1 and the frame start 529 for frame N+2, as well as triggers the encoder to encode 530 the rendered frame N+1”); obtaining the game picture based on the rendering completion status indicating that rendering of the game picture has been completed by using the game texture data; and encoding, using the image encoder, the game picture (Fig. 5 Par. 32 and 35 “triggers the encoder to encode 530 the rendered frame N+1, resulting in an encoded frame N+1 (encoded frame 531) that is inserted into a data stream being transmitted to the client device by the server streaming application of the gaming server” see also Par. 20 “When the rendering of the current frame is complete (as indicated by, for example, the video game instance 108 issuing a present call), at block 416 the UMD 316-directs the encoder 322 to encode the rendered video frame to generate an encoded video frame using any of a variety of video encoding protocols.”).
Further Shen et al. discusses a labels a series of sequentially rendered and encoded frames (Fig. 5 “N, N+1, N_2, etc.”) and teaches where the game engine rendering the frames issues the present call “synchronization object” via an API (Par. 20). However, Shen et al. does not explicitly teach a texture handle, the texture handle indicating a game picture rendered based on the game texture data that is sent from the renderer to the encoder.
Amer et al. teaches a system for cloud gaming where the rending and encoding system for the system includes a texture handle, the texture handle indicating a game picture rendered based on the game texture data that is sent from the renderer to the encoder (Fig. 2 and Par. 27 “In another embodiment, rendering unit 215 notifies encoder 230 of repeated frames in buffer 220. For example, in this embodiment, rendering unit 215 specifies the index numbers of frames that are repeated frames.” Where the index number of a frame constitutes a “texture handle indicating a game picture rendered based on the game texture data”).
It would be desirable to modify the system of Shen et al. to include communication of frame index numbers as taught by Amer et al. in order to allow the rendering and encoding pipeline to improve efficiency to save bandwidth and quality in the cloud gaming application while still ensuring proper synchronization as discussed in Amer et al. Par. 28 and 29.
Therefore, it would have been obvious to one of ordinary skill in the art at the time of filing of the invention to modify the system of Shen et al. to include communication of frame index numbers as taught by Amer et al.
Claims 2-4, 10, 13-15, and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Shen et al., US 2024/0196033, Amer et al., US 2019/0104311, further in view of unrealengine.com webpage titled “Pixel Streaming: delivering high-quality UE4 content to any device, anywhere” by Sebastien Miglio (hereinafter Miglio).
In Reference to Claims 2, 13 and 18
Shen et al. and Amer et al. teach an apparatus, medium, and method as described above in reference to Claims 1, 12, and 17. Shen et al. teaches where the cloud server comprises the image encoder (Fig. 3 and Par. 17), and further comprising: generating, by an engine in the cloud server corresponding to the target game, the game texture data (Fig. 4 ref. 414 and Par. 20 where examiner considers a video game process for rendering frame of a virtual game world as described in Par. 20 an engine) and transmitting, by the engine, to the image encoder and using a process communication protocol, the texture handle and the synchronization object of the game texture data (Par. 20 “the video game instance 108 renders a video frame, makes a present call to the API” and “When the rendering of the current frame is complete (as indicated by, for example, the video game instance 108 issuing a present call), at block 416 the UMD 316-directs the encoder 322 to encode the rendered video frame to generate an encoded video frame using any of a variety of video encoding protocols.”), wherein the engine and the image encoder in the cloud server belong to two independent processing processes, and wherein the engine and the image encoder interact based on the process communication protocol (Fig. 3 and Par. 20 which teach distinct rendering and encoding processes, as well as Par. 17 which teaches a separate “Server Streaming Application 318” for encoding frames rendered by the “Video Game 324” where the two communicate using API calls mediated by the UMD).
However, Shen et al. does not teach where the game texture data generated by the engine in a cloud server is by an engine plug-in.
Miglio teaches where rendering of a game is performed by an engine plug-in (Page 4 “Pixel Streaming plugin - This plugin runs inside Unreal Engine. It encodes the final results of every rendered frame using H.264 video compression, packs those video frames along with the game audio into a media stream, and sends it to the WebRTC Proxy Server”).
It would be desirable to modify the system, medium, and method of Shen et al. and Amer to use an engine plug-in to render frames on the cloud server as taught by Miglio in order to render game frames for use in a web browser on the client device allowing the user to play game with a much less powerful device as described on Miglio page 3.
Therefore, it would have been obvious to one of ordinary skill in the art at the time of filing of the invention to modify the system, medium, and method of Shen et al. and Amer to use an engine plug-in to render frames on the cloud server as taught by Miglio.
In Reference to Claim 3, 14, and 19
Shen et al., Amer et al. and Miglio teach invoking the engine plug-in to arrange a plurality of generated game texture data, to obtain a target texture queue (Amer al. Fig. 2 and Par. 25-28 “buffer 220”); obtaining first game texture data from the target texture queue as a render target; rendering the game picture by using the first game texture data corresponding to the render target (Shen et al. Fig. 4-5 and Par. 20, 32, and 35. See also Amer et al. Fig. 2 and Par. 25); and transmitting a texture handle and a synchronization object of the game texture data corresponding to the render target to the image encoder through the process communication protocol (Shen et al. Par. 20 which teaches the API and UMD sending the present call from the renderer to the encoder and Amer et al. Par. 27 which teaches the sending the texture handle as well).
In Reference to Claim 4, 15, and 20
Shen et al., Amer et al. and Miglio teaches where based on the reading result indicating a reading success, the method further comprises acquiring, by the image encoder, the rendering completion status for the game picture from the synchronization object by invoking the image encoder to parse the synchronization object; and acquiring the rendering completion status for the game picture from the parsed synchronization object (Shen et al. Par. 20 which teaches sending the present call API call to the encoder to indicate that rendering of a frame is complete and encoding can begin, and Amer et al. Par. 27-28 which teaches that the encoder retrieves frames from the buffer and encodes uniquely rendered frames and skip frames based on the determination of the type of frame).
In Reference to Claim 10
Shen et al., Amer et al. and Miglio teach where after the encoding the game picture by using the image encoder, the method further comprises obtaining next image encoded data; and transmitting, using on a real-time communication protocol, the next image encoded data to a game client running the target game; obtaining a next piece of game texture data adjacent to the game texture data from the target texture queue; and rendering the game picture according to the next piece of game texture data (Shen et al. Fig. 1, 4-5 and Par. 1, 18-21 which teach transmitting in real-time the rendered and encoded frames over a network to a client device running the game. And Par. 20 “This subprocess then repeats for the generation, encoding, and transmission of the next video frame at the indicated frame rate.” See Amer et al. as described above which teaches the target texture queue).
Claims 11 is rejected under 35 U.S.C. 103 as being unpatentable over Shen et al., US 2024/0196033, Amer et al., US 2019/0104311, unrealengine.com webpage Miglio and Posin, US 2017/0354878.
In Reference to Claim 11
Shen et al., Amer et al. and Miglio teaches a method as described above in reference to Claim 10. However, the do not teach encrypting the image encoded data to obtain encrypted encoded data; and transmitting the encrypted encoded data to the game client using the real-time communication protocol; decrypting the encrypted encoded data through the game client to obtain the image encoded data; and causing to be output the game picture of the target game.
Posin teaches a system for cloud gaming which teaches encrypting the image encoded data to obtain encrypted encoded data; and transmitting the encrypted encoded data to the game client using the real-time communication protocol; decrypting the encrypted encoded data through the game client to obtain the image encoded data; and causing to be output the game picture of the target game (Fig. 1B and Par. 59-64).
It would be desirable to modify the method of Shen et al., Amer et al. and Miglio to include encryption of encoded game video data as taught by Posin in order to provide enhanced data security.
Therefore, it would have been obvious to one of ordinary skill in the art at the time of filing of the invention to modify the method of Shen et al., Amer et al. and Miglio to include encryption of encoded game video data as taught by Posin.
Allowable Subject Matter
Claims 5-9 and 16 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CARL V LARSEN whose telephone number is (571)270-3219. The examiner can normally be reached Monday through Friday; 10:00 am - 6:30 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Dmitry Suhol can be reached at (571) 272-4430. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CARL V LARSEN/ Examiner, Art Unit 3715
Prosecution Insights