DETAILED ACTION
Notice of AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers submitted under 35 U.S.C. 119(a)-(d), which papers have been placed of record in the file.
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 4/11/2023 and 1/3/2024 have been considered by the examiner.
Claim Objections
Claims 3-9 and 12-18 are objected to because of the following informalities:
In Claims 3 and 12, Lines 5, the terms “receiving, from the sever, a second interactive” should be changed to “receiving, from the server, a second interactive”, to correct the spelling for the term server in accordance with specification Paragraph [0093], in order to avoid clarity issues and maintain consistency. Appropriate correction is required.
In Claims 4 and 13, Lines 1, the terms “the display mode is synchronous display” should be changed to “the display mode is a synchronous display”, in order to avoid clarity issues. Appropriate correction is required.
In Claim 4 and 13, Lines 4, the terms “determining a display mode of the first image element” should be changed to “determining the display mode of the first image element”, in accordance with specification Paragraph [0025], in order for clarity of the record and to prevent a rejection under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph. See MPEP § 2173.05(d).
In Claims 5-7 and 14-16, Lines 1, the terms “wherein, the display mode is” should be changed to “wherein the display mode is”, remove the “comma” after the term “wherein” in order to avoid clarity issues and maintain consistency. Appropriate correction is required.
In Claims 5 and 14, Lines 1, the terms “the display mode is synchronous display” should be changed to “the display mode is a synchronous display”, in order to avoid clarity issues. Appropriate correction is required.
In Claims 6 and 15, Lines 1, the terms “the display mode is transparency synthesis display” should be changed to “the display mode is a transparency synthesis display”, in order to avoid clarity issues. Appropriate correction is required.
In Claims 7 and 16, Lines 1, the terms “the display mode is separate display” should be changed to “the display mode is a separate display”, in order to avoid clarity issues. Appropriate correction is required.
In Claim 8 and 17, Lines 6, the terms “rendering function name, so as to render” should be changed to “rendering function name,, in order for clarity of the record and to prevent a rejection under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph. See MPEP § 2173.05(d).
In Claims 9 and 18, Lines 1, the terms “wherein, the image frame is” should be changed to “wherein the image frame is”, remove the “comma” after the term “wherein” in order to avoid clarity issues and maintain consistency. Appropriate correction is required.
In Claim 16, Line 6, the terms “second image element, so as to display” should be changed to “second image element, in order for clarity of the record and to prevent a rejection under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph. See MPEP § 2173.05(d).
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claims 1-4, 6-13, and 15-21, are rejected under 35 U.S.C. 102(a)(1) as being anticipated by BURKE (US 20220193541 A1), hereinafter referenced as BURKE.
Regarding claim 1, BURKE teaches an image frame display method (Figs. 2A-2B, 4-6, and 11, Paragraph [0054]-BURKE discloses as used herein, a video game gameplay stream (herein also referred to as an encoded gameplay stream, gameplay stream, and so on) refers to a stream comprising encoded image frames generated by a video game. The video game gameplay stream may further include encoded audio. For example, a video gameplay stream may be an MPEG-4 video which, when decoded, represents display data and audio data generated by a video game. Optionally, a video game gameplay stream may comprise encoded image frames of either user interface elements or gameplay elements. That is, a multitude of video game gameplay streams may be generated, and a user device may decode and combine each stream for presentation to a user of the user device. Please also read Paragraph [0051].) performed by a computer device (Figs. 1, 4-6, and 11, #1100 called a user computing system, Paragraph [0060]-BURKE discloses thus, the image frame may be presented locally via a display of a user device, or may be provided for presentation over a network (e.g., the internet) to a user device. Further in Paragraph [0165]-BURKE discloses display output signals may be produced by the display I/O 36 and can include signals for displaying visual content produced by the computing system 1100 on a display device, such as graphics, user interfaces, video, and/or other visual content. Please also read Paragraph [0068].) and the method comprising:
receiving a first rendering instruction (Figs. 2A-6, Paragraph [0081]-BURKE discloses a first image frame 112 represents the user-interface elements included in the video game image scene 10. Further in Paragraph [0176]-BURKE discloses any process descriptions, elements or blocks in the flow diagrams described herein and/or depicted in the attached figures should be understood as potentially representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or elements in the process.) transmitted by a server (Figs. 1 and 4-8B, illustrates transmitted by a server, #100 called a Stream Optimization System, Paragraph [0064]-BURKE discloses the stream optimization system 100 (wherein the stream optimization system is a server) includes a game engine 110 that can execute a video game, respond to user input 152 received from a the user device 150, and present rendered output from the video game to a stream encoding engine 120. As described above, the stream optimization system 100 may receive a request from the user device 150 to execute a particular video game. The stream optimization system 100 may then run the particular video game, for example emulate a software environment in which the particular video game would normally run. Further in Paragraph [0071]-BURKE discloses as will be described, the stream encoding engine 120 can provide an encoded gameplay stream 122 to the user device 150 for presentation on the user device 150. Please also read Paragraphs [0045] and [0069].);
rendering at least one first image element (Figs. 1-6, Paragraph [0061]-BURKE discloses the user interface elements 12A-12C may be overlaid on the non-user interface elements 14 and may optionally be rendered more simply.) based on the first rendering instruction (Figs. 1-6, Paragraph [0064]-BURKE discloses for example, user interface element 12A (e.g., a user's place in a car race) may be rendered as one or more colors. That is, this user interface element 12A may be designed to be easily legible by a user of the video game. Similarly, user interface element 12C may be designed to be easily decipherable to indicate a speed at which the user's car is traveling. For example, the user interface element 12C may be designed to be distinguishable from the non-user interface elements 14 that are proximate to the user interface element 12C. Further in Paragraph [0081]-BURKE discloses a first image frame 112 (wherein a first image frame 112 includes first image elements) represents the user-interface elements included in the video game image scene 10. Please also read Paragraphs [0069] and [0176].);
receiving at least one second image element transmitted by the server (Figs. 1-6, illustrates transmitted and rendered by the server, Paragraph [0045]-BURKE discloses an example video game stream may include encoded video generated from image frames rendered by a video game. For example, a video game may generate display data, such as successive image frames (e.g., 30, 45, 60, times per second), and this display data may be encoded. Example encoders can include H.264, MPEG-4, High Efficiency Video Coding (HEVC), VP8, VP9, and so on. Thus, the display data generated by the video game may be encoded to package the display data into information suitable for transmission over a network. Additionally, the display data may be encoded to reduce an amount of information which is being provided over the network (e.g., 3, 5, 15, 20, megabytes per second and so on). Thus, a burden on a bandwidth available to a user device may be reduced via the encoding. Please also read Paragraph [0061].), the at least one second image element being rendered by the server (Figs. 1-6, Paragraph [0064]-BURKE discloses as will be described in more detail below, a system described herein (e.g., the goal optimization system 100) (wherein goal optimization system is the server) may generate a video game gameplay stream to be provided to a user device, and enable remote play of the video game by a user of the user device. To ensure that the user device has sufficient available bandwidth to timely download the video gameplay stream, and provide user input to the system, the system can compress the video game gameplay stream. As an example, the video game scene 10 illustrated in FIG. 1 may be included in a video game gameplay stream. The system may compress the video game scene 10 during encoding of the video game scene 10 into the video game gameplay stream. As described above, the non-user interface elements 14 may include realistic detail, and may be rendered at a high resolution. Further in Paragraph [0081]-BURKE discloses a second image frame 114 (wherein a second image frame 114 includes second image elements) represents the gameplay elements included the video game image scene 10. Please also read Paragraph [0069].);
and displaying an image frame (Figs. 1-6, Paragraph [0164]-BURKE discloses user I/O 34 is used to send and receive commands between processing unit 1102 and user devices, such as game controllers. In some embodiments, the user I/O 34 can include touchscreen inputs. As previously described, the touchscreen can be a capacitive touchscreen, a resistive touchscreen, or other type of touchscreen technology that is configured to receive user input through tactile inputs from the user. Display I/O 36 provides input/output functions that are used to display images from the game being played. Network I/O 38 is used for input/output functions for a network. Network I/O 38 may be used during execution of a game, such as when a game is being played online or being accessed online.) based on the at least one first image element and the at least one second image element (Figs. 2A-6, illustrates transmitted by a server, Paragraph [0081]-BURKE discloses as illustrated in FIG. 1, the user interface elements 12A-12C are thus included as an overlay on the gameplay elements. Thus, a combination of the first image frame 112 overlaid on the second image frame 114 can result in the video game image scene 10. These image frames 112, 114, may be combined (e.g., packaged) by the stream encoding engine 120 and utilized to generate the encoded gameplay stream 122. Optionally, the image frames 112, 114, may be provided as separate encoded gameplay streams to the user device 150. In this example, the user device 150 may combine the received streams. Please also read Paragraph [0180].).
Regarding claim 2, BURKE teaches the method according to claim 1, BURKE further teaches further comprising:
obtaining an interactive instruction indicating a display mode (Figs. 1-2C, Paragraph [0041]-BURKE discloses the users may receive streaming video of real-time video game gameplay generated by the system, and may provide user input to control features of the streamed video. As an example, a user may utilize a user device to stream a role-playing video game. The user may provide user input to the user device, which can forward the user input to the system to update the role-playing video game. Further in Paragraph [0176]-BURKE discloses any process descriptions, elements or blocks in the flow diagrams described herein and/or depicted in the attached figures should be understood as potentially representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or elements in the process.) of the at least one first image element and the at least one second image element (Fig. 1-2C, Paragraph [0041]-BURKE discloses for example, the system can ensure that user interface elements (e.g., text, menus, and so on) (wherein user interface elements are at least one first image element) are presented in high resolution (e.g., a resolution natively rendered by the video game). In this example, the system may adjust quality of non-user interface elements, such as the video game gameplay itself (e.g., characters, environments, and so on). That is, the system may adaptively adjust a bitrate of the portions of the streamed video related to these non-user interface elements (wherein non-user interface elements are at least one second image element) while preserving the bitrate of the user interface elements. In this way, the user's perception of the reduction in quality may be reduced as any text, menus, and so on, may be appear in high quality. Please also read Paragraphs [0061] and [0081].);
and combining the at least one first image element and the at least one second image element in accordance with the interactive instruction to form the image frame (Figs. 1-2C, Paragraph [0041]-BURKE discloses while the user interface elements (wherein user interface elements are at least one first image element) may be included in the streamed video at high quality, due to the techniques described herein a size associated with the streamed video (e.g., measured in megabytes per second) may be conserved. As described herein, the system can cause non-user interface elements occluded by, or behind, user interface elements (wherein occluded by, or behind is combining) to be reduced in quality in the streamed video. Since these non-user interface elements (wherein non-user interface elements are at least one second image element) may be partially hidden, or otherwise adjusted in appearance by, the user interface elements, the system can reduce their quality as compared to other non-user interface elements. For example, a user viewing the streamed video may have a greater difficulty identifying specific details of a video game character partially covered by a user interface element. Thus, the system can encode these non-user interface elements to be more aggressively compressed or otherwise reduced in size. Please also read Paragraphs [0061] and [0081].).
Regarding claim 3, BURKE teaches the method according to claim 1, BURKE further teaches wherein the receiving an interactive instruction transmitted by the server (Figs. 1-2C and 7, Paragraph [0070]-BURKE discloses the stream optimization system 100 includes a game engine 110 that can execute a video game, respond to user input 152 (wherein user input to execute a game is an interactive instruction) received from a the user device 150, and present rendered output from the video game to a stream encoding engine 120. Further in Paragraph [0072]-BURKE discloses the stream optimization system 100 may provide the encoded gameplay stream 122 to the user device 150 utilizing one or more network protocols. Please also read Paragraphs [0057] and [0176].) comprises:
receiving, from the server (Figs. 1 and 7, #100 called a stream optimization system, Paragraph [0070]), a first interactive instruction (Figs. 1-2C and 7, Paragraph [0070]-BURKE discloses the stream optimization system 100 includes a game engine 110 that can execute a video game, respond to user input 152 (wherein user input to execute a game is a first interactive instruction) received from a the user device 150, and present rendered output from the video game to a stream encoding engine 120. Further in Paragraph [0071]-BURKE discloses as will be described, the stream encoding engine 120 can provide an encoded gameplay stream 122 to the user device 150 for presentation on the user device 150. Please also read Paragraphs [0057] and [0176].) that corresponds to the first image element (Figs. 1-2C and 7, Paragraph [0047]-BURKE discloses for example, in a particular video game (e.g., a role-playing game), text may be presented. The system can separately encode this text (wherein text is the first image element) as compared to video game gameplay (e.g., non-user interface elements). Since this text may be displayed for a threshold amount of time to afford time to read the text (e.g., 2 seconds, 3 seconds, and so on), the system may provide the text once and then indicate that a receiving user device is to present the text until instructed otherwise. Please also read Paragraph [0081].);
and receiving, from the sever (Figs. 1 and 7, #100 called a stream optimization system, Paragraph [0070]), a second interactive instruction (Figs. 1-2C and 7, Paragraph [0071]-BURKE discloses as will be described, the stream encoding engine 120 can provide an encoded gameplay stream 122 to the user device 150 for presentation on the user device 150. In this way, a user of the user device 150 can view gameplay generated by a video game. The user device 150 may receive user input from the user, for example via a dedicated controller, via a touch-screen interface (e.g., on a mobile device), and so on. The user device 150 may provide the user input 152 to the stream optimization system 100 for processing. For example, the game engine 110 can receive the user input 152, and utilize the user input as would ordinarily be utilized (e.g., if the user was locally playing the video game) (wherein playing the video game is a second interactive instruction). The game engine 110 may update state information of the video game, such as player location, game environment information, and so on. In response, the game engine 110 may update display data generated by execution of the video game. This updated display data may be provided to the stream encoding engine 120, which can provide the encoded gameplay stream 122 to the user device. Please also read Paragraphs [0057] and [0176].) that corresponds to the second image element (Figs. 1-2C and 7, Paragraph [0053]-BURKE discloses as used herein, gameplay elements (wherein gameplay action is the second image element) may include elements included in image frames rendered by a video game that are related to gameplay action. For example, user interface elements may be overlaid on the gameplay action and may be enable adjustments to the gameplay action or provide information describing aspects of the gameplay action. Example gameplay elements may include characters, non-playable characters, a game world or environment, and so on. Optionally, these gameplay elements may be modified or adjusted by the video game at a greater frequency than the user interface elements. For example, an environment may be adjusted as a user controlled character moves about a game world. Please also read Paragraph [0081].).
Regarding claim 4, BURKE teaches the method according to claim 3, BURKE further teaches wherein the displaying an image frame based on the at least one first image element, the at least one second image element (Figs. 2A-6, illustrates transmitted by a server, Paragraph [0081]-BURKE discloses as illustrated in FIG. 1, the user interface elements 12A-12C are thus included as an overlay on the gameplay elements. Thus, a combination of the first image frame 112 overlaid on the second image frame 114 can result in the video game image scene 10. These image frames 112, 114, may be combined (e.g., packaged) by the stream encoding engine 120 and utilized to generate the encoded gameplay stream 122. Optionally, the image frames 112, 114, may be provided as separate encoded gameplay streams to the user device 150. In this example, the user device 150 may combine the received streams. Please also read Paragraph [0180].), and the interactive instruction (Figs. 1-2C and 7, Paragraph [0070]-BURKE discloses the stream optimization system 100 includes a game engine 110 that can execute a video game, respond to user input 152 (wherein user input to execute a game is an interactive instruction) received from a the user device 150, and present rendered output from the video game to a stream encoding engine 120. Further in Paragraph [0072]-BURKE discloses the stream optimization system 100 may provide the encoded gameplay stream 122 to the user device 150 utilizing one or more network protocols. Please also read Paragraphs [0057] and [0176].) comprises:
determining a display mode of the first image element and the second image element (Figs. 1-6, Paragraph [0099]-BURKE discloses the system may obtain an image frame that includes the user interface elements (wherein image frame that includes user interface elements is first image element), and a different image frame that includes the gameplay elements (wherein image frame that includes game play elements is second image element). Further in Paragraph [0116]-BURKE discloses for example, the user interface element stream may include pixel information, such as color information and transparency information. The user device can utilize this information to overlay (e.g., blend) the user interface elements on top of the gameplay elements. Once combined, the user device can present the combination to the user. Optionally, the user device may select from among one or more streams generated by the remote system according to a bandwidth available to the user device. For example, as a network speed reduces, the user device may select a gameplay element stream generated using a lower bitrate. In this example, the received user interface element stream may still be of a high bitrate (e.g., as described above). Please also read Paragraph [0071].) based on the first interactive instruction and the second interactive instruction (Figs. 1-6, Paragraph [0072]-BURKE discloses to implement this technique, the stream optimization system 100 can detect a bandwidth available to the user device 150, and adjust a quality of the encoded gameplay stream 122 (e.g., in substantially real-time). The stream optimization system 100 can encode the stream 122 according to different bitrates, and the user device 150 may select from these different encodings depending on its available resources (e.g., bandwidth) (wherein user device may select is the first interactive instruction). Further in Paragraph [0111]-BURKE discloses if the system determines that there has been no update to the user interface elements, the system can utilize the stored image frame and prior mask information to adjust quality of the current gameplay elements. Similarly, if the system determines that there has been an update (wherein update is the second interactive instruction), the system can utilize the current user interface elements and mask information as described in FIGS. 2C and 4.);
and displaying the at least one first image element and the at least one second image element according to the display mode of the first image element and the second image element (Figs. 1-6, Paragraph [0116]-BURKE discloses for example, the user interface element stream may include pixel information, such as color information and transparency information. The user device can utilize this information to overlay (e.g., blend) the user interface elements on top of the gameplay elements. Once combined, the user device can present the combination to the user. Please also read Paragraph [0102] and [0105].).
Regarding claim 6, BURKE teaches the method according to claim 4, BURKE further teaches wherein, the display mode is transparency synthesis display (Figs. 1-6, Paragraph [0116]-BURKE discloses the user device can decode the streams (e.g., using a software or hardware decoder), and can combine the streams. For example, the user interface element stream may include pixel information, such as color information and transparency information. The user device can utilize this information to overlay (e.g., blend) the user interface elements on top of the gameplay elements. Once combined, the user device can present the combination to the user.), the first interactive instruction comprises a first interaction parameter (Figs. 1-6, Paragraph [0090]-BURKE discloses FIG. 2C illustrates first mask information 116A and second mask information 116B. As described above, the mask information 116A can indicate a compression or reduction in quality to be applied to portions of image frame 114 which includes the gameplay elements (wherein first mask information is a first interactive instruction). Further in Paragraph [0091]-BURKE discloses the mask information 116A indicates a measure of how affected image frame 114 will be once the user interface elements are overlaid when presented on the user device. To indicate the measures, the mask information 116A may graphically depict image frame 112 modified to adjust portions of each user interface element. For example, each pixel may be assigned a particular a color, such as a gray scale color, with darker colors representing a greater measure. Thus, a portion of image frame 114 that is under a darkly colored user interface element will be more hidden, or affected, than a portion of image frame 114 that is under a lighter color. Please also Read Paragraph [0176].), the second interactive instruction comprises a second interaction parameter (Figs. 1-6, Paragraph [0090]-BURKE discloses FIG. 2C illustrates first mask information 116A and second mask information 116B. As described above, the mask information 116A can indicate a compression or reduction in quality to be applied to portions of image frame 114 which includes the gameplay elements (wherein second mask information is a second interactive instruction). Further in Paragraph [0090]-BURKE discloses the mask information 116B can indicate a compression or reduction in quality to be applied to portions of image frame 112. Further in Paragraph [0092]-BURKE discloses the mask information 116A further includes user interface element 12B. As illustrated, user interface element 12B includes two portions 236, 238, which are colored differently. Portion 236 is a particular color associated with hiding gameplay elements (e.g., black, as described above). Thus, this portion 236 may appear as opaque when presented to a user. In contrast, portion 238 is a different color (e.g., a shade of gray). Therefore, this portion 238 may be visible to the user of the user device 150 but may be less legible than other gameplay elements. The portion 238 may appear as slightly transparent, translucent, or with a particular effect applied to it (e.g., an appearance of water or glass). Please also Read Paragraph [0176].), and the first interaction parameter and the second interaction parameter comprise transparency information of the first image element and the second image element, respectively (Figs. 1-6, Paragraph [0063]-BURKE discloses user interface element 12B may represent a location of the car in a game world of the video game. This user interface element 12B may similarly be distinguishable from underlying non-user interface elements 14. Optionally, the user interface element 12B may, at least in part, be rendered as transparent (wherein rendered transparent is the display mode). For example, portion 18A may represent a track on which the car is driving, and may include a representation of the user's car. As another example, portion 18B may represent game world areas proximate to the track. Portion 18A may optionally be rendered as substantially opaque, while portion 18B may be at least partially transparent. In this way, the non-user interface elements 14 under this portion 18B may be partially visible. Please also read Paragraphs [0061] and [0093].);
and the displaying the at least one first image element and the at least one second image element according to the display mode of the first image element and the second image element (Figs. 1-6, Paragraph [0116]-BURKE discloses for example, the user interface element stream may include pixel information, such as color information and transparency information. The user device can utilize this information to overlay (e.g., blend) the user interface elements on top of the gameplay elements. Once combined, the user device can present the combination to the user. Please also read Paragraphs [0102] and [0105].) comprises:
determining transparency of the at least one first image element and the at least one second image element based on the transparency information of the at least one first image element and the at least one second image element (Figs. 1-6, Paragraph [0063]-BURKE discloses user interface element 12B may represent a location of the car in a game world of the video game. This user interface element 12B may similarly be distinguishable from underlying non-user interface elements 14. Optionally, the user interface element 12B may, at least in part, be rendered as transparent (wherein rendered transparent is the display mode is a transparency synthesis display). For example, portion 18A may represent a track on which the car is driving, and may include a representation of the user's car. As another example, portion 18B may represent game world areas proximate to the track. Portion 18A may optionally be rendered as substantially opaque, while portion 18B may be at least partially transparent. In this way, the non-user interface elements 14 under this portion 18B may be partially visible. Please also read Paragraph [0093].);
and performing transparency synthesis display on the at least one first image element and the at least one second image element (Figs. 1-6, Paragraph [0063]-BURKE discloses as illustrated, portion 18A of the user interface element 12B is rendered as substantially opaque. Therefore, the portion of the race track 16B under the portion 18A is not visible in the video game scene 10. In contrast, portion 18B of the user interface element 12B is partially transparent. Thus, the portion of the race track 16B under portion 18B is partially visible in the example of FIG. 1. For example, particular effects may be applied (e.g., shader effects) that modify an appearance of the portion of the race track 16B.) based on the transparency of the at least one first image element and the at least one second image element (Figs. 1-6, Paragraph [0063]-BURKE discloses as described above, the non-user interface elements 14 may include realistic detail, and may be rendered at a high resolution. Thus, in the video game gameplay stream, a threshold amount of information included in the stream (e.g., 70%, 80%, 90%) may be associated with the non-user interface elements 14. In contrast, and as described above, the user interface elements 12A-12C may be rendered to be easily legible by a user (wherein legible by a user is based on the transparency of the elements). While these elements may also be rendered at a same high resolution, the elements 14 will represent 30%, 20%, 10%, and so on, of information included in the stream. Further in Paragraph [0065]-BURKE discloses for example, the system may generate different video game gameplay streams, with each stream including the non-user interface elements 14 rendered at a respective bitrate. As the bandwidth available to a user device fluctuates, a different one of these streams may be selected to be provided to the user device. Since each stream may include the user interface elements 12A-12C rendered in high quality, the user will be assured to easily decipher the user interface elements 12A-12C.).
Regarding claim 7, BURKE teaches the method according to claim 4, BURKE further teaches wherein, the display mode is separate display (Figs. 1-6, Paragraph [0116]-BURKE discloses Optionally, the user device may select from among one or more streams generated by the remote system according to a bandwidth available to the user device. (wherein selecting among one or more streams is the display mode is separate display). Please also read Paragraph [0164].), the displaying the at least one first image element and the at least one second image element according to the display mode of the first image element and the second image element (Figs. 1-6, Paragraph [0101]-BURKE discloses the system obtains information separating user interface elements from gameplay elements. As the video game executes, the system can obtain output rendered by the video game. As described above, the video game may render elements using draw calls and then update (e.g., paint) the elements onto an image frame. The system may obtain an image frame that includes the user interface elements, and a different image frame that includes the gameplay elements. Further in Paragraph [0109]-BURKE discloses since the user interface elements may not update as often as gameplay elements, for example a health bar or a menu may be included in rendered image frames for several seconds without being updated in appearance by the video game. The system may therefore reduce a size of the stream by excluding user interface elements from the gameplay stream until they change (wherein excluding user interface elements from the gameplay stream until they change is where the display mode is a separate display).) comprises:
separately displaying the at least one first image element and the at least one second image element (Figs. 1-6, Paragraph [0116]-BURKE discloses the system receives one or more gameplay streams from the remote system. As described above, the user device may receive a first stream that includes gameplay elements, and a second stream that includes user interface elements. The user device can decode the streams (e.g., using a software or hardware decoder), and can combine the streams. For example, the user interface element stream may include pixel information, such as color information and transparency information. The user device can utilize this information to overlay (e.g., blend) the user interface elements on top of the gameplay elements. Once combined, the user device can present the combination to the user. Optionally, the user device may select from among one or more streams generated by the remote system according to a bandwidth available to the user device. Please also read Paragraph [0164].).
Regarding claim 8, BURKE teaches the method according to claim 1, BURKE further teaches wherein the rendering the at least one first image element (Figs. 1-6, Paragraph [0061]-BURKE discloses the user interface elements 12A-12C may be overlaid on the non-user interface elements 14 and may optionally be rendered more simply.) based on the first rendering instruction (Figs. 1-6, Paragraph [0064]-BURKE discloses for example, user interface element 12A (e.g., a user's place in a car race) may be rendered as one or more colors. That is, this user interface element 12A may be designed to be easily legible by a user of the video game. Similarly, user interface element 12C may be designed to be easily decipherable to indicate a speed at which the user's car is traveling. For example, the user interface element 12C may be designed to be distinguishable from the non-user interface elements 14 that are proximate to the user interface element 12C. Further in Paragraph [0081]-BURKE discloses a first image frame 112 (wherein a first image frame 112 includes first image elements) represents the user-interface elements included in the video game image scene 10. Please also read Paragraphs [0069] and [0176].) comprises:
obtaining a rendering function name (Figs. 1-6, Paragraph [0082]-BURKE discloses to generate the rendered image frames 112, 114, the game engine 110 may execute a version of the video game which has been customized. For example, when rendering a particular image frame (e.g., the video game scene 10), the video game may perform multitudes of draw calls (to render portions of the particular image frame. Example draw calls may include rendering characters, environments, a crowd in a sports game, and so on. Please also refer to Paragraph [0167].) comprised in the first rendering instruction (Figs. 1-6, Paragraph [0082]-BURKE discloses the stream encoding engine 120 may generate two or more gameplay streams. In this example, the stream encoding engine 120 may encode image frame 114 and include this encoded image frame in a first gameplay stream 122 directed to gameplay elements. When the video game draws user interface elements (e.g., as described above), the stream encoding engine 120 may switch to a second stream and included image frame 112 in this second stream. The user device 150 may receive these streams, and combine the streams. For example, the user device 150 may overlay decoded user interface image frames on corresponding decoded gameplay image frames. Please also read Paragraph [0176].), and related parameters used during rendering the at least one first image element (Figs. 1-6, Paragraph [0063]-BURKE discloses user interface element 12B may represent a location of the car in a game world of the video game. This user interface element 12B may similarly be distinguishable from underlying non-user interface elements 14. Optionally, the user interface element 12B may, at least in part, be rendered as transparent. For example, portion 18A may represent a track on which the car is driving, and may include a representation of the user's car. As another example, portion 18B may represent game world areas proximate to the track. Portion 18A may optionally be rendered as substantially opaque, while portion 18B may be at least partially transparent. In this way, the non-user interface elements 14 under this portion 18B may be partially visible. Please also read Paragraph [0056].);
and calling, based on the rendering function name (Figs. 1-6, Paragraph [0013]-BURKE discloses the server system described herein may execute a video game, and utilize metadata generated by the video game during gameplay to identify portions of rendered output (e.g., image frames) that are (1) video game gameplay or (2) user interface elements. For example, when rendering an image frame, the video game may utilize specific draw calls to render portions of the image frame.), a function interface corresponding to the rendering function name (Figs. 1-6, Paragraph [0101]-BURKE discloses the video game may render elements using draw calls (wherein draw is the rendering function name) and then update (e.g., paint) the elements onto an image frame.), so as to render the at least one first image element through the function interface and the related parameters (Figs. 1-6, Paragraph [0101]-BURKE discloses the system obtains information separating user interface elements from gameplay elements. As the video game executes, the system can obtain output rendered by the video game. As described above, the video game may render elements using draw calls and then update (e.g., paint) the elements onto an image frame. The system may obtain an image frame that includes the user interface elements, and a different image frame that includes the gameplay elements.).
Regarding claim 9, BURKE teaches the method according to claim 1, BURKE further teaches wherein, the image frame is a virtual scene picture (Fig. 1, #10 called a video game scene, Paragraph [0060]-BURKE discloses FIG. 1 illustrates an example video game scene 10 of a video game. The video game scene 10 may be an example of an image frame generated by a video game for presentation to a user of the video game.), the first image element (Figs. 1-2C, Paragraph [0081]-BURKE discloses a first image frame 112 represents the user-interface elements (wherein user-interface elements is the first image element) included in the video game image scene 10.) comprises at least one of an icon (Figs. 1-2C, illustrates one of an icon, Paragraph [0041]-BURKE discloses for example, the system can ensure that user interface elements (e.g., text, menus, and so on) are presented in high resolution (e.g., a resolution natively rendered by the video game). Further in Paragraph [0061]-BURKE discloses the example gameplay elements may be separated according to user interface elements 12A-12C (wherein 12C is an icon), and non-user interface elements 14. The non-user interface elements 14 may include gameplay elements 16A-16B.), a graphic button of a virtual control (Figs. 1-2C, Paragraph [0074]-BURKE discloses for example, the mobile device may present a representation of user input commands (e.g., an ‘A’ button, a triangle command, and so on). The user may interact with this representation, and the game application 160 may provide information to the stream optimization system 100 identifying the type of command selected based on the interaction. Please also read Paragraphs [0041] and [0061].), and a graphic comprising text content (Figs. 1-2C, illustrates a graphic comprising text content, Paragraph [0061]-BURKE discloses the example gameplay elements may be separated according to user interface elements 12A-12C (wherein 12C is a graphic comprising text content), and non-user interface elements 14. The non-user interface elements 14 may include gameplay elements 16A-16B. Please also read Paragraph [0096].), superimposed on the virtual scene picture (Figs. 1-2C, illustrates the virtual scene picture, Paragraph [0062]-BURKE discloses the user interface elements 12A-12C may be overlaid on the non-user interface elements 14 and may optionally be rendered more simply.);
and the second image element (Figs. 1-6, Paragraph [0061]-BURKE discloses as illustrated in FIG. 1, the video game scene 10 includes gameplay elements of the video game, which in the example is a car racing video game. The example gameplay elements may be separated according to user interface elements 12A-12C, and non-user interface elements 14. The non-user interface elements 14 may include gameplay elements 16A-16B (wherein non-user interface elements are the second image element). Further in Paragraph [0081]-BURKE discloses a second image frame 114 represents the gameplay elements included the video game image scene 10.) comprises an image used for displaying the virtual scene in the virtual scene picture (Figs. 1-6, illustrates a virtual scene picture, Paragraph [0061]-BURKE discloses the non-user interface elements 14 may include gameplay elements 16A-16B. As an example, gameplay element 16A may represent a car being controlled by a user of the video game. As another example, gameplay element 16B may represent a game world or environment in which the car is being driven (e.g., a race track). (wherein 16B is an image used for displaying the virtual scene).
Regarding claim 10, BURKE teaches a computer device (Figs. 1, 4-6, and 11, #1100 called a user computing system, Paragraph [0161]), comprising a processor (Figs. 1 and 11, #1102 called a processing unit, Paragraph [0161]-BURKE discloses as shown, the computing system 1100 includes a processing unit 1102 that interacts with other components of the computing system 1100 and also components external to the computing system 1100.) and a memory (Figs. 1 and 11, #1103 called a storage, Paragraph [0163]-BURKE discloses processing unit 1102 can communicate through I/O 32 to store data, such as game state data and any shared data files. In addition to storage 1103 and removable storage media 44, the computing system 1100 is also shown including ROM (Read-Only Memory) 46 and RAM 48.), the memory storing at least one computer instruction (Figs. 1 and 11, #1103 called a storage, Paragraph [0167]-BURKE discloses program code can be stored in ROM 46, RAM 48, or storage 1103 (which might comprise hard disk, other magnetic storage, optical storage, solid state drives, and/or other non-volatile storage, or a combination or variation of these).), and the at least one computer instruction (Figs. 3-6 and 9-11, Paragraph [0176]-BURKE discloses any process descriptions, elements or blocks in the flow diagrams described herein and/or depicted in the attached figures should be understood as potentially representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or elements in the process.), when executed by the processor, causing the computer device to implement an image frame display method (Figs. 2A-2B, 4-6, 11, #1103 called a storage, Paragraph [0164]-BURKE discloses user I/O 34 is used to send and receive commands between processing unit 1102 and user devices, such as game controllers. In some embodiments, the user I/O 34 can include touchscreen inputs. Further in Paragraph [0164]-BURKE discloses display I/O 36 provides input/output functions that are used to display images from the game being played. Network I/O 38 is used for input/output functions for a network. Network I/O 38 may be used during execution of a game, such as when a game is being played online or being accessed online. Please also read Paragraph [0180].) including:
receiving a first rendering instruction (Figs. 2A-6, Paragraph [0081]-BURKE discloses a first image frame 112 represents the user-interface elements included in the video game image scene 10. Further in Paragraph [0176]-BURKE discloses any process descriptions, elements or blocks in the flow diagrams described herein and/or depicted in the attached figures should be understood as potentially representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or elements in the process. Further in Paragraph [0176]-BURKE discloses any process descriptions, elements or blocks in the flow diagrams described herein and/or depicted in the attached figures should be understood as potentially representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or elements in the process.) transmitted by a server (Figs. 1 and 4-8B, illustrates transmitted by a server, #100 called a Stream Optimization System, Paragraph [0064]-BURKE discloses the stream optimization system 100 (wherein the stream optimization system is a server) includes a game engine 110 that can execute a video game, respond to user input 152 received from a the user device 150, and present rendered output from the video game to a stream encoding engine 120. As described above, the stream optimization system 100 may receive a request from the user device 150 to execute a particular video game. The stream optimization system 100 may then run the particular video game, for example emulate a software environment in which the particular video game would normally run. Further in Paragraph [0071]-BURKE discloses as will be described, the stream encoding engine 120 can provide an encoded gameplay stream 122 to the user device 150 for presentation on the user device 150. Please also read Paragraphs [0045] and [0069].);
rendering at least one first image element (Figs. 1-6, Paragraph [0061]-BURKE discloses the user interface elements 12A-12C may be overlaid on the non-user interface elements 14 and may optionally be rendered more simply.) based on the first rendering instruction (Figs. 1-6, Paragraph [0064]-BURKE discloses for example, user interface element 12A (e.g., a user's place in a car race) may be rendered as one or more colors. That is, this user interface element 12A may be designed to be easily legible by a user of the video game. Similarly, user interface element 12C may be designed to be easily decipherable to indicate a speed at which the user's car is traveling. For example, the user interface element 12C may be designed to be distinguishable from the non-user interface elements 14 that are proximate to the user interface element 12C. Further in Paragraph [0081]-BURKE discloses a first image frame 112 (wherein a first image frame 112 includes first image elements) represents the user-interface elements included in the video game image scene 10. Please also read Paragraphs [0069] and [0176].);
receiving at least one second image element transmitted by the server (Figs. 1-6, illustrates transmitted and rendered by the server, Paragraph [0045]-BURKE discloses an example video game stream may include encoded video generated from image frames rendered by a video game. For example, a video game may generate display data, such as successive image frames (e.g., 30, 45, 60, times per second), and this display data may be encoded. Example encoders can include H.264, MPEG-4, High Efficiency Video Coding (HEVC), VP8, VP9, and so on. Thus, the display data generated by the video game may be encoded to package the display data into information suitable for transmission over a network. Additionally, the display data may be encoded to reduce an amount of information which is being provided over the network (e.g., 3, 5, 15, 20, megabytes per second and so on). Thus, a burden on a bandwidth available to a user device may be reduced via the encoding. Please also read Paragraph [0061].), the at least one second image element being rendered by the server (Figs. 1-6, illustrates transmitted and rendered by the server, Paragraph [0064]-BURKE discloses as will be described in more detail below, a system described herein (e.g., the goal optimization system 100) (wherein goal optimization system is the server) may generate a video game gameplay stream to be provided to a user device, and enable remote play of the video game by a user of the user device. To ensure that the user device has sufficient available bandwidth to timely download the video gameplay stream, and provide user input to the system, the system can compress the video game gameplay stream. As an example, the video game scene 10 illustrated in FIG. 1 may be included in a video game gameplay stream. The system may compress the video game scene 10 during encoding of the video game scene 10 into the video game gameplay stream. As described above, the non-user interface elements 14 may include realistic detail, and may be rendered at a high resolution. Further in Paragraph [0081]-BURKE discloses a second image frame 114 (wherein a second image frame 114 includes second image elements) represents the gameplay elements included the video game image scene 10. Please also read Paragraphs [0045] and [0069].);
and displaying an image frame (Figs. 1-6, Paragraph [0164]-BURKE discloses user I/O 34 is used to send and receive commands between processing unit 1102 and user devices, such as game controllers. In some embodiments, the user I/O 34 can include touchscreen inputs. As previously described, the touchscreen can be a capacitive touchscreen, a resistive touchscreen, or other type of touchscreen technology that is configured to receive user input through tactile inputs from the user. Display I/O 36 provides input/output functions that are used to display images from the game being played. Network I/O 38 is used for input/output functions for a network. Network I/O 38 may be used during execution of a game, such as when a game is being played online or being accessed online.) based on the at least one first image element and the at least one second image element (Figs. 2A-6, illustrates transmitted by a server, Paragraph [0081]-BURKE discloses as illustrated in FIG. 1, the user interface elements 12A-12C are thus included as an overlay on the gameplay elements. Thus, a combination of the first image frame 112 overlaid on the second image frame 114 can result in the video game image scene 10. These image frames 112, 114, may be combined (e.g., packaged) by the stream encoding engine 120 and utilized to generate the encoded gameplay stream 122. Optionally, the image frames 112, 114, may be provided as separate encoded gameplay streams to the user device 150. In this example, the user device 150 may combine the received streams. Please also read Paragraph [0180].).
Regarding claim 11, BURKE teaches the computer device according to claim 10, BURKE further teaches wherein the method further comprises:
obtaining an interactive instruction indicating a display mode (Figs. 1-2C, Paragraph [0041]-BURKE discloses the users may receive streaming video of real-time video game gameplay generated by the system, and may provide user input to control features of the streamed video. As an example, a user may utilize a user device to stream a role-playing video game. The user may provide user input to the user device, which can forward the user input to the system to update the role-playing video game. Further in Paragraph [0176]-BURKE discloses any process descriptions, elements or blocks in the flow diagrams described herein and/or depicted in the attached figures should be understood as potentially representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or elements in the process.) of the at least one first image element and the at least one second image element (Fig. 1-2C, Paragraph [0041]-BURKE discloses for example, the system can ensure that user interface elements (e.g., text, menus, and so on) (wherein user interface elements are at least one first image element) are presented in high resolution (e.g., a resolution natively rendered by the video game). In this example, the system may adjust quality of non-user interface elements, such as the video game gameplay itself (e.g., characters, environments, and so on). That is, the system may adaptively adjust a bitrate of the portions of the streamed video related to these non-user interface elements (wherein non-user interface elements are at least one second image element) while preserving the bitrate of the user interface elements. In this way, the user's perception of the reduction in quality may be reduced as any text, menus, and so on, may be appear in high quality. Please also read Paragraphs [0061] and [0081].);
and combining the at least one first image element and the at least one second image element in accordance with the interactive instruction to form the image frame (Figs. 1-2C, Paragraph [0041]-BURKE discloses while the user interface elements (wherein user interface elements are at least one first image element) may be included in the streamed video at high quality, due to the techniques described herein a size associated with the streamed video (e.g., measured in megabytes per second) may be conserved. As described herein, the system can cause non-user interface elements occluded by, or behind, user interface elements (wherein occluded by, or behind is combining) to be reduced in quality in the streamed video. Since these non-user interface elements (wherein non-user interface elements are at least one second image element) may be partially hidden, or otherwise adjusted in appearance by, the user interface elements, the system can reduce their quality as compared to other non-user interface elements. For example, a user viewing the streamed video may have a greater difficulty identifying specific details of a video game character partially covered by a user interface element. Thus, the system can encode these non-user interface elements to be more aggressively compressed or otherwise reduced in size. Please also read Paragraphs [0061] and [0081].).
Regarding claim 12, BURKE teaches the computer device according to claim 10, BURKE further teaches wherein the receiving an interactive instruction transmitted by the server (Figs. 1-2C and 7, Paragraph [0070]-BURKE discloses the stream optimization system 100 includes a game engine 110 that can execute a video game, respond to user input 152 (wherein user input to execute a game is an interactive instruction) received from a the user device 150, and present rendered output from the video game to a stream encoding engine 120. Further in Paragraph [0072]-BURKE discloses the stream optimization system 100 may provide the encoded gameplay stream 122 to the user device 150 utilizing one or more network protocols. Please also read Paragraphs [0057] and [0176].) comprises:
receiving, from the server (Figs. 1 and 7, #100 called a stream optimization system, Paragraph [0070]), a first interactive instruction (Figs. 1-2C and 7, Paragraph [0070]-BURKE discloses the stream optimization system 100 includes a game engine 110 that can execute a video game, respond to user input 152 (wherein user input to execute a game is a first interactive instruction) received from a the user device 150, and present rendered output from the video game to a stream encoding engine 120. Further in Paragraph [0071]-BURKE discloses as will be described, the stream encoding engine 120 can provide an encoded gameplay stream 122 to the user device 150 for presentation on the user device 150. Please also read Paragraphs [0057] and [0176].) that corresponds to the first image element (Figs. 1-2C and 7, Paragraph [0047]-BURKE discloses for example, in a particular video game (e.g., a role-playing game), text may be presented. The system can separately encode this text (wherein text is the first image element) as compared to video game gameplay (e.g., non-user interface elements). Since this text may be displayed for a threshold amount of time to afford time to read the text (e.g., 2 seconds, 3 seconds, and so on), the system may provide the text once and then indicate that a receiving user device is to present the text until instructed otherwise. Please also read Paragraph [0081].);
and receiving, from the sever (Figs. 1 and 7, #100 called a stream optimization system, Paragraph [0070]), a second interactive instruction (Figs. 1-2C and 7, Paragraph [0071]-BURKE discloses as will be described, the stream encoding engine 120 can provide an encoded gameplay stream 122 to the user device 150 for presentation on the user device 150. In this way, a user of the user device 150 can view gameplay generated by a video game. The user device 150 may receive user input from the user, for example via a dedicated controller, via a touch-screen interface (e.g., on a mobile device), and so on. The user device 150 may provide the user input 152 to the stream optimization system 100 for processing. For example, the game engine 110 can receive the user input 152, and utilize the user input as would ordinarily be utilized (e.g., if the user was locally playing the video game) (wherein playing the video game is a second interactive instruction). The game engine 110 may update state information of the video game, such as player location, game environment information, and so on. In response, the game engine 110 may update display data generated by execution of the video game. This updated display data may be provided to the stream encoding engine 120, which can provide the encoded gameplay stream 122 to the user device. Please also read Paragraphs [0057] and [0176].) that corresponds to the second image element (Figs. 1-2C and 7, Paragraph [0053]-BURKE discloses as used herein, gameplay elements (wherein gameplay action is the second image element) may include elements included in image frames rendered by a video game that are related to gameplay action. For example, user interface elements may be overlaid on the gameplay action and may be enable adjustments to the gameplay action or provide information describing aspects of the gameplay action. Example gameplay elements may include characters, non-playable characters, a game world or environment, and so on. Optionally, these gameplay elements may be modified or adjusted by the video game at a greater frequency than the user interface elements. For example, an environment may be adjusted as a user controlled character moves about a game world. Please also read Paragraph [0081].).
Regarding claim 13, BURKE teaches the computer device according to claim 11, BURKE further teaches wherein the displaying an image frame based on the at least one first image element, the at least one second image element (Figs. 2A-6, illustrates transmitted by a server, Paragraph [0081]-BURKE discloses as illustrated in FIG. 1, the user interface elements 12A-12C are thus included as an overlay on the gameplay elements. Thus, a combination of the first image frame 112 overlaid on the second image frame 114 can result in the video game image scene 10. These image frames 112, 114, may be combined (e.g., packaged) by the stream encoding engine 120 and utilized to generate the encoded gameplay stream 122. Optionally, the image frames 112, 114, may be provided as separate encoded gameplay streams to the user device 150. In this example, the user device 150 may combine the received streams. Please also read Paragraph [0180].), and the interactive instruction (Figs. 1-2C and 7, Paragraph [0070]-BURKE discloses the stream optimization system 100 includes a game engine 110 that can execute a video game, respond to user input 152 (wherein user input to execute a game is an interactive instruction) received from a the user device 150, and present rendered output from the video game to a stream encoding engine 120. Further in Paragraph [0072]-BURKE discloses the stream optimization system 100 may provide the encoded gameplay stream 122 to the user device 150 utilizing one or more network protocols. Please also read Paragraphs [0057] and [0176].) comprises:
determining a display mode of the first image element and the second image element (Figs. 1-6, Paragraph [0099]-BURKE discloses the system may obtain an image frame that includes the user interface elements (wherein image frame that includes user interface elements is first image element), and a different image frame that includes the gameplay elements (wherein image frame that includes game play elements is second image element). Further in Paragraph [0116]-BURKE discloses for example, the user interface element stream may include pixel information, such as color information and transparency information. The user device can utilize this information to overlay (e.g., blend) the user interface elements on top of the gameplay elements. Once combined, the user device can present the combination to the user. Optionally, the user device may select from among one or more streams generated by the remote system according to a bandwidth available to the user device. For example, as a network speed reduces, the user device may select a gameplay element stream generated using a lower bitrate. In this example, the received user interface element stream may still be of a high bitrate (e.g., as described above). Please also read Paragraph [0071].) based on the first interactive instruction and the second interactive instruction (Figs. 1-6, Paragraph [0072]-BURKE discloses to implement this technique, the stream optimization system 100 can detect a bandwidth available to the user device 150, and adjust a quality of the encoded gameplay stream 122 (e.g., in substantially real-time). The stream optimization system 100 can encode the stream 122 according to different bitrates, and the user device 150 may select from these different encodings depending on its available resources (e.g., bandwidth) (wherein user device may select is the first interactive instruction). Further in Paragraph [0111]-BURKE discloses if the system determines that there has been no update to the user interface elements, the system can utilize the stored image frame and prior mask information to adjust quality of the current gameplay elements. Similarly, if the system determines that there has been an update (wherein update is the second interactive instruction), the system can utilize the current user interface elements and mask information as described in FIGS. 2C and 4.);
and displaying the at least one first image element and the at least one second image element according to the display mode of the first image element and the second image element (Figs. 1-6, Paragraph [0116]-BURKE discloses for example, the user interface element stream may include pixel information, such as color information and transparency information. The user device can utilize this information to overlay (e.g., blend) the user interface elements on top of the gameplay elements. Once combined, the user device can present the combination to the user. Please also read Paragraph [0102] and [0105].).
Regarding claim 15, BURKE teaches the computer device according to claim 12, BURKE further teaches wherein, the display mode is transparency synthesis display (Figs. 1-6, Paragraph [0116]-BURKE discloses the user device can decode the streams (e.g., using a software or hardware decoder), and can combine the streams. For example, the user interface element stream may include pixel information, such as color information and transparency information. The user device can utilize this information to overlay (e.g., blend) the user interface elements on top of the gameplay elements. Once combined, the user device can present the combination to the user.), the first interactive instruction comprises a first interaction parameter (Figs. 1-6, Paragraph [0090]-BURKE discloses FIG. 2C illustrates first mask information 116A and second mask information 116B. As described above, the mask information 116A can indicate a compression or reduction in quality to be applied to portions of image frame 114 which includes the gameplay elements (wherein first mask information is a first interactive instruction). Further in Paragraph [0091]-BURKE discloses the mask information 116A indicates a measure of how affected image frame 114 will be once the user interface elements are overlaid when presented on the user device. To indicate the measures, the mask information 116A may graphically depict image frame 112 modified to adjust portions of each user interface element. For example, each pixel may be assigned a particular a color, such as a gray scale color, with darker colors representing a greater measure. Thus, a portion of image frame 114 that is under a darkly colored user interface element will be more hidden, or affected, than a portion of image frame 114 that is under a lighter color. Please also Read Paragraph [0176].), the second interactive instruction comprises a second interaction parameter (Figs. 1-6, Paragraph [0090]-BURKE discloses FIG. 2C illustrates first mask information 116A and second mask information 116B. As described above, the mask information 116A can indicate a compression or reduction in quality to be applied to portions of image frame 114 which includes the gameplay elements (wherein second mask information is a second interactive instruction). Further in Paragraph [0090]-BURKE discloses the mask information 116B can indicate a compression or reduction in quality to be applied to portions of image frame 112. Further in Paragraph [0092]-BURKE discloses the mask information 116A further includes user interface element 12B. As illustrated, user interface element 12B includes two portions 236, 238, which are colored differently. Portion 236 is a particular color associated with hiding gameplay elements (e.g., black, as described above). Thus, this portion 236 may appear as opaque when presented to a user. In contrast, portion 238 is a different color (e.g., a shade of gray). Therefore, this portion 238 may be visible to the user of the user device 150 but may be less legible than other gameplay elements. The portion 238 may appear as slightly transparent, translucent, or with a particular effect applied to it (e.g., an appearance of water or glass). Please also Read Paragraph [0176].), and the first interaction parameter and the second interaction parameter comprise transparency information of the first image element and the second image element, respectively (Figs. 1-6, Paragraph [0063]-BURKE discloses user interface element 12B may represent a location of the car in a game world of the video game. This user interface element 12B may similarly be distinguishable from underlying non-user interface elements 14. Optionally, the user interface element 12B may, at least in part, be rendered as transparent (wherein rendered transparent is the display mode). For example, portion 18A may represent a track on which the car is driving, and may include a representation of the user's car. As another example, portion 18B may represent game world areas proximate to the track. Portion 18A may optionally be rendered as substantially opaque, while portion 18B may be at least partially transparent. In this way, the non-user interface elements 14 under this portion 18B may be partially visible. Please also read Paragraphs [0061] and [0093].);
and the displaying (Figs. 1-6 and 11, Paragraph [0164]-BURKE discloses user I/O 34 is used to send and receive commands between processing unit 1102 and user devices, such as game controllers. In some embodiments, the user I/O 34 can include touchscreen inputs. As previously described, the touchscreen can be a capacitive touchscreen, a resistive touchscreen, or other type of touchscreen technology that is configured to receive user input through tactile inputs from the user. Display I/O 36 provides input/output functions that are used to display images from the game being played. Network I/O 38 is used for input/output functions for a network. Network I/O 38 may be used during execution of a game, such as when a game is being played online or being accessed online.) the at least one first image element and the at least one second image element according to the display mode of the first image element and the second image (Figs. 1-6, Paragraph [0116]-BURKE discloses for example, the user interface element stream may include pixel information, such as color information and transparency information. The user device can utilize this information to overlay (e.g., blend) the user interface elements on top of the gameplay elements. Once combined, the user device can present the combination to the user. Please also read Paragraph [0102] and [0105].) element comprises:
determining transparency of the at least one first image element and the at least one second image element based on the transparency information of the at least one first image element and the at least one second image element (Figs. 1-6, Paragraph [0063]-BURKE discloses user interface element 12B may represent a location of the car in a game world of the video game. This user interface element 12B may similarly be distinguishable from underlying non-user interface elements 14. Optionally, the user interface element 12B may, at least in part, be rendered as transparent (wherein rendered transparent is the display mode is a transparency synthesis display). For example, portion 18A may represent a track on which the car is driving, and may include a representation of the user's car. As another example, portion 18B may represent game world areas proximate to the track. Portion 18A may optionally be rendered as substantially opaque, while portion 18B may be at least partially transparent. In this way, the non-user interface elements 14 under this portion 18B may be partially visible. Please also read Paragraphs [0061] and [0093].);
and performing transparency synthesis display on the at least one first image element and the at least one second image element (Figs. 1-6, Paragraph [0063]-BURKE discloses as illustrated, portion 18A of the user interface element 12B is rendered as substantially opaque. Therefore, the portion of the race track 16B under the portion 18A is not visible in the video game scene 10. In contrast, portion 18B of the user interface element 12B is partially transparent. Thus, the portion of the race track 16B under portion 18B is partially visible in the example of FIG. 1. For example, particular effects may be applied (e.g., shader effects) that modify an appearance of the portion of the race track 16B.) based on the transparency of the at least one first image element and the at least one second image element (Figs. 1-6, Paragraph [0063]-BURKE discloses as described above, the non-user interface elements 14 may include realistic detail, and may be rendered at a high resolution. Thus, in the video game gameplay stream, a threshold amount of information included in the stream (e.g., 70%, 80%, 90%) may be associated with the non-user interface elements 14. In contrast, and as described above, the user interface elements 12A-12C may be rendered to be easily legible by a user (wherein legible by a user is based on the transparency of the elements). While these elements may also be rendered at a same high resolution, the elements 14 will represent 30%, 20%, 10%, and so on, of information included in the stream. Further in Paragraph [0065]-BURKE discloses For example, the system may generate different video game gameplay streams, with each stream including the non-user interface elements 14 rendered at a respective bitrate. As the bandwidth available to a user device fluctuates, a different one of these streams may be selected to be provided to the user device. Since each stream may include the user interface elements 12A-12C rendered in high quality, the user will be assured to easily decipher the user interface elements 12A-12C.).
Regarding claim 16, BURKE teaches the computer device according to claim 12, BURKE further teaches wherein, the display mode is separate display (Figs. 1-6, Paragraph [0116]-BURKE discloses Optionally, the user device may select from among one or more streams generated by the remote system according to a bandwidth available to the user device. (wherein selecting among one or more streams is the display mode is separate display). Please also read Paragraph [0164].), the displaying the at least one first image element and the at least one second image element according to the display mode of the first image element and the second image element (Figs. 1-6, Paragraph [0101]-BURKE discloses the system obtains information separating user interface elements from gameplay elements. As the video game executes, the system can obtain output rendered by the video game. As described above, the video game may render elements using draw calls and then update (e.g., paint) the elements onto an image frame. The system may obtain an image frame that includes the user interface elements, and a different image frame that includes the gameplay elements. Further in Paragraph [0109]-BURKE discloses since the user interface elements may not update as often as gameplay elements, for example a health bar or a menu may be included in rendered image frames for several seconds without being updated in appearance by the video game. The system may therefore reduce a size of the stream by excluding user interface elements from the gameplay stream until they change (wherein excluding user interface elements from the gameplay stream until they change is where the display mode is a separate display).) comprises:
separately displaying the at least one first image element and the at least one second image element (Figs. 1-6, Paragraph [0116]-BURKE discloses the system receives one or more gameplay streams from the remote system. As described above, the user device may receive a first stream that includes gameplay elements, and a second stream that includes user interface elements. The user device can decode the streams (e.g., using a software or hardware decoder), and can combine the streams. For example, the user interface element stream may include pixel information, such as color information and transparency information. The user device can utilize this information to overlay (e.g., blend) the user interface elements on top of the gameplay elements. Once combined, the user device can present the combination to the user. Optionally, the user device may select from among one or more streams generated by the remote system according to a bandwidth available to the user device.), so as to display the image frame (Figs. 1-6, Paragraph [0164]-BURKE discloses user I/O 34 is used to send and receive commands between processing unit 1102 and user devices, such as game controllers. In some embodiments, the user I/O 34 can include touchscreen inputs. As previously described, the touchscreen can be a capacitive touchscreen, a resistive touchscreen, or other type of touchscreen technology that is configured to receive user input through tactile inputs from the user. Display I/O 36 provides input/output functions that are used to display images from the game being played. Network I/O 38 is used for input/output functions for a network. Network I/O 38 may be used during execution of a game, such as when a game is being played online or being accessed online.).
Regarding claim 17, BURKE teaches the computer device according to claim 10, BURKE further teaches wherein the rendering the at least one first image element (Figs. 1-6, Paragraph [0061]-BURKE discloses the user interface elements 12A-12C may be overlaid on the non-user interface elements 14 and may optionally be rendered more simply.) based on the first rendering instruction (Figs. 1-6, Paragraph [0064]-BURKE discloses for example, user interface element 12A (e.g., a user's place in a car race) may be rendered as one or more colors. That is, this user interface element 12A may be designed to be easily legible by a user of the video game. Similarly, user interface element 12C may be designed to be easily decipherable to indicate a speed at which the user's car is traveling. For example, the user interface element 12C may be designed to be distinguishable from the non-user interface elements 14 that are proximate to the user interface element 12C. Further in Paragraph [0081]-BURKE discloses a first image frame 112 (wherein a first image frame 112 includes first image elements) represents the user-interface elements included in the video game image scene 10. Please also read Paragraph [0069] and [0176].) comprises:
obtaining a rendering function name (Figs. 1-6, Paragraph [0082]-BURKE discloses to generate the rendered image frames 112, 114, the game engine 110 may execute a version of the video game which has been customized. For example, when rendering a particular image frame (e.g., the video game scene 10), the video game may perform multitudes of draw calls (to render portions of the particular image frame. Example draw calls may include rendering characters, environments, a crowd in a sports game, and so on. Please also refer to Paragraph [0167].) comprised in the first rendering instruction (Figs. 1-6, Paragraph [0082]-BURKE discloses the stream encoding engine 120 may generate two or more gameplay streams. In this example, the stream encoding engine 120 may encode image frame 114 and include this encoded image frame in a first gameplay stream 122 directed to gameplay elements. When the video game draws user interface elements (e.g., as described above), the stream encoding engine 120 may switch to a second stream and included image frame 112 in this second stream. The user device 150 may receive these streams, and combine the streams. For example, the user device 150 may overlay decoded user interface image frames on corresponding decoded gameplay image frames. Please also read Paragraph [0176].), and related parameters used during rendering the at least one first image element (Figs. 1-6, Paragraph [0063]-BURKE discloses user interface element 12B may represent a location of the car in a game world of the video game. This user interface element 12B may similarly be distinguishable from underlying non-user interface elements 14. Optionally, the user interface element 12B may, at least in part, be rendered as transparent. For example, portion 18A may represent a track on which the car is driving, and may include a representation of the user's car. As another example, portion 18B may represent game world areas proximate to the track. Portion 18A may optionally be rendered as substantially opaque, while portion 18B may be at least partially transparent. In this way, the non-user interface elements 14 under this portion 18B may be partially visible. Please also read Paragraph [0056].);
and calling, based on the rendering function name (Figs. 1-6, Paragraph [0013]-BURKE discloses the server system described herein may execute a video game, and utilize metadata generated by the video game during gameplay to identify portions of rendered output (e.g., image frames) that are (1) video game gameplay or (2) user interface elements. For example, when rendering an image frame, the video game may utilize specific draw calls to render portions of the image frame.), a function interface corresponding to the rendering function name, so as to render the at least one first image element through the function interface and the related parameters (Figs. 1-6, Paragraph [0101]-BURKE discloses the system obtains information separating user interface elements from gameplay elements. As the video game executes, the system can obtain output rendered by the video game. As described above, the video game may render elements using draw calls and then update (e.g., paint) the elements onto an image frame. The system may obtain an image frame that includes the user interface elements, and a different image frame that includes the gameplay elements.).
Regarding claim 18, BURKE teaches the computer device according to claim 10, BURKE further teaches wherein, the image frame is a virtual scene picture (Fig. 1, #10 called a video game scene, Paragraph [0060]-BURKE discloses FIG. 1 illustrates an example video game scene 10 of a video game. The video game scene 10 may be an example of an image frame generated by a video game for presentation to a user of the video game.), the first image element (Figs. 1-2C, Paragraph [0081]-BURKE discloses a first image frame 112 represents the user-interface elements (wherein user-interface elements is the first image element) included in the video game image scene 10.) comprises at least one of an icon (Figs. 1-2C, illustrates one of an icon, Paragraph [0041]-BURKE discloses for example, the system can ensure that user interface elements (e.g., text, menus, and so on) are presented in high resolution (e.g., a resolution natively rendered by the video game). Further in Paragraph [0061]-BURKE discloses the example gameplay elements may be separated according to user interface elements 12A-12C (wherein 12C is an icon), and non-user interface elements 14. The non-user interface elements 14 may include gameplay elements 16A-16B.), a graphic button of a virtual control (Figs. 1-2C, Paragraph [0074]-BURKE discloses for example, the mobile device may present a representation of user input commands (e.g., an ‘A’ button, a triangle command, and so on). The user may interact with this representation, and the game application 160 may provide information to the stream optimization system 100 identifying the type of command selected based on the interaction. Please also read Paragraphs [0041] and [0061].), and a graphic comprising text content (Figs. 1-2C, illustrates a graphic comprising text content, Paragraph [0061]-BURKE discloses the example gameplay elements may be separated according to user interface elements 12A-12C (wherein 12C is a graphic comprising text content), and non-user interface elements 14. The non-user interface elements 14 may include gameplay elements 16A-16B. Please also read Paragraph [0096].), superimposed on the virtual scene picture (Figs. 1-2C, illustrates the virtual scene picture, Paragraph [0062]-BURKE discloses the user interface elements 12A-12C may be overlaid on the non-user interface elements 14 and may optionally be rendered more simply.);
and the second image element (Figs. 1-6, Paragraph [0061]-BURKE discloses as illustrated in FIG. 1, the video game scene 10 includes gameplay elements of the video game, which in the example is a car racing video game. The example gameplay elements may be separated according to user interface elements 12A-12C, and non-user interface elements 14. The non-user interface elements 14 may include gameplay elements 16A-16B (wherein non-user interface elements are the second image element). Further in Paragraph [0081]-BURKE discloses a second image frame 114 represents the gameplay elements included the video game image scene 10.) comprises an image used for displaying the virtual scene in the virtual scene picture (Figs. 1-6, illustrates a virtual scene picture, Paragraph [0061]-BURKE discloses the non-user interface elements 14 may include gameplay elements 16A-16B. As an example, gameplay element 16A may represent a car being controlled by a user of the video game. As another example, gameplay element 16B may represent a game world or environment in which the car is being driven (e.g., a race track). (wherein 16B is an image used for displaying the virtual scene).
Regarding claim 19, BURKE teaches a non-transitory computer-readable storage medium, storing at least one computer program, the computer program, when executed by a processor of a computer device (Figs. 1 and 11, #1100 called a user computing system, Paragraph [0171]-BURKE discloses all of the processes described herein may be embodied in, and fully automated via, software code modules executed by a computing system that includes one or more computers or processors. The code modules may be stored in any type of non-transitory computer-readable medium or other computer storage device. Some or all the methods may be embodied in specialized computer hardware. Please also read Paragraph [0161].), causing the computer device to implement an image frame display method (Figs. 2A-2B, 4-6, 11, #1100 called a user computing system, Paragraph [0054]-BURKE discloses as used herein, a video game gameplay stream (herein also referred to as an encoded gameplay stream, gameplay stream, and so on) refers to a stream comprising encoded image frames generated by a video game. The video game gameplay stream may further include encoded audio. For example, a video gameplay stream may be an MPEG-4 video which, when decoded, represents display data and audio data generated by a video game. Optionally, a video game gameplay stream may comprise encoded image frames of either user interface elements or gameplay elements. That is, a multitude of video game gameplay streams may be generated, and a user device may decode and combine each stream for presentation to a user of the user device. Please also read Paragraph [0051].) including:
receiving a first rendering instruction (Figs. 2A-6, Paragraph [0081]-BURKE discloses a first image frame 112 represents the user-interface elements included in the video game image scene 10. Further in Paragraph [0176]-BURKE discloses any process descriptions, elements or blocks in the flow diagrams described herein and/or depicted in the attached figures should be understood as potentially representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or elements in the process.) transmitted by a server (Figs. 1 and 4-8B, illustrates transmitted by a server, #100 called a Stream Optimization System, Paragraph [0064]-BURKE discloses the stream optimization system 100 (wherein the stream optimization system is a server) includes a game engine 110 that can execute a video game, respond to user input 152 received from a the user device 150, and present rendered output from the video game to a stream encoding engine 120. As described above, the stream optimization system 100 may receive a request from the user device 150 to execute a particular video game. The stream optimization system 100 may then run the particular video game, for example emulate a software environment in which the particular video game would normally run. Further in Paragraph [0071]-BURKE discloses as will be described, the stream encoding engine 120 can provide an encoded gameplay stream 122 to the user device 150 for presentation on the user device 150. Please also read Paragraphs [0045] and [0069].);
rendering at least one first image element (Figs. 1-6, Paragraph [0061]-BURKE discloses the user interface elements 12A-12C may be overlaid on the non-user interface elements 14 and may optionally be rendered more simply.) based on the first rendering instruction (Figs. 1-6, Paragraph [0064]-BURKE discloses for example, user interface element 12A (e.g., a user's place in a car race) may be rendered as one or more colors. That is, this user interface element 12A may be designed to be easily legible by a user of the video game. Similarly, user interface element 12C may be designed to be easily decipherable to indicate a speed at which the user's car is traveling. For example, the user interface element 12C may be designed to be distinguishable from the non-user interface elements 14 that are proximate to the user interface element 12C. Further in Paragraph [0081]-BURKE discloses a first image frame 112 (wherein a first image frame 112 includes first image elements) represents the user-interface elements included in the video game image scene 10. Please also read Paragraphs [0069] and [0176].);
receiving at least one second image element transmitted by the server (Figs. 1-6, illustrates transmitted and rendered by the server, Paragraph [0045]-BURKE discloses an example video game stream may include encoded video generated from image frames rendered by a video game. For example, a video game may generate display data, such as successive image frames (e.g., 30, 45, 60, times per second), and this display data may be encoded. Example encoders can include H.264, MPEG-4, High Efficiency Video Coding (HEVC), VP8, VP9, and so on. Thus, the display data generated by the video game may be encoded to package the display data into information suitable for transmission over a network. Additionally, the display data may be encoded to reduce an amount of information which is being provided over the network (e.g., 3, 5, 15, 20, megabytes per second and so on). Thus, a burden on a bandwidth available to a user device may be reduced via the encoding. Please also read Paragraph [0061].), the at least one second image element being rendered by the server (Figs. 1-6, illustrates transmitted and rendered by the server, Paragraph [0061]-BURKE discloses as illustrated in FIG. 1, the video game scene 10 includes gameplay elements of the video (Figs. 1-6, illustrates transmitted and rendered by the server, Paragraph [0064]-BURKE discloses as will be described in more detail below, a system described herein (e.g., the goal optimization system 100) (wherein goal optimization system is the server) may generate a video game gameplay stream to be provided to a user device, and enable remote play of the video game by a user of the user device. To ensure that the user device has sufficient available bandwidth to timely download the video gameplay stream, and provide user input to the system, the system can compress the video game gameplay stream. As an example, the video game scene 10 illustrated in FIG. 1 may be included in a video game gameplay stream. The system may compress the video game scene 10 during encoding of the video game scene 10 into the video game gameplay stream. As described above, the non-user interface elements 14 may include realistic detail, and may be rendered at a high resolution. Further in Paragraph [0081]-BURKE discloses a second image frame 114 (wherein a second image frame 114 includes second image elements) represents the gameplay elements included the video game image scene 10. Please also read Paragraphs [0045] and [0069].);
receiving an interactive instruction transmitted by the server (Figs. 1-2A, illustrates transmitted by the server, Paragraph [0057]-BURKE discloses as used herein in reference to user interactions with data displayed by a computing system, “user input” is a broad term that refers to any type of input provided by a user that is intended to be received and/or stored by a system, to cause an update to data that is displayed by the system, and/or to cause an update to the way that data is displayed by the system. Further in Paragraph [0176]-BURKE discloses any process descriptions, elements or blocks in the flow diagrams described herein and/or depicted in the attached figures should be understood as potentially representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or elements in the process. Please also read Paragraphs [0016]-[0017] and [0045].);
and displaying an image frame (Figs. 1-6, Paragraph [0164]-BURKE discloses user I/O 34 is used to send and receive commands between processing unit 1102 and user devices, such as game controllers. In some embodiments, the user I/O 34 can include touchscreen inputs. As previously described, the touchscreen can be a capacitive touchscreen, a resistive touchscreen, or other type of touchscreen technology that is configured to receive user input through tactile inputs from the user. Display I/O 36 provides input/output functions that are used to display images from the game being played. Network I/O 38 is used for input/output functions for a network. Network I/O 38 may be used during execution of a game, such as when a game is being played online or being accessed online.) based on the at least one first image element, the at least one second image element (Figs. 2A-6, illustrates transmitted by a server, Paragraph [0081]-BURKE discloses as illustrated in FIG. 1, the user interface elements 12A-12C are thus included as an overlay on the gameplay elements. Thus, a combination of the first image frame 112 overlaid on the second image frame 114 can result in the video game image scene 10. These image frames 112, 114, may be combined (e.g., packaged) by the stream encoding engine 120 and utilized to generate the encoded gameplay stream 122. Optionally, the image frames 112, 114, may be provided as separate encoded gameplay streams to the user device 150. In this example, the user device 150 may combine the received streams. Please also read Paragraph [0180].), and the interactive instruction (Figs. 1-2A, illustrates transmitted by the server, Paragraph [0057]-BURKE discloses as used herein in reference to user interactions with data displayed by a computing system, “user input” is a broad term that refers to any type of input provided by a user that is intended to be received and/or stored by a system, to cause an update to data that is displayed by the system, and/or to cause an update to the way that data is displayed by the system. Further in Paragraph [0180]-BURKE discloses providing, to the user device over the network, the encoded gameplay stream for presentation, wherein the encoded gameplay stream is configured to be output as a series of rendered image frames on the user device. Please also read Paragraphs [0016] and [0017].).
Regarding claim 20, BURKE teaches the non-transitory computer-readable storage medium according to claim 19, BURKE further teaches wherein the method further comprises:
obtaining an interactive instruction indicating a display mode (Figs. 1-2C, Paragraph [0041]-BURKE discloses the users may receive streaming video of real-time video game gameplay generated by the system, and may provide user input to control features of the streamed video. As an example, a user may utilize a user device to stream a role-playing video game. The user may provide user input to the user device, which can forward the user input to the system to update the role-playing video game. Further in Paragraph [0176]-BURKE discloses any process descriptions, elements or blocks in the flow diagrams described herein and/or depicted in the attached figures should be understood as potentially representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or elements in the process.) of the at least one first image element and the at least one second image element (Fig. 1-2C, Paragraph [0041]-BURKE discloses for example, the system can ensure that user interface elements (e.g., text, menus, and so on) (wherein user interface elements are at least one first image element) are presented in high resolution (e.g., a resolution natively rendered by the video game). In this example, the system may adjust quality of non-user interface elements, such as the video game gameplay itself (e.g., characters, environments, and so on). That is, the system may adaptively adjust a bitrate of the portions of the streamed video related to these non-user interface elements (wherein non-user interface elements are at least one second image element) while preserving the bitrate of the user interface elements. In this way, the user's perception of the reduction in quality may be reduced as any text, menus, and so on, may be appear in high quality. Please also read Paragraphs [0061] and [0081].);
and combining the at least one first image element and the at least one second image element in accordance with the interactive instruction to form the image frame (Figs. 1-2C, Paragraph [0041]-BURKE discloses while the user interface elements (wherein user interface elements are at least one first image element) may be included in the streamed video at high quality, due to the techniques described herein a size associated with the streamed video (e.g., measured in megabytes per second) may be conserved. As described herein, the system can cause non-user interface elements occluded by, or behind, user interface elements (wherein occluded by, or behind is combining) to be reduced in quality in the streamed video. Since these non-user interface elements (wherein non-user interface elements are at least one second image element) may be partially hidden, or otherwise adjusted in appearance by, the user interface elements, the system can reduce their quality as compared to other non-user interface elements. For example, a user viewing the streamed video may have a greater difficulty identifying specific details of a video game character partially covered by a user interface element. Thus, the system can encode these non-user interface elements to be more aggressively compressed or otherwise reduced in size. Please also read Paragraphs [0061] and [0081].).
Regarding claim 21, BURKE teaches the non-transitory computer-readable storage medium according to claim 19, BURKE further teaches wherein, the image frame is a virtual scene picture (Fig. 1, #10 called a video game scene, Paragraph [0060]-BURKE discloses FIG. 1 illustrates an example video game scene 10 of a video game. The video game scene 10 may be an example of an image frame generated by a video game for presentation to a user of the video game.), the first image element (Figs. 1-2C, Paragraph [0081]-BURKE discloses a first image frame 112 represents the user-interface elements (wherein user-interface elements is the first image element) included in the video game image scene 10.) comprises at least one of an icon (Figs. 1-2C, illustrates one of an icon, Paragraph [0041]-BURKE discloses for example, the system can ensure that user interface elements (e.g., text, menus, and so on) are presented in high resolution (e.g., a resolution natively rendered by the video game). Further in Paragraph [0061]-BURKE discloses the example gameplay elements may be separated according to user interface elements 12A-12C (wherein 12C is an icon), and non-user interface elements 14. The non-user interface elements 14 may include gameplay elements 16A-16B.), a graphic button of a virtual control (Figs. 1-2C, Paragraph [0074]-BURKE discloses for example, the mobile device may present a representation of user input commands (e.g., an ‘A’ button, a triangle command, and so on). The user may interact with this representation, and the game application 160 may provide information to the stream optimization system 100 identifying the type of command selected based on the interaction. Please also read Paragraphs [0041] and [0061].), and a graphic comprising text content (Figs. 1-2C, illustrates a graphic comprising text content, Paragraph [0061]-BURKE discloses the example gameplay elements may be separated according to user interface elements 12A-12C (wherein 12C is a graphic comprising text content), and non-user interface elements 14. The non-user interface elements 14 may include gameplay elements 16A-16B. Please also read Paragraph [0096].), superimposed on the virtual scene picture (Figs. 1-2C, illustrates the virtual scene picture, Paragraph [0062]-BURKE discloses the user interface elements 12A-12C may be overlaid on the non-user interface elements 14 and may optionally be rendered more simply.);
and the second image element (Figs. 1-6, Paragraph [0061]-BURKE discloses as illustrated in FIG. 1, the video game scene 10 includes gameplay elements of the video game, which in the example is a car racing video game. The example gameplay elements may be separated according to user interface elements 12A-12C, and non-user interface elements 14. The non-user interface elements 14 may include gameplay elements 16A-16B (wherein non-user interface elements are the second image element). Further in Paragraph [0081]-BURKE discloses a second image frame 114 represents the gameplay elements included the video game image scene 10.) comprises an image used for displaying the virtual scene in the virtual scene picture (Figs. 1-6, illustrates a virtual scene picture, Paragraph [0061]-BURKE discloses the non-user interface elements 14 may include gameplay elements 16A-16B. As an example, gameplay element 16A may represent a car being controlled by a user of the video game. As another example, gameplay element 16B may represent a game world or environment in which the car is being driven (e.g., a race track). (wherein 16B is an image used for displaying the virtual scene).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 5 and 14, are rejected under 35 U.S.C. 103 as being unpatentable over BURKE (US 20220193541 A1), hereinafter referenced as BURKE in view of KARLSSON (US 20200306631 A1).
Regarding claim 5, BURKE teaches the method according to claim 4, BURKE is silent on the term synchronization time indication. Although BURKE further teaches wherein, the display mode is synchronous display (Figs. 1-6, Paragraph [0164]-BURKE discloses user I/O 34 is used to send and receive commands between processing unit 1102 and user devices, such as game controllers. In some embodiments, the user I/O 34 can include touchscreen inputs. As previously described, the touchscreen can be a capacitive touchscreen, a resistive touchscreen, or other type of touchscreen technology that is configured to receive user input through tactile inputs from the user. Display I/O 36 provides input/output functions that are used to display images from the game being played. Network I/O 38 is used for input/output functions for a network. Network I/O 38 may be used during execution of a game, such as when a game is being played online or being accessed online. Further in Paragraph [0113]-BURKE discloses these streams may be packaged together (e.g., along with sync or timing information), and the user device may receive the packaged stream. Upon receipt, the user device can decode the streams and present the gameplay to a user. Please also read Paragraph [0215].), the first interactive instruction comprises a first interaction parameter (Figs. 1-6, Paragraph [0045]-BURKE discloses an example video game stream may include encoded video generated from image frames rendered by a video game. For example, a video game may generate display data, such as successive image frames (e.g., 30, 45, 60, times per second), and this display data may be encoded. Example encoders can include H.264, MPEG-4, High Efficiency Video Coding (HEVC), VP8, VP9, and so on. Please also read Paragraphs [0176] and [0215].), the second interactive instruction comprises a second interaction parameter (Figs. 1-6, Paragraph [0045]-BURKE discloses additionally, the display data may be encoded to reduce an amount of information which is being provided over the network (e.g., 3, 5, 15, 20, megabytes per second and so on). Thus, a burden on a bandwidth available to a user device may be reduced via the encoding. Further in Paragraph [0207]-BURKE discloses wherein user interface elements are included in the gameplay stream at a first frequency, and wherein gameplay elements are included in the stream at a second frequency, the second frequency corresponding to a refresh rate of the video game, and the first frequency being less than the second frequency. Please also read Paragraphs [0176] and [0215].),
and the displaying (Figs. 1-6, Paragraph [0164]-BURKE discloses user I/O 34 is used to send and receive commands between processing unit 1102 and user devices, such as game controllers. In some embodiments, the user I/O 34 can include touchscreen inputs. As previously described, the touchscreen can be a capacitive touchscreen, a resistive touchscreen, or other type of touchscreen technology that is configured to receive user input through tactile inputs from the user. Display I/O 36 provides input/output functions that are used to display images from the game being played. Network I/O 38 is used for input/output functions for a network. Network I/O 38 may be used during execution of a game, such as when a game is being played online or being accessed online.) the at least one first image element and the at least one second image element according to the display mode of the first image element and the second image element (Figs. 1-6, Paragraph [0113]-BURKE discloses at block 510 the system provides a stream to the user device. As described above, the system may provide a first stream that includes user interface elements and a second stream that includes gameplay elements to the user device. Optionally, these streams may be packaged together (e.g., along with sync or timing information), and the user device may receive the packaged stream. Upon receipt, the user device can decode the streams and present the gameplay to a user. Please also read Paragraphs [0164] and [0180].).
BURKE fails to explicitly teach and the first interaction parameter and the second interaction parameter comprise synchronization time indication information of the first image element and the second image element, respectively; and the displaying the at least one first image element and the at least one second image element according to the display mode of the first image element and the second image element comprises: synchronously displaying image elements among the at least one first image element and the at least one second image element that match the synchronization time indication information.
However, KARLSSON explicitly teaches and the first interaction parameter and the second interaction parameter (Figs. 1-2F, Paragraph [0103]-KARLSSON discloses the user requests to play a game application 110 through the dynamic client 112. The dynamic client 112 transmits the request to the game application streaming system 136. Based on the request the game application streaming system 136 can instantiate a client instance 140 for the user. In some embodiments, the dynamic client 112 may communicate with the interactive computing system 130 or another management plane layer, which can be configured to provide instructions to the game application streaming system 136 to instantiate a client instance 140 on behalf of the user computing system 102. The client instance 140 can execute the game application 110, in the same manner that the game application 110 is executed on a user computing system 102. Please also read Paragraph [0104] and [0033].) comprise synchronization time indication information of the first image element and the second image element, respectively (Figs. 1-2F, Paragraph [0033]-KARLSSON discloses the simulation engine 122 can read in game rules and generates game state based on input received from one or more users. The simulation engine 122 can control execution of individual objects, such as virtual components, virtual effects and/or virtual characters, within the game application. Further in Paragraph [0035]-KARLSSON discloses the presentation engine 124 can use the graphical state data to generate and render frames for output to a display within the game application. The presentation engine 124 can combine the virtual objects, such as virtual characters, animate objects, inanimate objects, background objects, lighting, reflection, and the like, in order to generate a full scene and a new frame for display. The presentation engine 124 takes into account the surfaces, colors textures, and other parameters during the rendering process. The presentation engine 124 can combine the virtual objects (e.g., lighting within the virtual environment and virtual character images with inanimate and background objects) to generate and render a frame. The execution of the presentation engine 124 is described in further detail herein.); and the displaying (Figs. 1-2F, Paragraph [0134]-KARLSSON discloses user I/O 34 is used to send and receive commands between processing unit 20 and user devices, such as game controllers. In some embodiments, the user I/O can include a touchscreen inputs. The touchscreen can be capacitive touchscreen, a resistive touchscreen, or other type of touchscreen technology that is configured to receive user input through tactile inputs from the user. Display I/O 36 provides input/output functions that are used to display images from the game being played. Network I/O 38 is used for input/output functions for a network. Network I/O 38 may be used during execution of a game.) the at least one first image element and the at least one second image element according to the display mode of the first image element and the second image element (Figs. 1-2F, illustrates synchronous display, Paragraph [0035]-KARLSSON discloses the presentation engine 124 can combine the virtual objects (e.g., lighting within the virtual environment and virtual character images with inanimate and background objects) to generate and render a frame. (wherein lighting within the virtual environment is at least one first image element and at least one second image element).) comprises: synchronously displaying image elements among the at least one first image element and the at least one second image element (Figs. 1-2F, illustrates synchronous display, Paragraph [0035]-KARLSSON discloses the presentation engine 124 can combine the virtual objects (e.g., lighting within the virtual environment and virtual character images with inanimate and background objects) to generate and render a frame. (wherein lighting within the virtual environment is at least one first image element and at least one second image element). Further in Paragraph [0091]-KARLSSON discloses at block 306, the simulation engine 122 generates graphical state data for state stream processes that are necessary for presentation engine 124 to render virtual objects within the virtual environment. The graphical state data can be a subset of the simulation state data. The simulation engine may not generate any data for state stream process where the state did not change when compared to the previous simulation state. Further in Paragraph [0092]-KARLSSON discloses at block 308, the simulation engine writes the graphical state data to the SSDP. The graphical state data for state stream process can be written to the SSDP as soon as the graphical state data is generated.) that match the synchronization time indication information (Figs. 1-2F, illustrates synchronous display, Paragraph [0072]-KARLSSON discloses as illustrated in FIG. 2C, each SSDP is composed of a plurality of individual state stream processes that are generated during each simulation cycle. The generation of the state stream processes are added to the SSDP, and after all the state stream processes have been completed, the SSDP will be finalized and available on the state stream. Further in Paragraph [0073]-KARLSSON discloses for each state stream process of a virtual object, the graphical state data can include static state information and dynamic state information. Static state information refers to state information that is the same for the entire life of an object. Dynamic state information refers to state information that changes over the life of an object. The full graphical state of an object at time X (that is, the time at which the SSDP is generated) is the static state information in conjunction with the interpolated dynamic state information. Please also read Paragraph [0035].).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of BURKE of having an image frame display method performed by a computer device and the method comprising: receiving a first rendering instruction transmitted by a server; rendering at least one first image element based on the first rendering instruction, with the teachings of KARLSSON of having and the first interaction parameter and the second interaction parameter comprise synchronization time indication information of the first image element and the second image element, respectively; and the displaying the at least one first image element and the at least one second image element according to the display mode of the first image element and the second image element comprises: synchronously displaying image elements among the at least one first image element and the at least one second image element that match the synchronization time indication information.
Wherein having BURKE’s image display system of and the first interaction parameter and the second interaction parameter comprise synchronization time indication information of the first image element and the second image element, respectively; and the displaying the at least one first image element and the at least one second image element according to the display mode of the first image element and the second image element comprises: synchronously displaying image elements among the at least one first image element and the at least one second image element that match the synchronization time indication information.
The motivation behind the modification would have been to obtain an image display system that improves gameplay experience and presentation of video game streams provided over the internet, since both BURKE and KARLSSON are systems that relate to video games. Wherein BURKE’s server system proactively adjust a quality of a video game stream (e.g., due to fluctuations in bandwidth, packet loss, and so on) while preserving the quality of user interface elements, while KARLSSON’s system increases gameplay quality, frame quality and/or resolution of the presentation of the game application and reduces the latency that the user experiences when playing the game application. Please see BURKE (US 20220193541 A1), Paragraphs [0006 and 0015] and KARLSSON (US 20200306631 A1), Paragraphs [0110 and 0112].
Regarding claim 14, BURKE teaches the computer device according to claim 12, BURKE is silent on the term synchronization time indication. Although BURKE further teaches wherein, the display mode is synchronous display (Figs. 1-6, Paragraph [0164]-BURKE discloses user I/O 34 is used to send and receive commands between processing unit 1102 and user devices, such as game controllers. In some embodiments, the user I/O 34 can include touchscreen inputs. As previously described, the touchscreen can be a capacitive touchscreen, a resistive touchscreen, or other type of touchscreen technology that is configured to receive user input through tactile inputs from the user. Display I/O 36 provides input/output functions that are used to display images from the game being played. Network I/O 38 is used for input/output functions for a network. Network I/O 38 may be used during execution of a game, such as when a game is being played online or being accessed online. Further in Paragraph [0113]-BURKE discloses these streams may be packaged together (e.g., along with sync or timing information), and the user device may receive the packaged stream. Upon receipt, the user device can decode the streams and present the gameplay to a user. Please also read Paragraph [0215].), the first interactive instruction comprises a first interaction parameter (Figs. 1-6, Paragraph [0045]-BURKE discloses an example video game stream may include encoded video generated from image frames rendered by a video game. For example, a video game may generate display data, such as successive image frames (e.g., 30, 45, 60, times per second), and this display data may be encoded. Example encoders can include H.264, MPEG-4, High Efficiency Video Coding (HEVC), VP8, VP9, and so on. Please also read Paragraphs [0176] and [0215].), the second interactive instruction comprises a second interaction parameter (Figs. 1-6, Paragraph [0045]-BURKE discloses additionally, the display data may be encoded to reduce an amount of information which is being provided over the network (e.g., 3, 5, 15, 20, megabytes per second and so on). Thus, a burden on a bandwidth available to a user device may be reduced via the encoding. Further in Paragraph [0207]-BURKE discloses wherein user interface elements are included in the gameplay stream at a first frequency, and wherein gameplay elements are included in the stream at a second frequency, the second frequency corresponding to a refresh rate of the video game, and the first frequency being less than the second frequency. Please also read Paragraphs [0176] and [0215].),
and the displaying (Figs. 1-6, Paragraph [0164]-BURKE discloses user I/O 34 is used to send and receive commands between processing unit 1102 and user devices, such as game controllers. In some embodiments, the user I/O 34 can include touchscreen inputs. As previously described, the touchscreen can be a capacitive touchscreen, a resistive touchscreen, or other type of touchscreen technology that is configured to receive user input through tactile inputs from the user. Display I/O 36 provides input/output functions that are used to display images from the game being played. Network I/O 38 is used for input/output functions for a network. Network I/O 38 may be used during execution of a game, such as when a game is being played online or being accessed online.) the at least one first image element and the at least one second image element according to the display mode of the first image element and the second image element (Figs. 1-6, Paragraph [0113]-BURKE discloses at block 510 the system provides a stream to the user device. As described above, the system may provide a first stream that includes user interface elements and a second stream that includes gameplay elements to the user device. Optionally, these streams may be packaged together (e.g., along with sync or timing information), and the user device may receive the packaged stream. Upon receipt, the user device can decode the streams and present the gameplay to a user. Please also read Paragraphs [0164] and [0180].).
BURKE fails to explicitly teach and the first interaction parameter and the second interaction parameter comprise synchronization time indication information of the first image element and the second image element, respectively; and the displaying the at least one first image element and the at least one second image element according to the display mode of the first image element and the second image element comprises: synchronously displaying image elements among the at least one first image element and the at least one second image element that match the synchronization time indication information.
However, KARLSSON explicitly teaches and the first interaction parameter and the second interaction parameter (Figs. 1-2F, Paragraph [0103]-KARLSSON discloses the user requests to play a game application 110 through the dynamic client 112. The dynamic client 112 transmits the request to the game application streaming system 136. Based on the request the game application streaming system 136 can instantiate a client instance 140 for the user. In some embodiments, the dynamic client 112 may communicate with the interactive computing system 130 or another management plane layer, which can be configured to provide instructions to the game application streaming system 136 to instantiate a client instance 140 on behalf of the user computing system 102. The client instance 140 can execute the game application 110, in the same manner that the game application 110 is executed on a user computing system 102. Please also read Paragraph [0104] and [0033].) comprise synchronization time indication information of the first image element and the second image element, respectively (Figs. 1-2F, Paragraph [0033]-KARLSSON discloses the simulation engine 122 can read in game rules and generates game state based on input received from one or more users. The simulation engine 122 can control execution of individual objects, such as virtual components, virtual effects and/or virtual characters, within the game application. Further in Paragraph [0035]-KARLSSON discloses the presentation engine 124 can use the graphical state data to generate and render frames for output to a display within the game application. The presentation engine 124 can combine the virtual objects, such as virtual characters, animate objects, inanimate objects, background objects, lighting, reflection, and the like, in order to generate a full scene and a new frame for display. The presentation engine 124 takes into account the surfaces, colors textures, and other parameters during the rendering process. The presentation engine 124 can combine the virtual objects (e.g., lighting within the virtual environment and virtual character images with inanimate and background objects) to generate and render a frame. The execution of the presentation engine 124 is described in further detail herein.); and the displaying (Figs. 1-2F, Paragraph [0134]-KARLSSON discloses user I/O 34 is used to send and receive commands between processing unit 20 and user devices, such as game controllers. In some embodiments, the user I/O can include a touchscreen inputs. The touchscreen can be capacitive touchscreen, a resistive touchscreen, or other type of touchscreen technology that is configured to receive user input through tactile inputs from the user. Display I/O 36 provides input/output functions that are used to display images from the game being played. Network I/O 38 is used for input/output functions for a network. Network I/O 38 may be used during execution of a game.) the at least one first image element and the at least one second image element according to the display mode of the first image element and the second image element (Figs. 1-2F, illustrates synchronous display, Paragraph [0035]-KARLSSON discloses the presentation engine 124 can combine the virtual objects (e.g., lighting within the virtual environment and virtual character images with inanimate and background objects) to generate and render a frame. (wherein lighting within the virtual environment is at least one first image element and at least one second image element).) comprises: synchronously displaying image elements among the at least one first image element and the at least one second image element (Figs. 1-2F, illustrates synchronous display, Paragraph [0035]-KARLSSON discloses the presentation engine 124 can combine the virtual objects (e.g., lighting within the virtual environment and virtual character images with inanimate and background objects) to generate and render a frame. (wherein lighting within the virtual environment is at least one first image element and at least one second image element). Further in Paragraph [0091]-KARLSSON discloses at block 306, the simulation engine 122 generates graphical state data for state stream processes that are necessary for presentation engine 124 to render virtual objects within the virtual environment. The graphical state data can be a subset of the simulation state data. The simulation engine may not generate any data for state stream process where the state did not change when compared to the previous simulation state. Further in Paragraph [0092]-KARLSSON discloses at block 308, the simulation engine writes the graphical state data to the SSDP. The graphical state data for state stream process can be written to the SSDP as soon as the graphical state data is generated.) that match the synchronization time indication information (Figs. 1-2F, illustrates synchronous display, Paragraph [0072]-KARLSSON discloses as illustrated in FIG. 2C, each SSDP is composed of a plurality of individual state stream processes that are generated during each simulation cycle. The generation of the state stream processes are added to the SSDP, and after all the state stream processes have been completed, the SSDP will be finalized and available on the state stream. Further in Paragraph [0073]-KARLSSON discloses for each state stream process of a virtual object, the graphical state data can include static state information and dynamic state information. Static state information refers to state information that is the same for the entire life of an object. Dynamic state information refers to state information that changes over the life of an object. The full graphical state of an object at time X (that is, the time at which the SSDP is generated) is the static state information in conjunction with the interpolated dynamic state information. Please also read Paragraph [0035].).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of BURKE of having a computer device, comprising a processor and a memory, the memory storing at least one computer instruction, and the at least one computer instruction, when executed by the processor, causing the computer device to implement an image frame display method including: receiving a first rendering instruction transmitted by a server; rendering at least one first image element based on the first rendering instruction, with the teachings of KARLSSON of having and the first interaction parameter and the second interaction parameter comprise synchronization time indication information of the first image element and the second image element, respectively; and the displaying the at least one first image element and the at least one second image element according to the display mode of the first image element and the second image element comprises: synchronously displaying image elements among the at least one first image element and the at least one second image element that match the synchronization time indication information.
Wherein having BURKE’s image display system of and the first interaction parameter and the second interaction parameter comprise synchronization time indication information of the first image element and the second image element, respectively; and the displaying the at least one first image element and the at least one second image element according to the display mode of the first image element and the second image element comprises: synchronously displaying image elements among the at least one first image element and the at least one second image element that match the synchronization time indication information.
The motivation behind the modification would have been to obtain an image display system that improves gameplay experience and presentation of video game streams provided over the internet, since both BURKE and KARLSSON are systems that relate to video games. Wherein BURKE’s server system proactively adjust a quality of a video game stream (e.g., due to fluctuations in bandwidth, packet loss, and so on) while preserving the quality of user interface elements, while KARLSSON’s system increases gameplay quality, frame quality and/or resolution of the presentation of the game application and reduces the latency that the user experiences when playing the game application. Please see BURKE (US 20220193541 A1), Paragraphs [0006 and 0015] and KARLSSON (US 20200306631 A1), Paragraphs [0110 and 0112].
Conclusion
Listed below are the prior arts made of record and not relied upon but are considered pertinent to applicant’s disclosure.
(a) KESLIN (US 20170236245 A1)- A system, method, and computer program product are provided for remote rendering of computer graphics. The system includes a graphics application program resident at a remote server. The graphics application is invoked by a user or process located at a client. The invoked graphics application proceeds to issue graphics instructions. The graphics instructions are received by a remote rendering control system. Given that the client and server differ with respect to graphics context and image processing capability, the remote rendering control system modifies the graphics instructions in order to accommodate these differences. The modified graphics instructions are sent to graphics rendering resources, which produce one or more rendered images. Data representing the rendered images is written to one or more frame buffers. The remote rendering control system then reads this image data from the frame buffers. The image data is transmitted to the client for display or processing. In an embodiment of the system, the image data is compressed before being transmitted to the client. In such an embodiment, the steps of rendering, compression, and transmission can be performed asynchronously in a pipelined manner...... ...... (Fig. 1, Abstract).
(b) SMULLEN et al. (US 20200238175 A1)- This application is directed to a method of managing processing capability of a server system having one or more processing cores that further include multiple processing slices. Upon receiving requests to initiate online gaming sessions, the server system allocates each processing slice of the processing cores to a subset of the online gaming sessions to be executed thereon. A first processing slice is allocated to a first subset of the online gaming sessions including a first gaming session and a second gaming session. At the first processing slice, a time-sharing processing schedule is determined for the first subset of the online gaming sessions. In accordance with the time-sharing processing schedule, the first and second gaming sessions share a duty cycle of the first processing slice, and are executed dynamically and in parallel according to real-time data processing need of the first and second gaming sessions...... ...... (Figs. 1A-2, Abstract).
(c) PERRY et al. (US 20120004042 A1)- A computing system is configured to execute a computer program on a server and to provide a video stream of the program output to a geographically remote client over a communication network. The computing system is further configured to provide executable content of the computer program to the client over the communication network in parallel with the video stream. When a sufficient amount of the executable content has been provided to the client execution of the computer program is transitioned from the server to the client. The transition optionally includes communicating a state of the computer program from the server to the client. The executable content can be provided to the client in an order that is determined based on the state of the computer program. Those parts of the executable content deemed most likely to be necessary to support game play on the client are given priority...... ...... (Figs. 1-4, Abstract).
(d) KRUGLICK (US 20150273328 A1)- Technologies and implementations for providing and online gaming experience are generally disclosed. In one example, a computer-implemented method to allocate rendering tasks to resources of a server system comprise determining at least a portion of a first game world view viewable at a first game client of a plurality of game clients, receiving at least a portion of a game input from the first game client, determining a change from the first game world state to a second game world state, determining based on the second game world state, by the server system, a second game world view, determining a first render task to update the first game element and a second render task to update the second game element and allocating, the first render task to a first game element resource and the second render task to a second game element resource...... ...... (Figs. 1-3, Abstract).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LESLEY L. MIRANDA whose telephone number is (571) 272-6052. The examiner can normally be reached Monday - Friday, 9:30am to 6:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, CHINEYERE D WILLS-BURNS can be reached on (571) 272-9752. The fax phone number for the organization where this application or proceeding is assigned is (571) 273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or (571) 272-1000.
/LESLEY L MIRANDA/Examiner, Art Unit 2673
/CHINEYERE WILLS-BURNS/Supervisory Patent Examiner, Art Unit 2673
Prosecution Insights