DETAILED ACTION
A. This action is in response to the following communications: Transmittal of Request for Continued Examination filed 06/30/2026.
B. Claims 1-20 remains pending.
Continued Examination Under 37 CFR 1.114
C. A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 06/30/2026 has been entered.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Spiegel, Evan et al. (US Pub. 2024/0249318 A1), herein referred to as “Spiegel” in view of Pardeshi, Siddhant et al. (US Pub. 2021/0397971 A1), herein referred to as “Pardeshi”.
As for claims 1, 10 and 19, Spiegel teaches. A method and corresponding system of claim 10 and one or more processors of claim 19 processing circuity to cause a client device comprising:
storing, by a remote system, first information representative of a previous state of a gaming application;
(par. 67 game system 226 provides various gaming functions that allows for a user to invoke playing specific games wherein the game system enables audio, video and text messaging within the context of gameplay, leaderboards and in-game rewards
Par.68 generating, using a remote system as an external resource system 228 provides an interface for the interaction client 104 to communicate with remote servers (e.g., third-party servers 112) to launch or access external resources, i.e., applications or applets. Spiegel further teaches in figure 3D, item 314 based at least on first input data received from a client device, image data representative of at least one or more frames associated with a state of a gaming application; as receive additional media assets such as images and video. Then in paragraph 40 mentioning of a gaming system to be used as the system with intended purpose of playing games in paragraph 41 and in paragraph 67 discusses that the game system 226 provides various gaming functions within the context of the interaction client 104. The interaction client 104 provides a game interface providing a list of available games that can be launched by a user within the context of the interaction client 104 and played with other users of the interaction system 100.
Spiegel teaches providing, using the remote system the image data for presenting the one or more frames using the client device, in paragraph 267 by means of a Convolutional Neural Networks (CNNs): CNNs may be used for image recognition and computer vision tasks. CNNs may for example, be designed to extract features from images. These extracted images capture the state of the media content, which can be of a game being played be a gaming system thereby current content of a game played would be the state of said game to extract information about said state of game.
Generating, using a remote system (par. 68 external resource system 228 provides an interface for the interaction client 104 to communicate with remote servers (e.g., third-party servers 112) to launch or access external resources, i.e., applications or applets) and based at least on first input data received from a client device, image data representative of at least one or more frames associated with a state of a gaming application (Fig. 3D, 314 receive additional media assets such as images and video; par. 40 gaming system; par. 41 playing games; par. 67 game system 226 provides various gaming functions within the context of the interaction client 104. The interaction client 104 provides a game interface providing a list of available games that can be launched by a user within the context of the interaction client 104 and played with other users of the interaction system 100);
Providing, using the remote system the image data for presenting the one or more frames using the client device (par. 267 Convolutional Neural Networks (CNNs): CNNs may be used for image recognition and computer vision tasks. CNNs may for example, be designed to extract features from images);
determining, using the remote system and based at least on processing the image data representative of the one or more frames, first information representative of the state of the gaming application (par. 21 Interactive platforms (e.g., social platforms, social media platforms AR platforms, applications, messaging platforms, AR applications, operating systems, gaming systems or applications, systems with which a user interacts, and the like) may provide a way for users to interact with other users with similar interests. FIG. 1 is a block diagram showing an example interaction system 100 of an interactive platform for facilitating interactions (e.g., exchanging text messages, conducting text audio and video calls, or playing games) over a network; par. 324 hardware environment for executing the software described on a computing environment).;
Note: that when “interaction system 100” is mentioned it can be in the form of exchanging text messages, conducting text audio and video calls, or playing games; in the analysis we use “playing games” as its intended purpose as posed by the claim limitations.
Further note: The interaction system 100 includes multiple client systems 102, each of which hosts multiple applications, including an interaction client 104 and other applications 106. The client system can be a gaming hardware which executes gaming applications and communicates with other gaming hardware where second to the nth user reside for interaction.
PNG
media_image1.png
523
400
media_image1.png
Greyscale
Receiving, using the remote system and from the client device, a query related to second information associated with the state of the gaming application (par. 116 the system 300 uses artificial intelligence to create content based upon received additional context such as information related product specs and relevant assets, since the information can be that of a played game, then information about the game played and/or state of game based upon image/video data analyzed can generated content on this information; par. 43 The data exchanged between the interaction clients 104 (e.g., interactions 120) and between the interaction clients 104 and the interaction server system 110 includes functions (e.g., commands to invoke functions) and payload data (e.g., text, audio, video, or other multimedia data; par. 47 Specifically, the API server 122 provides a set of interfaces (e.g., routines and protocols) that can be called or queried by the interaction client 104 and other applications 106 to invoke functionality of the interaction servers 124. Functions supported at least sending interaction data from client to client, communication of media files (e.g. images and video) from client to server), collection of media data, messages, application events etc.); par. 83 prompt or query by user can be composed of text data, audio data, image data, video data, electronic documents, links to data and the like);
generating, using the remote system and based at least on the one or more language models processing second input data representative of the first information representative of the state of the gaming application and the query, output data representative of a response that include the second information associated with the state of the gaming application (par. 45 The interaction server system 110 supports various services and operations that are provided to the interaction clients 104. Such operations include transmitting data to, receiving data from, and processing data generated by the interaction clients 104. This data may include message content, client device information, geolocation information, media augmentation and overlays, message content persistence conditions, interactive platform information, and live event information. Par. 76 implementation of a chatbot system 300 which is designed to communicate with a user on a client device to employ natural language processing (NLP) and machine learning (ML)/ artificial intelligence techniques to understand user’s input and generate a response; par. 77 the natural language understanding (NLU) components and dialogue management components are responsible for understanding user’s intent and extracting relevant information from user’s input; par. 78 the chatbot system utilizes generative AI model such as large language model (LLM) to improve NLU note fig.6 and 7) ;
and
causing using the remote system the client device to output the response associated with the query (par. 45 Data exchanges within the interaction system 100 are invoked and controlled through functions available via user interfaces (UIs) of the interaction clients 104); par. 86 The chatbot system 300 generates a raw response 362 based on the prompt of the user, generates an adjusted input prompt 368 based on filtering the raw response 362, and generates a new response based on adjusted input prompt 368).
Spiegel does not go into further detail about processing frames; however in the same field of endeavor Pardeshi teaches storing, by a remote system, first information representative of a previous state of a gaming application (par. 62 different types of recommendations 300 can be provided for a player based at least in part upon state determinations or suggestions for future state. In at least one embodiment, an object 302 such as a spider may appear in a game, as illustrated in a gameplay frame of FIG. 3A. In at least one embodiment, current and past state information can be used to determine an appropriate suggestion or recommendation for a player, as may be based upon a current role or goal for that player as determined using that state information); generating, using the remote system and based at least on first input data received from a client device, image data representative of at least one or more frames associated with a current state of the gaming application; providing, using the remote system, the image data for presenting the one or more frames using the client device; processing, using the remote system, the one or more frames to determine at least one of first text represented by the one or more frames or second text describing one or more graphical elements represented by the one or more frames (par. 61 provide game highlight and recommendation generation based on has game state analysis and suggestion. In at least one embodiment, context information can be cached and retained across events and other metadata. In at least one embodiment, such an approach can make sense of such events and various other situational and behavioral factors in a game. In at least one embodiment, game highlight forecasting can be provided, whereby predictions can be made as to whether subsequent video frames may reflect an event worthy of highlights; par. 62 and figures 3A-D; user queries input into the gaming application (i.e. playing the game); determining, using the remote system, second information representative of the current state of the gaming application, the second information including the at least one of the first text or the second text; updating, by the remote system, the first information representative of the previous state using the second information representative of the current state; generating, using the remote system and based at least on one or more language models processing second input data representative of the second information and a query associated with the current state of the gaming application, output data representative of a response that includes third information associated with the current state of the gaming application and causing using the remote system, the client device to output the response (par. 61 In at least one embodiment, this can involve suggesting actions a player may want to enact, rather than predicting whether such an action is about to occur. In at least one embodiment, such a system can be deployed to enable semi-supervised, real-time player state suggestions based on multiple inputs and context across game events; par. 62 might be suggested when that spider may change state at some point, which might trigger a necessary action by this player. In at least one embodiment, an alternative recommendation 308 might be to kill this spider as illustrated in FIG. 3C, such as where this spider is now an enemy or might otherwise harm this player. In at least one embodiment, an alternative recommendation 310 might be provided to not kill this spider as illustrated in FIG. 3D, such as where that spider would not now do a player harm or where that spider might help a player to achieve a particular result or goal. In at least one embodiment, this illustrates how changes in state can impact recommendations for a player for a same in-game event based on changes in associated objectives. In at least one embodiment, other types of recommendations or information can be provided as well, as may be provided in a separate window or display, or through audio or haptic mechanisms. In at least one embodiment, highlighting of objects to bring them to an attention of a player may be used during initial coaching of a player, but that highlighting might diminish over time as that player's skill increases; par. 51 , inputs can correspond to any information that can describe or affect state for a game, as may relate to a condition of a player relative to a game or a condition of a game itself. In at least one embodiment, game events can be acquired directly via a game engine or indirectly through a deep-learning game event recognition engine. In at least one embodiment, one or more neural networks can be used to recognize objects, actions, occurrences, audio, or scenes in input game media and provide keywords that can be provided as input to provide context for one or more game events, or that can help to identify types of events that occur during gameplay. In at least one embodiment, game data to be received can relate to a discrete set of information that is pre-determined and can be used to fill fields of a relevant schema for that type of data).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Pardeshi in to Spiegel because Pardeshi suggests using functionality within gaming application (such as Spiegel suggests gaming application) and Pardeshi specifically states in paragraph 404 “although discussion above sets forth example implementations of described techniques, other architectures may be used to implement described functionality, and are intended to be within scope of this disclosure. Furthermore, although specific distributions of responsibilities are defined above for purposes of discussion, various functions and responsibilities might be distributed and divided in different ways, depending on circumstances”.
As for claims 2 and 11, Spiegel teaches. The method of claim 1, further comprising:
retrieving, from one or more databases and based at least on at least one of the first information or the query, contextual information describing a context associated with the gaming application, wherein the second input data is further representative of the contextual information (par. 83 user can input multiple data prompts that can include text, image, audio, video and/or documents as input into the chatbot system to derive a response from user’s intent through prompt input by said user).
Pardeshi also teaches contextual information describing a context associated with the gaming application, wherein the second input data is further representative of the contextual information (par. 61 suggesting actions a player may want to enact, rather than predicting whether such an action is about to occur. In at least one embodiment, such a system can be deployed to enable semi-supervised, real-time player state suggestions based on multiple inputs and context across game events).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Pardeshi in to Spiegel because Pardeshi suggests using functionality within gaming application (such as Spiegel suggests gaming application) and Pardeshi specifically states “although discussion above sets forth example implementations of described techniques, other architectures may be used to implement described functionality, and are intended to be within scope of this disclosure. Furthermore, although specific distributions of responsibilities are defined above for purposes of discussion, various functions and responsibilities might be distributed and divided in different ways, depending on circumstances”.
As for claims 3 and 12, Spiegel teaches. The method of claim 2, wherein the contextual information comprises third text describing at least a portion of one or more of:
one or more documents associated with the gaming application (par. 83 document);
one or more videos associated with the gaming application (par. 83 video);
one or more instances of user speech associated with the gaming application (par. 83 audio); or
one or more graphics associated with the gaming application (par. 83 image data).
Pardeshi also teaches contextual information describing a context associated with the gaming application, wherein the second input data is further representative of the contextual information (par. 61 suggesting actions a player may want to enact, rather than predicting whether such an action is about to occur. In at least one embodiment, such a system can be deployed to enable semi-supervised, real-time player state suggestions based on multiple inputs and context across game events; par. 62 different text describing contextual information about the gameplay state).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Pardeshi in to Spiegel because Pardeshi suggests using functionality within gaming application (such as Spiegel suggests gaming application) and Pardeshi specifically states “although discussion above sets forth example implementations of described techniques, other architectures may be used to implement described functionality, and are intended to be within scope of this disclosure. Furthermore, although specific distributions of responsibilities are defined above for purposes of discussion, various functions and responsibilities might be distributed and divided in different ways, depending on circumstances”.
As for claims 4 and 13, Spiegel teaches. The method of claim 1, further comprising:
storing data representative of at least one of one or more previous queries associated with the gaming application or one or more previous responses associated with the gaming application, wherein the second input data is further representative of the at least one of the one or more previous queries or the one or more previous responses (par. 139 By analyzing the complete exchange comprising the full conversation, the LLM 338 can better understand the intent and meaning behind the user's queries. Additionally, having access to the full conversation flow enables the LLM 338 to maintain continuity and generate responses that logically follow from previous parts of the dialog).
As for claims 5, 11 and 14, Spiegel teaches. The method of claim 1, further wherein determination of the contextual information associated with the state of the gaming application comprises: generating one or more embeddings representative of the state information and the query; determining, based at least on searching the one or more databases, using one or more embeddings associated with second information, one or more databases to identify contextual information describing a context associated with the current state of the gaming application, wherein the second input data is further representative of the contextual information;
Specifically for claim 14 least image data representative of one or more frames, and wherein the determination of the first state information representative of the state associated with of the gaming application comprises: determining, based at least on the image data, at least one of: first text represented by the one or more frames; or second text describing one or more elements graphically represented by the one or more frames; and generating the state information to include the at least one of the first text or the second text describing the state of the gaming application (Fig. 3, item 324, par. 120 generative content from a LLM and par. 57-59 augmentation to video frames prompted into the system to add text, logos, animation, sound effects, pictures etc.. into the frames of the interaction system 100 ran on interaction client 104; par. 83 Regardless of the data type of the prompt 328, keyword attribution and expansion may be used to automatically generate a cluster of keywords or attributes that are associated with the received prompt 328. For example, image recognition may be deployed to identify objects and location associated with image data and to generate a keyword cluster or cloud that is then associated with the image-based prompt).
Pardeshi also teaches further comprising searching, using one or more embeddings associated with second information, one or more databases to identify contextual information describing a context associated with the current state of the gaming application, wherein the second input data is further representative of the contextual information (par. 62 different types of recommendations 300 can be provided for a player based at least in part upon state determinations or suggestions for future state. In at least one embodiment, an object 302 such as a spider may appear in a game, as illustrated in a gameplay frame of FIG. 3A. In at least one embodiment, current and past state information can be used to determine an appropriate suggestion or recommendation for a player, as may be based upon a current role or goal for that player as determined using that state information).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Pardeshi in to Spiegel because Pardeshi suggests using functionality within gaming application (such as Spiegel suggests gaming application) and Pardeshi specifically states “although discussion above sets forth example implementations of described techniques, other architectures may be used to implement described functionality, and are intended to be within scope of this disclosure. Furthermore, although specific distributions of responsibilities are defined above for purposes of discussion, various functions and responsibilities might be distributed and divided in different ways, depending on circumstances”.
As for claim 6, Spiegel teaches. The method of claim 1, further comprising obtaining content data that includes one or more of: first audio data representative of a first sound that is output using the client device; second audio data representative of a second sound captured using the client device; or third input data representative of one or more inputs received using the client device wherein the determining the second information representative of the current state of the gaming application is further based at least on processing the content data (fig. 3, item 322, par. 119 and par. 57-59 augmentation to video frames prompted into the system to add text, logos, animation, sound effects, pictures etc.. into the frames of the interaction system 100 ran on interaction client 104).
Pardeshi also teaches (par. 62 different types of recommendations 300 can be provided for a player based at least in part upon state determinations or suggestions for future state. In at least one embodiment, an object 302 such as a spider may appear in a game, as illustrated in a gameplay frame of FIG. 3A. In at least one embodiment, current and past state information can be used to determine an appropriate suggestion or recommendation for a player, as may be based upon a current role or goal for that player as determined using that state information).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Pardeshi in to Spiegel because Pardeshi suggests using functionality within gaming application (such as Spiegel suggests gaming application) and Pardeshi specifically states “although discussion above sets forth example implementations of described techniques, other architectures may be used to implement described functionality, and are intended to be within scope of this disclosure. Furthermore, although specific distributions of responsibilities are defined above for purposes of discussion, various functions and responsibilities might be distributed and divided in different ways, depending on circumstances”.
As for claims 7 and 15, Spiegel teaches. The method of claim 1, wherein the query is associated with how to perform a task related to the current state of the gaming application and the third information response includes one or more instructions for how to perform the task related to the current state of the gaming application (par. 120, fig. 3 item 324 using LLM for generative output based upon input of image and text to derive context of user needs and par. 57-59 augmentation to video frames prompted into the system to add text, logos, animation, sound effects, pictures etc.. into the frames of the interaction system 100 ran on interaction client 104; “A media overlay may include text or image data that can be overlaid on top of a photograph taken by the client system 102 or a video stream produced by the client system 102. In some examples, the media overlay may be a location overlay (e.g., Venice beach), a name of a live event, or a name of a merchant overlay (e.g., Beach Coffee House). In further examples, the image processing system 202 uses the geolocation of the client system 102 to identify a media overlay that includes the name of a merchant at the geolocation of the client system 102. The media overlay may include other indicia associated with the merchant. The media overlays may be stored in the databases 128 and accessed through the database server 126”).
Pardeshi also teaches (par. 62 different types of recommendations 300 can be provided for a player based at least in part upon state determinations or suggestions for future state. In at least one embodiment, an object 302 such as a spider may appear in a game, as illustrated in a gameplay frame of FIG. 3A. In at least one embodiment, current and past state information can be used to determine an appropriate suggestion or recommendation for a player, as may be based upon a current role or goal for that player as determined using that state information).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Pardeshi in to Spiegel because Pardeshi suggests using functionality within gaming application (such as Spiegel suggests gaming application) and Pardeshi specifically states “although discussion above sets forth example implementations of described techniques, other architectures may be used to implement described functionality, and are intended to be within scope of this disclosure. Furthermore, although specific distributions of responsibilities are defined above for purposes of discussion, various functions and responsibilities might be distributed and divided in different ways, depending on circumstances”.
As for claims 8 and 16, Spiegel teaches. The method of claim 1, further comprising:
generating the second input data to represent at least one or more first vectors representative of the second information and one or more second vectors representative of the query; and generating, based at least on one or more third vectors represented by the output data, third text corresponding to the response, generate, based at least on one or more fourth vectors represented by the output data fourth text corresponding to reponse(par. 90-91 chatbot system collects set of prompts and additional information during interactive session wherein the chatbot system maps set of prompts to a set of keywords and concepts comprising user intent vector; par. 244 use of Support Vector Machines (SVM) for supervised learning algorithm used for classification, regression and other tasks, the classification functions as a pointer in the art to map intent for a user based upon historic interaction data).
Pardeshi also teaches (par. 62 a graphical vector can be overlaid or replace another graphic to denote highlighting of an area and based upon user query input for the gaming application the system will predict and output text based upon graphical contextual information inferred; par. 51 , inputs can correspond to any information that can describe or affect state for a game, as may relate to a condition of a player relative to a game or a condition of a game itself. In at least one embodiment, game events can be acquired directly via a game engine or indirectly through a deep-learning game event recognition engine. In at least one embodiment, one or more neural networks can be used to recognize objects, actions, occurrences, audio, or scenes in input game media and provide keywords that can be provided as input to provide context for one or more game events, or that can help to identify types of events that occur during gameplay. In at least one embodiment, game data to be received can relate to a discrete set of information that is pre-determined and can be used to fill fields of a relevant schema for that type of data).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Pardeshi in to Spiegel because Pardeshi suggests using functionality within gaming application (such as Spiegel suggests gaming application) and Pardeshi specifically states “although discussion above sets forth example implementations of described techniques, other architectures may be used to implement described functionality, and are intended to be within scope of this disclosure. Furthermore, although specific distributions of responsibilities are defined above for purposes of discussion, various functions and responsibilities might be distributed and divided in different ways, depending on circumstances”.
As for claim 9, Spiegel teaches. The method of claim 1, wherein the causing the client device to output the response associated with the query comprises transmitting, to the client device, data that causes one or more of:
the client device to output sound associated with the response;
the client device to display third text associated with the response; or
the client device to display one or more graphical elements associated with the response (par. 83 output can be in the form of audio, document, image, text, etc.).
As for claim 17, Spiegel teaches. The system of claim 10, wherein the one or more processors are further to: receive the first data using at least one of a client device presenting content associated with the application or a system streaming the application, wherein: the query is received using the client device; and
the second information is sent to the client device in order to cause the client device to output the second information (par. 41 network environment for streaming application to interaction clients from interaction server; functionality described in claim 1 related to user query is discussed above).
As for claims 18 and 20, Spiegel teaches. The system of claim 10, wherein the system is comprised in at least one of:
a control system for an autonomous or semi-autonomous machine;
a perception system for an autonomous or semi-autonomous machine;
a system for performing one or more simulation operations;
a system for performing one or more digital twin operations;
a system for performing light transport simulation;
a system for performing collaborative content creation for 3D assets;
a system for performing one or more deep learning operations;
a system implemented using an edge device;
a system implemented using a robot;
a system for performing one or more generative AI operations;
a system for performing operations using one or more large language models (LLMs);
a system for performing one or more conversational AI operations;
a system for generating synthetic data;
a system for presenting at least one of virtual reality content, augmented reality content, or mixed reality content;
a system incorporating one or more virtual machines (VMs);
a system implemented at least partially in a data center; or
a system implemented at least partially using cloud computing resources (note the analysis of the previous claims which touch base on a plurality of these topics, the claim as of now requires only one to be true, that said generative AI model is used in the creation of output from the prompt input of the user; par. 78).
(Note:) It is noted that any citation to specific, pages, columns, lines, or figures in the prior art references and any interpretation of the references should not be considered to be limiting in any way. A reference is relevant for all it contains and may be relied upon for all that it would have reasonably suggested to one having ordinary skill in the art. In re Heck, 699 F.2d 1331, 1332-33, 216 USPQ 1038, 1039 (Fed. Cir. 1983) (quoting In re Lemelson, 397 F.2d 1006,1009, 158 USPQ 275, 277 (CCPA 1968)).
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Inquires
Any inquiry concerning this communication should be directed to NICHOLAS AUGUSTINE at telephone number (571)270-1056.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
PNG
media_image2.png
213
559
media_image2.png
Greyscale
/NICHOLAS AUGUSTINE/Primary Examiner, Art Unit 2178 August 13, 2026
Prosecution Insights