DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-20 are pending and examined in this office action.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3-4, 7, 10-12, 16-18 are rejected under 35 U.S.C. 103 as being unpatentable over Pisoni et al. (US patent 11967033) hereinafter Pisoni, in view of Volum et al. (US PGPUB 2023/0125036) hereinafter Volum.
Per claim 1, Pisoni discloses a method comprising: obtaining input data indicating one or more objects for a virtual environment; generating, using one or more language models and based at least on the input data, one or more tokens representative of one or more portions of code for rendering the one or more objects in the virtual environment (claims 1, 4; at a language model, receiving user input for rendering virtual objects in a virtual environment; generating a plurality of tokens representing a textual answer to the natural language input; the tokens are used for rendering virtual objects in the virtual environment); causing, using a simulation system and based at least on the text representing the code, the one or more objects to be rendered in the virtual environment (claims 1, 4; using the tokens to render virtual objects in the virtual environment).
Pisoni does not explicitly teach generating, using the one or more tokens and based at least on one or more syntax rules, text representing the code. However, Volum suggests the above (paragraphs [0004][0022]; receiving user input for generating 3D objects in a virtual environment, a language model generates programmatic code (text), which is used for rendering 3D objects in the virtual environment). Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Pisoni and Volum to generate tokens based on user input to a model, and to generate program code based on the tokens to render objects in a virtual environment, as this is the normal process used by a language model to generate outputs (see Hsiao, paragraph [0016], input to tokens, then to program code).
Per claim 3, Pisoni further suggests generating, based at least on processing the audio data using one or more automatic speech recognition (ASR) systems, text data corresponding to the speech; and applying the text data to the one or more language models, wherein the generating of the one or more tokens is based at least on the one or more language models processing the text data (claim 1; column 5, line 1-21; user input audio is processed and converted to text data, the text data is provided to the language model to generate tokens).
Per claim 4, Volum further suggests wherein the input data includes text data, the text data obtained using a graphical user interface presented on a user device, the graphical user interface associated with the simulation system that is rendering the virtual environment (claim 1; paragraphs [0032][0086]; user input text data into a text box, the language model renders virtual objects in a virtual environment based on user input).
Claims 7 + 12 recite similar limitations as claim 1. Thus, claims 7 + 12 are rejected under similar rationales as claim 1.
Claims 10-11 recite similar limitations as claims 2 and 3. Thus, claims 10-11 are rejected under similar rationales as claims 2 and 3.
Per claim 16, Pisoni further suggests the one or more features of the simulation environment comprise one or more simulated agents, and at least one of one or more behaviors or one or more attributes of the one or more simulated agents are defined using the code (claim 1; column 8, line 1-35; the LLM generates code to render a motor vehicle (agent) in a simulation environment; based on the user input, the LLM associate workflows (attributes) with the motor vehicle in the simulation environment).
Per claim 17, Volum further suggests wherein the code includes one or more application programming interface (API) calls to one or more APIs for rendering the one or more features of the simulation environment (claim 1; paragraphs [0023][0086][0128]; a LLM generates output that includes API calls for rendering objects in the simulation environment, using rendering libraries).
Per claim 18, Pisoni further suggests wherein the system is comprised in at least one of:
a system for performing one or more simulation operations (claim 1; column 8, line 1-35; the LLM generates code to render a motor vehicle in a simulation environment).
Claims 2 and 8 are rejected under 35 U.S.C. 103 as being unpatentable over Pisoni, in view of Volum, and in view of Wang et al. (US PGPUB 2022/0382527) hereinafter Wang.
Per claim 2, Pisoni does not explicitly teach applying, to the one or more language models, at least: a training dataset including one or more examples of code executable by the simulation system; and training data representing one or more user requests; generating, using the one or more language models and based at least on the training data and the training dataset, output data representing one or more lines of code; and updating one or more parameters of the one or more language models based at least on one or more differences between the one or more lines of code and one or more ground truth lines of code executable by the simulation system. However, Wang suggests the above (claims 1, 10; paragraphs [0014][0024]; receiving and applying a training dataset to a model, the training dataset includes program language code and natural language text (user requests); generating predicted program language code (output code); computing a loss (difference) by comparing the predicted PL segment and the PL segment (ground truth); and updating the model based on the loss via backpropagation). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Pisoni, Volum and Wang, to optimize a language model using the training method described in Wang, so the language model can produce more accurate results.
Claim 8 is rejected under similar rationales as claim 2.
Claims 5 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Pisoni, in view of Volum, and in view of Clement et al. (US PGPUB 2022/0308848) hereinafter Clement.
Per claim 5, Pisoni does not explicitly teach generating, using the one or more language models and based at least on the input data, one or more input tokens representative of the input data; and generating, using the one or more language models, one or more embeddings corresponding to the one or more input tokens, the one or more embeddings including one or more positional encodings, wherein the generating of the one or more tokens representative of the one or more portions of the code is based at least on the one or more embeddings and the one or more positional encodings. However, Clement suggests the above (claims 8-13; paragraph [0076]; receive input data, generate a sequence of tokens; the sequences of tokens are then mapped into numeric vectors and then into respective embeddings and positional embeddings, there is an embedding for each token in the vocabulary of a particular programming language and a corresponding positional embedding; generating an output token from a token from input sequence). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Pisoni, Volum and Clement to generate output from input data using embeddings with one or more positional encodings, as this method can produce more accurate code translation (Clement, paragraph [0028]).
Claim 13 is rejected under similar rationales as claim 5.
Claims 6 and 14-15 are rejected under 35 U.S.C. 103 as being unpatentable over Pisoni, in view of Volum, and in view of Qin (US PGPUB 20240346256).
Per claim 6, Pisoni does not explicitly teach obtaining, from one or more databases and based at least on analyzing the input data, one or more sources of information related to rendering objects in the virtual environment; and generating updated input data by adding the one or more sources of information to the input data, wherein the generating the one or more tokens is based at least on the updated input data. However, Qin suggests the above (paragraphs [0003][0018][0100]; analyzing a query (input data), retrieving augmentation information related to the query from a domain specific knowledge base; generating an augmented prompt (updated input data) based at least on the query and the retrieved augmentation information; performing tokenization on augmentation information). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Pisoni, Volum and Qin to generate updated input data based on retrieved augmentation information, this will produce more accurate response from a language model (Qin, paragraph [0018]).
Claim 14 is rejected under similar rationales as claim 6.
Per claim 15, Qin further suggests wherein the one or more sources of information comprise one or more documents including one or more examples of code (paragraphs [0003][0018][0100]; analyzing a query (input data), retrieving augmentation information (code examples) related to the query from a domain specific knowledge base; generating an augmented prompt (updated input data) based at least on the query and the retrieved augmentation information; performing tokenization on augmentation information). Pisoni further discloses generating code for rendering in the simulation environment one or more simulated vehicles (claim 1; column 8, line 1-35; the LLM generates code to render a motor vehicle (agent) in a simulation environment).
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Pisoni, in view of Volum, and in view of Mountjoy (US PGPUB 2011/0202845).
Per claim 9, Pisoni does not teach wherein the simulation environment is rendered using one or more light transport simulation algorithms. However, Mountjoy suggests the above (paragraph [0038]; a render engine module can render objects using ray tracing to give light effects and give the image a photorealistic appearance). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Pisoni, Volum and Mountjoy to use ray tracing to render a virtual environment, to give the virtual environment a photorealistic appearance.
Claims 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Volum, in view of Levinson et al. (US PGPUB 2017/0132334) hereinafter Levinson, and Mountjoy (US PGPUB 2011/0202845).
Per claim 19, Volum discloses a processing circuitry to generate, using one or more language models that process input data associated with a simulated environment, text representing one or more instructions that, when executed, cause a simulation system to update a representation of the simulated environment to include one or more simulated agents having one or more attributes corresponding to one or more first parameters indicated in the input data (claims 1, 6; paragraph [0043]; a language model receives a user input, generates code to render virtual objects in a simulation environment, including adding or updating a virtual object in the simulation environment; the virtual object can be an NPC (agent) having attributes specified in the user input).
Volum does not explicitly teach the simulated agent having one or more randomized attributes corresponding to one or more second parameters omitted from the input data. However, Levinson suggests the above (paragraph [0159]; the object data characterizer may implement randomized data, based on probabilities, relating to predicted range of motion, based on the randomized data, a simulator may be able to simulate possibly rare behaviors of an object, such as a dog randomly leaping up and running into the street). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Volum and Levinson to assign randomized attributes to a virtual object in a simulation environment, this would make the simulation environment more realistic.
Volum does not explicitly teach the simulated agent the simulated environment rendered by the simulation system using one or more light transport simulation algorithms. However, Mountjoy suggests the above (paragraph [0038]; a render engine module can render objects using ray tracing to give light effects and give the image a photorealistic appearance). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Volum, Levinson and Mountjoy to use ray tracing to render a virtual environment, to give the virtual environment a photorealistic appearance.
Per claim 20, Volum further suggests wherein the one or more processors are comprised in at least one of: a system for performing one or more conversational AI operations (paragraph [0004]; to enable a user to facilitate user interactions with a conversational agent).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HANG PAN whose telephone number is (571)270-7667. The examiner can normally be reached 9 AM to 5 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chat Do can be reached at 571-272-3721. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HANG PAN/Primary Examiner, Art Unit 2193
Prosecution Insights