Prosecution Insights
Last updated: October 28, 2026
Application No. 18/472,941

PROCESSING SENSOR DATA USING LANGUAGE MODELS IN MAP GENERATION SYSTEMS AND APPLICATIONS

Final Rejection §103
Filed
Sep 22, 2023
Priority
Jun 16, 2023 — provisional 63/521,627
Examiner
YAMAMOTO, JOSEPH JEREMY
Art Unit
2656
Tech Center
2600 — Communications
Assignee
NVIDIA Corporation
OA Round
4 (Final)
72%
Grant Probability
Favorable
5-6
OA Rounds
0m
Est. Remaining
99%
With Interview

Examiner Intelligence

Grants 72% — above average
72%
Career Allowance Rate
38 granted / 53 resolved
+9.7% vs TC avg
Strong +31% interview lift
Without
With
+31.1%
Interview Lift
resolved cases with interview
Typical timeline
2y 8m
Avg Prosecution
6 currently pending
Career history
70
Total Applications
across all art units

Statute-Specific Performance

§101
20.8%
-19.2% vs TC avg
§103
49.8%
+9.8% vs TC avg
§102
6.8%
-33.2% vs TC avg
§112
21.3%
-18.7% vs TC avg
Black line = Tech Center average estimate • Based on career data from 53 resolved cases

Office Action

§103
Notice of Pre-AIA or AIA Status The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . DETAILED ACTION Claims 1-20 are pending. Claims 1, 11, and 16 are independent. Claims 2-10 depends from Claim 1. Claims 12-15 depends from Claim 11. Claims 17-20 depends from Claim 16. This Application was published as U.S. 2024/0419903. Response to Amendment Examiner thanks Applicant for response filed on 16 Jun 2026 which has been correspondingly accepted and considered in this office action. Claims 1-20 are pending. Response to Arguments Applicant's arguments filed 16 Jun 2026 pages 8-26 have been fully considered but they are not persuasive. Each argument of Applicant’s arguments will be addressed in turn. With regards to the Information Disclosure Statement: Applicant's has paid the appropriate fees and submitted information disclosure statement in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner. With regards to Provisional Double Patenting: Applicant's remarks of 16 Jun 2026 page 9 have been considered, and the rejection will be maintained until the filing of a terminal disclaimer. With regards to 35 USC § 101: Applicant's arguments filed 16 Jun 2026 and amendments to independent claims 1, 11, and 16 have been fully considered and are persuasive. As a result, the 35 USC § 101 rejection is withdrawn. With regards to 35 USC § 103: Applicant's arguments filed 16 Jun 2026 have been fully considered but they are not persuasive. Applicant arguments will be addressed in turn. Claim 1: Applicant has amended claim 1; however, the Applicant arguments on pages 21-23 of 16 Jun 2026 Applicant arguments and amendments do not materially change the rejection or references previously presented. Li et al. (US2024/0395261 hereinafter Li) teaches tokenized representation that includes a feature vector that corresponds to the semantic, geographic, and topological characteristics of the environment as explained in previous office actions. An addition for this amendment is the inclusion of a latent space, which applicant admits that embeddings are points in a latent space because applicant uses terms interchangeable as “embeddings or points in a latent space.” (Applicant arguments page 23 16 Jun 2026; See Applicant Specification Par [0054]) Since Li also teaches mapping tokens to a “sequence of vectors sequence of vectors (also referred to as “embeddings”) which provide semantic information about the words represented in the prompt 103” (Par [0031]) then Li also teaches a latent space. Furthermore, Dintenfass (US2018/0158157 hereinafter Dintenfass) teaches generating a map which represents the inferred structural relationship because Dintenfass generates an augmented reality, so the Dintenfass reference is valid and remains to teach the remaining claim limitations. Thus for all the reasons presented above, Applicant arguments for 35 USC § 103 rejection of claim 1 is not persuasive. Claims 11 and 16: Applicant argues on pages 23-24 of 16 Jun 2026 Applicant arguments that are similar to the argument in claim 1. For brevity, the reasoning is similar to claim 1, and for all the reasons presented above, Applicant arguments for 35 USC § 103 rejections of claims 11 and 16 are not persuasive. Dependent Claims 2-6, 7-10, 12-15, and 17-20: Applicant argues on pages 24-25 of 16 Jun 2026 Applicant arguments that dependent claims are allowable since the independent claims are allowable. For all the reasons presented above, Applicant arguments for 35 USC § 103 rejections of dependent claims 2-6, 7-10, 12-15, and 17-20 are not persuasive. Thus for all the reasons presented above for claims 1-20, Applicant arguments for 35 USC § 103 rejections are not persuasive. Information Disclosure Statement The information disclosure statement (IDS) submitted on 16 Jun 2026 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner. Provisional Double Patenting As was presented in 24 Mar 2026 office action, and remarks received from Applicant remarks of 16 Jun 2026 page 9, the provisional double patenting rejecting will be maintained until the filing of a terminal disclaimer by the Applicant. Claim Rejections - 35 USC § 103 The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. Claims 1-5, and 7-20 are rejected under 35 U.S.C. 103 as being unpatentable over Li et al.(US2024/0395261) hereinafter Li in view of Dintenfass (US2018/0158157) hereinafter Dintenfass. With regards to claim 1, Li teaches: A method, comprising: generating, based at least on a set of observations captured using one or more sensors, [Li Fig 1 item 112 Par [0030] teaches “sensors 112 may include any suitable sensor technology that can be used to detect changes in a surrounding environment”] a tokenized representation of at least a portion of an environment based at least on the set of observations; [Li Par [0031] teaches “prompt creation component 114 is configured to generate a prompt 103 based on environmental changes detected by the sensors 112. For example, the prompt 103 may include a sequence of tokens that are suitable for processing by the NLP 120”] the tokenized representation including at least one feature vector generated by encoding features, including non-linguistic attributes of the environment, extracted from the set of observations, [Li Fig 1 teaches the prompt includes a “sequence of tokens” … [which] “may be further mapped to a sequence of vectors (also referred to as “embeddings”) which provide semantic information about the words represented in the prompt 103.” (Par [0031]) where the “virtual assistant may combine input data from different types of sensors (such as cameras and microphones) into the same NLP prompt.” (Par [0026]) where data from a camera includes visible attributes of the environment which are non-linguistic attributes as exemplified in Li Fig 3 which shows how image data (303) from a sensor is used to generate a prompt for a NLP interface, which includes non-linguistic elements such as image data extracted from the set of observations] wherein the at least one feature vector corresponds to at least one point in a latent space representing the environment and encodes semantic, geographic, and topological characteristics of one or more objects in the environment; [Li Fig 1 teaches feature vector that corresponds to a mapping “to a sequence of vectors (also referred to as “embeddings”) which provide semantic information about the words represented in the prompt 103.” (Par [0031]) which represents an embedding or points in a latent space, where another term for an embedding or vector space is a latent space. (see Nasir et al. (US2024/0177513) Par [0021]) Furthermore, Li Figures 2-3 teaches VA system that represents multiple modalities that include image and audio data, where image data represents the visible inferred structural relationships between the topological, geometric, or relational information of the features because images visible show these relationships, where an image is a visible representation of the structures in the image which are an inferred structural relationships, and Li creates tokens or prompts (Li Par [0004]) which represent the tokenized representation of the set of observations.] processing the tokenized representation with a language model, [Li Par [0031] teaches “prompt creation component 114 may encode the text into a sequence of tokens that can be processed by the NLP 120”] to generate a tokenized description of at least the portion of the environment, the tokenized description representing an inferred structural relationship between two or more of the features [Li teaches “the tokens may be further mapped to a sequence of vectors (also referred to as “embeddings”) which provide semantic information about the words represented in the prompt 103.” (Par [0031]) Li Figures 2-3 teach VA system represents multiple modalities that include image and audio data, where image data represents the visible inferred structural relationships between the topological, geometric, or relational information of the features because images visible show these relationships, and Li creates tokens or prompts (Li Par [0004]) which represent the tokenized representation of the set of observations] representing by the at least one feature vector, where the inferred structural relationship is determined based at least on semantic, topological, geometric, kinematic, or relational information associated with the one or more objects encoded in the tokenized representation; and [Li Fig 1 teaches feature vector that corresponds to a mapping “to a sequence of vectors (also referred to as “embeddings”) which provide semantic information about the words represented in the prompt 103.” (Par [0031]) which represents an embedding or vector space, where another term for an embedding or vector space is a latent space. (see Nasir et al. (US2024/0177513) Par [0021]) Furthermore, Li Figures 2-3 teaches VA system that represents multiple modalities that include image and audio data, where image data represents the visible inferred structural relationships between the topological, geometric, or relational information of the features because images visible show these relationships, where an image is a visible representation of the structures in the image which are an inferred structural relationships, and Li creates tokens or prompts (Li Par [0004]) which represent the tokenized representation of the set of observations.] With regards to claim 1, Li fails to teach: generating, using the tokenized description, map of at least the portion of the environment, the map representing the inferred structural relationship. With regards to claim 1, Dintenfass teaches: generating, using the tokenized description, map of at least the portion of the environment, [Dintenfass Fig 7 item 714 teaches “augmented reality user device 200 generates a map based on neighborhood information provided by the virtual assessment data 111” (Par [0162]) where at “step 712, the augmented reality user device 200 receives virtual assessment data 111 from the remote server 102 in response to sending the property token 110 to the remote server 102.” (Par [0161]) the map representing the inferred structural relationship. [Dintenfass Fig 7 item 720 teaches a map as representing the “virtual object in the real scene in front of the user” which is an inferred structural relationship. It would be obvious to one of ordinary skill in the art at time of applicant’s effective filing date to combine the virtual assistant taught by Li with the augmented reality device of Dintenfass. The motivation to combine the inventions of Li and Dintenfass is because Dintenfass teaches “augmented reality user device 200 uses geographic location information provided by a GPS sensor with a map database to determine the location of the user 106” (Par [0042]) which increases the capabilities of the invention of Li to interact with user using user location data] With regards to claim 2, Li in view of Dintenfass teaches: All the limitations of claim 1 wherein the tokenized representation of the set of observations is generated using the language model, a second language model, or a data encoder. [Li teaches “prompt 103 may include a sequence of tokens that are suitable for processing by the NLP 120” (Par [0031]) where NLP is a natural language processing model] With regards to claim 3, Li in view of Dintenfass teaches: All the limitations of claim 1 wherein the map is generated using the tokenized description corresponding to the set of observations using an automated end-to-end process. [Dintenfass Fig 7 teaches end-to-end process for steps 706-714] With regards to claim 4, Li in view of Dintenfass teaches: All the limitations of claim 1 wherein the map generated using the tokenized description includes one or more maps, or sets of map data, in one or more of a set of map formats. [Dintenfass Par [0162] teaches “augmented reality user device 200 generates a two-dimensional or a three-dimensional map that overlays the neighborhood information with a geographical map.”] With regards to claim 5, Li in view of Dintenfass teaches: All the limitations of claim 1 wherein the tokenized description is a tokenized text string representative of at least the portion of the environment, the tokenized text string including a sequence of tokens associated with objects in the environment. [Li Par [0031] teaches “tokens may be further mapped to a sequence of vectors (also referred to as “embeddings”) which provide semantic information about the words represented in the prompt 103” where semantic information is a feature of the environment] With regards to claim 7, Li in view of Dintenfass teaches: All the limitations of claim 1 wherein set of observations further includes at least one of lighting data, weather data, human annotations, prior map data, or other data relevant to use cases considered. [Li Par [0030 teaches “suitable sensor technologies may include audio and visual sensor technologies, among other examples. In some implementations, the sensors 112 may include one or more microphones that are configured to detect sounds (such as the user speech 102) propagating through the environment” where visual sensors like cameras includes lighting data, and audio sensors like microphones includes speech or human annotations] With regards to claim 8, Li in view of Dintenfass teaches: All the limitations of claim 1 wherein at least a subset of observations is captured using one or more sensors on a machine positioned in, or moving through, the portion of the environment. [Li Fig 1 Par [0029] teaches “environment 100 includes a user 101, a virtual assistant 110, and a natural language processor (NLP) 120” where “virtual assistant 110 includes one or more sensors 112” (Par [0030]) where the user and sensors are positioned in, or moving through, the portion of the environment where the user is located] With regards to claim 9, Li in view of Dintenfass teaches: All the limitations of claim 1 wherein the sensors include at least one of camera sensors, radar sensors, LiDAR sensors, ultrasonic sensors, or depth sensors. [Li Par [0082] teaches “sensors 502 may include one or more depth sensing technologies that can be used to determine the distance of Person X. Example suitable depth sensing technologies include radio detection and ranging (RADAR) and light detection and ranging (LiDAR), among other examples. In some other implementations, the subsystem controller 310 may estimate the distance of Person X based on images captured by a monocular camera”] With regards to claim 10, Li in view of Dintenfass teaches: All the limitations of claim 1 wherein the tokenized description of at least the portion of the environment generated by the language model includes at least one additional feature, corrected feature, or enhanced feature with respect to the features contained in the tokenized representation of the set of observations. [Li teaches “input sources 210 may include any suitable sensor technology that can be used to detect changes in a surrounding environment (such as cameras, motion sensors, or microphones). Thus, the input data 201 may include sensor data indicating the changes detected by the sensor” (Par [0041]) where motion sensors measure change in kinematics which is at least one additional feature] With regards to claim 11, Li teaches: A processor, comprising: one or more circuits to: [Li Par [0022] teaches “various illustrative logical blocks, modules, circuits and instructions described in connection with the embodiments disclosed herein may be executed by one or more processors (or a processing system”] generate, based at least on a large language model (LLM) processing sensor data obtained using one or more sensors, (Li Fig 1 item 112 sensors) a tokenized description of at least the portion of an environment, [Li Par [0031] teaches “prompt creation component 114 is configured to generate a prompt 103 based on environmental changes detected by the sensors 112. For example, the prompt 103 may include a sequence of tokens that are suitable for processing by the NLP 120”] the tokenized description representing an inferred structural relationship between features of at least the portion of the environment, the features represented by at least one feature vector generated by encoding features from the sensor data, including non-linguistic attributes of the environment, [Li Fig 1 teaches the prompt includes a “sequence of tokens” … [which] “may be further mapped to a sequence of vectors (also referred to as “embeddings”) which provide semantic information about the words represented in the prompt 103.” (Par [0031]) where the “virtual assistant may combine input data from different types of sensors (such as cameras and microphones) into the same NLP prompt.” (Par [0026]) where data from a camera includes visible attributes of the environment which are non-linguistic attributes as exemplified in Li Fig 3 which shows how image data (303) from a sensor is used to generate a prompt for a NLP interface, which includes non-linguistic elements such as image data extracted from the set of observations] wherein the at least one feature vector corresponds to at least one point in a latent space representing the environment and encodes semantic, geographic, and topological characteristics of one or more objects in the environment, [Li Fig 1 teaches feature vector that corresponds to a mapping “to a sequence of vectors (also referred to as “embeddings”) which provide semantic information about the words represented in the prompt 103.” (Par [0031]) which represents an embedding or points in a latent space, where another term for an embedding or vector space is a latent space. (see Nasir et al. (US2024/0177513) Par [0021]) Furthermore, Li Figures 2-3 teaches VA system that represents multiple modalities that include image and audio data, where image data represents the visible inferred structural relationships between the topological, geometric, or relational information of the features because images visible show these relationships, where an image is a visible representation of the structures in the image which are an inferred structural relationships, and Li creates tokens or prompts (Li Par [0004]) which represent the tokenized representation of the set of observations.] and determined based on at least one of semantic, topological, geometric, kinematic, or relational information of the features represented in the sensor data; and [Li teaches “the tokens may be further mapped to a sequence of vectors (also referred to as “embeddings”) which provide semantic information about the words represented in the prompt 103.” (Par [0031]) Li Figures 2-3 teach VA system represents multiple modalities that include image and audio data, where image data represents the visible inferred structural relationships between the topological, geometric, or relational information of the features because images visible show these relationships, and Li creates tokens or prompts (Li Par [0004]) which represent the tokenized representation of the set of observations] With regards to claim 11, Li fails to teach: generate, based at least on the tokenized description, a map for at least the portion of the environment, the map comprising a machine-readable representation configured for execution by a simulation system, and representing the inferred structural relationship. With regards to claim 11, Dintenfass teaches: generate, based at least on the tokenized description, a map for at least the portion of the environment, [Dintenfass Fig 7 item 714 teaches “augmented reality user device 200 generates a map based on neighborhood information provided by the virtual assessment data 111” (Par [0162]) where at “step 712, the augmented reality user device 200 receives virtual assessment data 111 from the remote server 102 in response to sending the property token 110 to the remote server 102.” (Par [0161])] the map comprising a machine-readable representation configured for execution by a simulation system, and [Dintenfass Fig 7 item 714 teaches “augmented reality user device 200” (Par [0162]) is a simulation system, and “Method 700 is employed by the processor 202 of the augmented reality user device 200 to generate property tokens 110 based on the user 106 of the augmented reality user device 200 and the location of the user 106” (Par [0156]) where processors provide machine-readable representations to be performed on the augmented reality simulation system. representing the inferred structural relationship. [Dintenfass Fig 7 item 720 teaches a map as representing the “virtual object in the real scene in front of the user” which is an inferred structural relationship. It would be obvious to one of ordinary skill in the art at time of applicant’s effective filing date to combine the virtual assistant taught by Li with the augmented reality device of Dintenfass. The motivation to combine the inventions of Li and Dintenfass is because Dintenfass teaches “augmented reality user device 200 uses geographic location information provided by a GPS sensor with a map database to determine the location of the user 106” (Par [0042]) which increases the capabilities of the invention of Li to interact with user using user location data] With regards to claim 12, Li in view of Dintenfass teaches: All the limitations of claim 11 wherein the tokenized description of at least the portion of the environment is generated using a tokenized representation of the sensor data. [Li Par [0031] teaches “prompt creation component 114 is configured to generate a prompt 103 based on environmental changes detected by the sensors 112. For example, the prompt 103 may include a sequence of tokens that are suitable for processing by the NLP 120”] With regards to claim 13, Li in view of Dintenfass teaches: All the limitations of claim 11 wherein the tokenized description is a tokenized text string representative of at least the portion of the environment, the tokenized text string including a sequence of tokens associated with one or more objects or the features in the portion of the environment. [Li Par [0031] teaches “tokens may be further mapped to a sequence of vectors (also referred to as “embeddings”) which provide semantic information about the words represented in the prompt 103” where semantic information are features in the portion of the environment] With regards to claim 14, Li in view of Dintenfass teaches: All the limitations of claim 11 wherein the tokenized description of at least the portion of the environment includes at least one additional feature, corrected feature, or enhanced feature with respect to the features contained in the tokenized representation. [Li teaches “input sources 210 may include any suitable sensor technology that can be used to detect changes in a surrounding environment (such as cameras, motion sensors, or microphones). Thus, the input data 201 may include sensor data indicating the changes detected by the sensor” (Par [0041]) where motion sensors measure change in kinematics which is at least one additional feature] With regards to claim 15, Li in view of Dintenfass teaches: All the limitations of claim 11 wherein the simulation system comprises at least one of: a system for performing simulation operations; a system for performing simulation operations to test or validate autonomous machine applications; a system for performing digital twin operations; a system for performing light transport simulation; a system for rendering graphical output; a system for performing deep learning operations; [Li teaches deep learning operations such as “suitable neural networks for natural language processing include recurrent neural networks (RNNs), long short-term memory (LSTM) networks, and transformers, among other example”(Par [0033])] a system for performing generative AI operations using a large language model (LLM); [Li teaches “natural language processing models (also referred to as large language models or “LLMs”)” (Par[0024])] a system implemented using an edge device; a system for generating or presenting virtual reality (VR) content; a system for generating or presenting augmented reality (AR) content; [Dintenfass Fig 1 teaches “augmented reality system 100”] a system for generating or presenting mixed reality (MR) content; a system incorporating one or more Virtual Machines (VMs); a system implemented at least partially in a data center; a system for performing hardware testing using simulation; a system for performing generative operations using a language model (LM); a system for synthetic data generation; a collaborative content creation platform for 3D assets; or a system implemented at least partially using cloud computing resources With regards to claim 16, Li teaches: A system, comprising: one or more processors [Li Par [0022] teaches “various illustrative logical blocks, modules, circuits and instructions described in connection with the embodiments disclosed herein may be executed by one or more processors (or a processing system”] based at least on a tokenized description of at least a portion of the environment, the tokenized description representing an inferred structural relationship between features of at least the portion of the environment, [Li Par [0031] teaches “prompt creation component 114 is configured to generate a prompt 103 based on environmental changes detected by the sensors 112. For example, the prompt 103 may include a sequence of tokens that are suitable for processing by the NLP 120”] the features represented by at least one feature vector generated by encoding features from a set of observations of the environment determined using one or more sensors, including non-linguistic attributes of the environment, [Li Fig 1 teaches the prompt includes a “sequence of tokens” … [which] “may be further mapped to a sequence of vectors (also referred to as “embeddings”) which provide semantic information about the words represented in the prompt 103.” (Par [0031]) where the “virtual assistant may combine input data from different types of sensors (such as cameras and microphones) into the same NLP prompt.” (Par [0026]) where data from a camera includes visible attributes of the environment which are non-linguistic attributes as exemplified in Li Fig 3 which shows how image data (303) from a sensor is used to generate a prompt for a NLP interface, which includes non-linguistic elements such as image data extracted from the set of observations] where the at least one feature vector corresponds to at least one point in a latent space representing the environment and encodes semantic, geographic, and topological characteristics of one or more objects in the environment, and [Li Fig 1 teaches feature vector that corresponds to a mapping “to a sequence of vectors (also referred to as “embeddings”) which provide semantic information about the words represented in the prompt 103.” (Par [0031]) which represents an embedding or vector space, where another term for an embedding or vector space is a latent space. (see Nasir et al. (US2024/0177513) Par [0021]) Furthermore, Li Figures 2-3 teaches VA system that represents multiple modalities that include image and audio data, where image data represents the visible inferred structural relationships between the topological, geometric, or relational information of the features because images visible show these relationships, where an image is a visible representation of the structures in the image which are an inferred structural relationships, and Li creates tokens or prompts (Li Par [0004]) which represent the tokenized representation of the set of observations.] generated based at least on a language model processing a set of observations of the environment determined using one or more sensors, wherein the inferred structural relationship is determined based at least on semantic, topological, geometric, kinematic, or relational information associated with the one or more objects represented by the at least one feature vector, [Li Fig 1 teaches feature vector that corresponds to a mapping “to a sequence of vectors (also referred to as “embeddings”) which provide semantic information about the words represented in the prompt 103.” (Par [0031]) which represents an embedding or points in a latent space, where another term for an embedding or vector space is a latent space. (see Nasir et al. (US2024/0177513) Par [0021]) Furthermore, Li Figures 2-3 teaches VA system that represents multiple modalities that include image and audio data, where image data represents the visible inferred structural relationships between the topological, geometric, or relational information of the features because images visible show these relationships, where an image is a visible representation of the structures in the image which are an inferred structural relationships, and Li creates tokens or prompts (Li Par [0004]) which represent the tokenized representation of the set of observations.] With regards to claim 16, Li fails to teach: to generate a map of an environment the generated map comprising a machine-readable representation configured to be executed by a simulation system, and representing the inferred structural relationship. With regards to claim 16, Dintenfass teaches: to generate a map of an environment [Dintenfass Fig 7 item 714 teaches “augmented reality user device 200 generates a map based on neighborhood information provided by the virtual assessment data 111” (Par [0162]) where at “step 712, the augmented reality user device 200 receives virtual assessment data 111 from the remote server 102 in response to sending the property token 110 to the remote server 102.” (Par [0161]) the generated map comprising a machine-readable representation configured to be executed by a simulation system, and [Dintenfass Fig 7 item 714 teaches “augmented reality user device 200” (Par [0162]) is a simulation system, and “Method 700 is employed by the processor 202 of the augmented reality user device 200 to generate property tokens 110 based on the user 106 of the augmented reality user device 200 and the location of the user 106” (Par [0156]) where processors provide machine-readable representations to be performed on the augmented reality simulation system. representing the inferred structural relationship. [Dintenfass Fig 7 item 720 teaches a map as representing the “virtual object in the real scene in front of the user” which is an inferred structural relationship. It would be obvious to one of ordinary skill in the art at time of applicant’s effective filing date to combine the virtual assistant taught by Li with the augmented reality device of Dintenfass. The motivation to combine the inventions of Li and Dintenfass is because Dintenfass teaches “augmented reality user device 200 uses geographic location information provided by a GPS sensor with a map database to determine the location of the user 106” (Par [0042]) which increases the capabilities of the invention of Li to interact with user using user location data] With regards to claim 17, Li in view of Dintenfass teaches: All the limitations of claim 16 wherein the language model is configured to process a tokenized representation of the set of observations. [Li Par [0031] teaches “prompt creation component 114 is configured to generate a prompt 103 based on environmental changes detected by the sensors 112. For example, the prompt 103 may include a sequence of tokens that are suitable for processing by the NLP 120”] Claim 18 is a system claim with limitations corresponding to the limitations of method Claim 13 and is rejected under similar rationale. Claim 19 is a system claim with limitations corresponding to the limitations of method Claim 14 and is rejected under similar rationale. Claim 20 is a system claim with limitations corresponding to the limitations of method Claim 15 and is rejected under similar rationale. Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Li et al(US2024/0395261) in view of Dintenfass (US2018/0158157) in further view of Bouguerra et al.(US2024/0354491) hereinafter Bouguerra. With regards to claim 6, Li in view of Dintenfass teaches: All the limitations of claim 5 With regards to claim 6, Li in view of Dintenfass fails to teach: wherein the tokenized text string is written in a road topology language (RTL) or a domain specific language (DSL). With regards to claim 6, Bouguerra teaches: wherein the tokenized text string is written in a road topology language (RTL) or a domain specific language (DSL). [Bouguerra teaches “digest engine 300 can be configured with one or more artificial intelligence (AI) algorithm(s) and/or machine learning (ML) model(s) and/or large language models (LLM)” (Par [0026]) and “ LLM can be configured to first tokenize text input into a sequence of words … [and] the digest engine 300 is not limited to utilizing a transformer large language model but can utilize any type of machine learning model(s) such as a zero-shot, domain-specific, fine-tuned, language representation, bidirectional encoder representations from transformers, multimodal, vector representation or any combination thereof.” It would be obvious to one of ordinary skill in the art at time of applicant’s effective filing date to combine the virtual assistant taught by Li in view of Dintenfass with the digest engine as taught by Bouguerra. The motivation to combine the inventions of Li and Dintenfass with the invention of Bouguerra is because Bouguerra uses an LLM and “Once trained the LLM is capable of generating a prompt to display to a user in the form of an output text, (e.g., automated output” (Par [0026]) which increases the capabilities of the invention of Li in view of Dintenfass to interact with users using prompts] Conclusion THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action. Any inquiry concerning this communication or earlier communications from the examiner should be directed to Joseph J Yamamoto whose telephone number is (571)272-4020. The examiner can normally be reached M-F 1000-1800 EST. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Bhavesh Mehta can be reached at 571-272-7453. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. JOSEPH J. YAMAMOTO Examiner Art Unit 2656 /BHAVESH M MEHTA/Supervisory Patent Examiner, Art Unit 2656
Read full office action

Prosecution Timeline

Show 7 earlier events
Feb 25, 2026
Examiner Interview Summary
Mar 06, 2026
Request for Continued Examination
Mar 09, 2026
Response after Non-Final Action
Mar 24, 2026
Non-Final Rejection mailed — §103
Jun 02, 2026
Applicant Interview (Telephonic)
Jun 02, 2026
Examiner Interview Summary
Jun 16, 2026
Response Filed
Sep 16, 2026
Final Rejection mailed — §103 (current)

Precedent Cases

Applications granted by this same examiner with similar technology

Patent 12751662
METHOD FOR EVALUATING POSSIBILITY OF DYSPHAGIA BY ANALYZING ACOUSTIC SIGNALS, AND SERVER AND NON-TRANSITORY COMPUTER-READABLE RECORDING MEDIUM PERFORMING SAME
2y 8m to grant Granted Oct 06, 2026
Patent 12749495
METHODS AND SYSTEMS OF NOISE AWARE AUDIO VISUAL SPEECH DENOISING
2y 7m to grant Granted Sep 29, 2026
Patent 12718031
NATURAL LANGUAGE GENERATOR DATA GENERATION
2y 6m to grant Granted Aug 25, 2026
Patent 12711943
CODE-MIXED SPEECH ENGINE IN A SPEECH SYNTHESIS SYSTEM
2y 10m to grant Granted Aug 18, 2026
Patent 12694893
AUDIO DATA PROCESSING
2y 5m to grant Granted Jul 28, 2026
Study what changed to get past this examiner. Based on 5 most recent grants.

Strategy Recommendation AI-generated — please review before filing

Get a prosecution strategy drawn from examiner precedents, rejection analysis, and claim mapping.
Typically takes 5-10 seconds — AI-generated, attorney review required before filing

Prosecution Projections

5-6
Expected OA Rounds
72%
Grant Probability
99%
With Interview (+31.1%)
2y 8m (~0m remaining)
Median Time to Grant
High
PTA Risk
Based on 53 resolved cases by this examiner. Grant probability derived from career allowance rate.

Sign in with your work email

Enter your email to receive a magic link. No password needed.

Personal email addresses (Gmail, Yahoo, etc.) are not accepted.

Free tier: 3 strategy analyses per month