DETAILED ACTION
This communication is in response to the Amendments and Arguments filed on 06/02/2026. Claims 1, 3-21 are pending and have been examined.
Any previous objection/rejection not mentioned in this Office Action has been withdrawn by the Examiner.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 06/02/2026 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 06/02/2026 has been entered.
Change of Examiner
The Examiner of record has changed from Feng-Tzer Tzeng to SPE Paras Shah.
Response to Amendments and Arguments
With respect to the 35 USC 101 rejections of claims 1-20, the Applicant asserts various arguments which will be addressed below:
Applicant asserts on page 11:
Applicant submits that claim 1, as amended, recites a specific set of operations for generating structured, machine-readable representations of an environment using one or more neural-network-based language models. In particular, claim 1 recites generating a first tokenized description of at least a portion of an environment from map data, and generating a second tokenized description that includes additional detail relating to at least one gap or omission in the map data, inferred using semantic, topological, geometric, kinematic, or relational information represented in the first tokenized description. Applicant respectfully submits that amended claim 1 is directed to a particular machine-implemented workflow for transforming map data into structured tokenized representations and automatically enhancing those representations using learned relationships encoded in a neural-network-based language model.
As described in the specification, map data is processed to extract features and generate a tokenized description of an environment using a language model, where the tokenized description captures semantic, topological, and geometric relationships among objects (Spec. [0038], [0040]-[0041], [0060]-[0062]). The specification further explains that a second language model processes the first tokenized description to generate a higher-resolution representation that includes additional inferred detail, such as lane structures, intersection topology, and other features that may not be present in the original map data (Spec. [0039], [0064]). The specification also explains that such language-model-based processing enables identification and correction of gaps or omissions in the input data and improves the completeness and accuracy of generated environment representations (Spec. [0041], [0051], [0064], [0090]).
The Examiner respectfully disagrees. More specifically the Applicant submits that the amended claim 1 is directed to “to a particular machine-implemented workflow for transforming map data into structured tokenized representations and automatically enhancing those representations using learned relationships encoded in a neural-network-based language model”. The claims make use of a “neural network-based learning model”. This model has been interpreted as an additional limitation. This additional limitation of the “neural network based language model” is simply being used a took for which the generation of the first and second tokenized description takes place. There is no specifics provided with respect to the “neural network language model” with respect to how this model is being trained or the architecture in sufficient detail. The Examiner reasserts that this has been interpreted as a general purpose LM and as noted in para 42 of the as filed specification defining this as a LLM. LLMs are widely known in the art and absent any specific training that is mentioned or specific architecture within this model, the model will be interpreted as a general purpose LLM. The Applicant also added language of “trained using map data”. However, there no details provided as to how the model is trained “using map”. Further, the Applicant’s statement of “generate a higher-resolution representation that includes additional inferred detail, such as lane structures, intersection topology, and other features that may not be present in the original map data” is not specifically claimed. The Examiner reminds the Applicant that the current claims must reflect these aspects in order to be considered. However, the claims only mention that the second tokenized description is includes detail “relating to at least one gap or omission”. This is not the same as “higher-resolution representation that includes additional inferred detail, such as lane structures, intersection topology, and other features that may not be present in the original map data”.
The Applicant asserts on page 12-13:
Moreover, features recited in the amended claim 1 cannot be practically performed in the human mind, much like the claims in Research Corp. Tech. v. Microsoft Corp., which were directed toward the manipulation of data structures in order to render an image. See, Research Corp. Tech. v. Microsoft Corp., 627 F.3d 859, 868 (Fed. Cir. 2010).
In a similar fashion, amended claim 1 requires extracting features from map data, encoding those features as representations, and generating tokenized descriptions of an environment using a neural-network-based language model. The specification explains that map data is processed to extract semantic, topological, and geometric features, which are encoded as feature vectors, embeddings, or points in a latent space and provided as input to a language model. See Spec [0060]-[0062].
The specification further describes that the language model generates a first tokenized description of the environment based on these encoded features, and that a second language model processes the first tokenized description to generate a higher-resolution representation that includes additional inferred detail, such as lane structures, intersection topology, and other environmental features. See Spec. [0038]-[0039], [0064], [0090]. The specification further explains that the language model can identify and correct gaps or omissions in the input map data and infer relationships among environment elements using learned representations. See Spec [0041], [0051], [0064].
The claimed operations cannot practically be performed in the human mind, particularly given the claimed feature extraction, encoding, and neural-network-based inference operations.
Accordingly, Applicant respectfully submits that claim 1 does not recite a mental process and therefore does not recite an abstract idea.
The Examiner respectfully disagrees. Majority of the arguments presented by the Applicant refer to aspects of the invention that are not currently claimed. In order for the claims to reflect the specific subject matter being mentioned in the Specification and for consideration by the Examiner, the claims should incorporate the subject matter noted. Among the details being noted which is not presently in the claims is “ map data is processed to extract semantic, topological, and geometric features, which are encoded as feature vectors, embeddings, or points in a latent space”, “generate a higher-resolution representation that includes additional inferred detail, such as lane structures, intersection topology, and other environmental features”. With the exception of the “neural network based language model” being interpreted as an additional element, each of the limitations can be reasonably be performed mentally. For example, a human can remember map information, location of places etc, and create a description from memory regarding the current environment where they are driving. Then, the human can based on that textual description of the environment create a more detailed representation by referring to additional information that they remember from a physical map to explain more about their location.
The Applicant asserts on page 13:
For example, embodiments of the present disclosure are directed to a computer- implemented framework for generating and enhancing structured, machine-readable representations of environments using neural-network-based language models. In particular, the claimed method generates a first tokenized description of an environment from map data and generates a second tokenized description that includes additional detail relating to gaps or omissions in the map data, inferred using semantic, topological, geometric, kinematic, or relational information. See, e.g., Spec. [0038]-[0039], [0064], [0090].
The specification explains that map data is processed to extract features, which may be encoded as feature vectors, embeddings, or latent-space representations and provided as input to a language model to generate tokenized descriptions of the environment. See Spec. [0060]-[0062]. The specification further explains that the tokenized description functions as a structured representation, such as an object graph, capturing relationships among environment elements. See Spec. [0040]-[0041], [0071].
Additionally, the specification describes that a second language model processes the first tokenized description to generate a higher-resolution representation that includes inferred environmental detail, such as lane structures, intersection topology, and additional objects, thereby filling gaps or omissions in the underlying map data. See Spec. [0039], [0064], [0090]. The specification further explains that these enhanced representations can be used to generate high- definition maps, simulation environments, and other machine-consumable outputs used in autonomous systems and related applications. See Spec. [0040], [0043], [0067].
Importantly, the claimed operations do not merely analyze or display information. Instead, they transform input map data into structured, tokenized representations and generate enhanced environment models that are usable by downstream computer systems, such as autonomous navigation systems and simulation platforms. This demonstrates that the claimed invention applies the recited concepts to achieve a concrete technological result-namely, improved generation, completeness, and usability of machine-readable environment representations.
Accordingly, the practical application of claim 1 is directed toward improving the operation of computer-based mapping, simulation, and environment modeling systems through automated generation and enhancement of structured environment representations using neural-network- based inference, rather than merely analyzing information in the abstract.
The Examiner respectfully disagrees. The Applicant has failed to provide what the additional limitations are that in combination with the judicial exception tie the claims to a practical application. Rather, as noted above, the Applicant re-iterates the claim language and that such is directed to computer implemented framework. Then, the Applicant continues to refer to the specification and notes aspects of the specification in explaining that the claims are directed towards improving operation of computer based mapping, simulation and environment modeling. The issue is all of these aspects are not reflected in the current claims as noted in the prior section above. The aspects related to “feature vectors, embeddings, or latent-space representations”, “generate a higher-resolution representation that includes inferred environmental detail, such as lane structures, intersection topology, and additional objects, thereby filling gaps or omissions in the underlying map data”, and “usable by downstream computer systems, such as autonomous navigation systems and simulation platforms” are not currently captured by the claims and therefore the Examiner has not considered them when evaluation of the claims as tied to a practical application. The claims only recite “computer” and “neural network based language model” both of which are used as a tool on which the abstract operates on and are general purpose elements.
Applicant asserts on page 14:
Applicant respectfully submits that amended claim 1 recites a set of steps that improves a technical field related to computer-implemented mapping and generation of structured environment representations. Conventional technologies for generating high-definition maps or environment models typically rely on multi-stage pipelines involving manual labeling, rule-based processing, and fragmented workflows that are time-intensive, compute-heavy, and difficult to scale or improve. See, e.g., Spec. [0002]. Such approaches are limited in their ability to generalize across environments, to infer missing information, and to effectively integrate semantic, topological, and geometric relationships from available data.
Embodiments of the present technology overcome these drawbacks by implementing a computational pipeline that processes map data to generate structured, tokenized representations of an environment using one or more neural-network-based language models. As described in the specification, map data is analyzed to extract features, which are encoded as feature vectors, embeddings, or latent-space representations and provided as input to a language model to generate a first tokenized description of the environment. See, e.g., Spec. [0060]-[0062]. A second language model then processes the first tokenized description to generate a higher-resolution representation that includes additional inferred detail, such as lane structures, intersection topology, and other environment features, including information relating to gaps or omissions in the original map data. See, e.g., Spec. [0039], [0064], and [0090]. This approach replaces rigid, rule-based map generation techniques with a data-driven framework that captures semantic and relational information in structured token sequences and uses learned relationships to infer missing or implicit environment features.
As set forth in the specification, conventional approaches suffer from inefficiency, lack of scalability, and inability to automatically infer and integrate multi-dimensional relationships within map data. See Spec. [0002]; see also Spec. [0050]-[0053]. The claims provide a solution by reciting a specific sequence of operations including generating a first tokenized description of an environment from map data, and generating a second tokenized description that incorporates additional inferred detail based on semantic, topological, geometric, kinematic, and relational information. See, e.g., Spec. [0038]-[0039], [0064], and [0090].
Further, the specification explains that these tokenized representations can be used to generate higher-quality maps, simulation environments, and machine-consumable representations that are utilized by autonomous systems and related applications. See Spec. [0040], [0043], and [0067]. By enabling automated generation of structured, machine-readable environment representations and filling gaps or omissions in underlying data, the claimed invention improves the accuracy, completeness, and efficiency of computer-based mapping and environment modeling systems.
Accordingly, amended claim 1 reflects a specific improvement in how computer systems generate and process environment representations using neural-network-based language models, including automated inference of missing information and enhancement of map data. For at least this reason, among others, Applicant respectfully requests withdrawal of the rejection of amended claim 1 under 35 U.S.C. § 101.
The Examiner respectfully disagrees. The aspects the applicant argues in the first two paragraphs of this section that the claims relate to “replaces rigid, rule-based map generation techniques with a data-driven framework that captures semantic and relational information in structured token sequences and uses learned relationships to infer missing or implicit environment features”. Although this may be true, such aspects can be also obtained based on manual/mental processes which uses the same data to recreate detailed versions thereof. Therefore, the asserted improvements are with respect to the abstract idea themselves rather than a showing that they improve the technology based on the additional elements of the claims. Further, the examiner has already noted above that “generate a higher-resolution representation that includes additional inferred detail, such as lane structures, intersection topology, and other environment features, including information relating to gaps or omissions in the original map data” is not recited in the claims although the broader aspects of gaps/omissions are. For example, this enhancement of map data that is specifically noted through the arguments and the ability improve accuracy of the computer thereof is not currently recited in the claims. Moreover, a human is also equally capable of filling in details of a map where such details are not present based on human knowledge.
Applicant argues on page 16-17:
In Enfish, LLC v. Microsoft Corp., 822 F.3d 1327 (Fed. Cir. 2016), the Federal Circuit held that a self-referential table recited in the claims is a "specific type of data structure designed to improve the way a computer stores and retrieves data in memory." Enfish, 822 F.3d at 1339. As such, the Federal Circuit determined that the claims are directed toward a specific implementation of a solution to a problem in software and therefore are not directed to an abstract idea.
Similarly, as explained in the present specification, the disclosed approach is directed to generating and utilizing structured, tokenized representations of an environment. In particular, the specification describes processing map data to extract semantic, topological, geometric, kinematic, and relational information, encoding such information as feature vectors or embeddings, and using a language model to generate a tokenized text string representation of the environment. See, e.g., Spec. [0060]-[0062]. The resulting tokenized description functions as a structured representation, such as an object graph, that captures relationships among environment elements. See Spec. [0040]-[0041], [0071].
As further disclosed in the Specification, at least one embodiment further processes this structured representation using a second language model to generate a higher-resolution representation that includes additional inferred detail, such as lane structures, intersection topology, and other environment features, including information relating to gaps or omissions in the underlying map data. See Spec. [0039], [0064], [0090]. The specification explains that this approach enables automated, data-driven enhancement of environment representations by inferring missing or implicit information based on learned relationships, rather than relying on rigid rules or manual processing. See Spec. [0041], [0051], [0064].
Applicant submits that such architecture enables transformation of input map data into structured, machine-readable representations and subsequent generation of enhanced environment models that are used by downstream computer systems, such as autonomous navigation systems and simulation platforms. See Spec. [0039]-[0040], [0043], and [0067]. As such, the claimed invention improves how computer systems represent, process, and utilize map and environment data by providing a unified, structured, and semantically rich representation that supports more accurate and scalable environment modeling.
Accordingly, Applicant respectfully submits that amended claim 1 is directed to a specific implementation of a solution to a problem in computer technology-namely, how to generate and enhance structured environment representations from incomplete or lower-resolution map data using learned relationships encoded in neural-network-based language models. The claim recites a concrete sequence of operations including generating a first tokenized description of an environment from map data and generating a second tokenized description that incorporates additional inferred detail to address gaps or omissions in the data. This constitutes a specific technological improvement in computer-based mapping and environment modeling systems, rather than an abstract idea.
The Examiner respectfully disagrees. All of the assertions provided in this section repeat from the prior sections which have already been addressed and therefore the Applicant is referred to the above sections.
Applicant asserts the claims 10 and 16 are allowable for the same reasons as in claim 1. The Examiner maintains the same arguments as presented with respect to claim 1 above.
Applicant asserts that claims 2-9, 11-15 and 17-20 incorporated the features from the independent and are therefore patent eligible under 35 USC 101 abstract. The Examiner maintains the same arguments as presented with respect to claim 1 above.
With respect to the 35 USC 103 rejections, the Applicant’s arguments are moot in view of new grounds for rejections.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1 and 3-21 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The independent claims 1, 10, and 16 relate to a method (claim 1), system (claim 10) and system (claim 16) where claims 2 and 10 relate to a statutory category. The claims further recite per claim 1 “generating, using a neural-network-based language model trained using map data, a first tokenized description of at least a portion of an environment; and generating, using the neural-network-based language model and the first tokenized description, a second tokenized description for at least the portion of the environment, the second tokenized description including additional detail relating to at least one gap or omission in the map data, with respect to the environment, inferred in part using at least one of semantic, topological, geometric, kinematic, or relational information represented in the first tokenized description”. Claim 16 recites similar limitations as in claims 1 and 10 but also includes the terms “LLM”, which has been itn
The limitation of claims 1 and 10 of “generating…” and “generating”..., as drafted covers mental activities. More specifically, a human generating based on memory a textual description of the environment based on map information and then using this to further include additional detail as the person remembers it by annotating the original map and including details that are known such as number of lanes on a road, whether a specific restaurant has now opened etc. Claim 16 recites similar features and encompass the same example as noted.
This judicial exception is not integrated into a practical application. In particular, claim 1 and 10 recites the additional limitations of “neural network based language model” and claim 16 recites “neural network based LLM”. Claim 10 and 16 recites a processor. Paragraph [0037] and [0040] describe the language model not be a LLM or any other language model which thus relates to a general purpose model. Paragraph [0145], of the specification as filed appears to provide a generic mention of “processor. Both the processor and the LLM are being used as a took on which the judicial exception operates thereon. Therefore, the Examiner construes these additional elements are general purpose elements. Thus, these additional elements do not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception since the additional elements as recited above are written at a high level of generality which are well known, understood and conventional. The claims are not patent eligible.
With respect to claim 3, the claims relate to “wherein the additional detail relates to one or more lanes or complex topographies inferred from the first tokenized description.” This relates to a human adding the additional details related to number of lanes in a highway. No additional limitations are present. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
With respect to claim 4, the claim relates to “wherein the additional detail relates to one or more objects identified for inclusion in the environment based at least on the first tokenized description and one or more real-world relationships learned by the neural-network-based language model.” This relates to a human using past knowledge and the first description to determine additional details of an object in an environment such as a restaurant/charging location that is now present in an area. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
With respect to claim 5, the claim relates to “wherein the map data corresponds to a standard definition (SD) map representation of at least the portion of the environment, and wherein the second tokenized description contains information corresponding to a high definition (HD) map version of at least the portion of the environment.” The claim reads on a human having a standard definition map and high definition map and creating the annotations using the high definition map information to supplement the standard definition based on print outs of these maps. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
With respect to claim 6, the claim relates to “extracting a set of features from the map data; and providing the set of features as input to the neural-network-based language model to generate the first tokenized description..” This relates to a human extracting features from the map and then using these features to generate textual information regarding them. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
With respect to claim 7, the claim relates to “ providing, as additional input to the neural-network-based language model, at least one of geographic location information or contextual information to be used to determine the additional detail.” This relates to a human using additional input of contextual information of geographic location to determine additional details to include. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
With respect to claim 8 and 14, the claim relates to “ wherein the first tokenized description and the second tokenized description are each a tokenized text string representative of at least the portion of the environment, the tokenized text string including a sequence of tokens associated with objects in the environment.” This relates to a human generating a textual description about the environment and the second strong generated with respect to objects. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
With respect to claim 9, the claim relates to “ wherein the tokenized text string is written in a road topology language (RTL) or a domain specific language (DSL).” This relates to a human generating a textual description in a specific format such as DSL. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
With respect to claim 11, the claim relates to “ wherein the first neural- network-based language model and the second neural-network-based language model correspond to different portions or different instances of a single language model.” This relates to a human using different instances of memory. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The Examiner has already noted in the independent claims of the NN based language model being any LLM.
With respect to claim 12, the claim relates to “ wherein the additional detail relates to one or more objects identified for inclusion in the environment based at least on the first tokenized description and one or more real-world relationships learned by the second neural-network-based language model.” This relates to a human including specific objects based on information understood and remembered based on locations such as whether a blue post office box is located on a street. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
With respect to claim 13, the claim relates to “wherein the one or more circuits are further to: extract a set of features from the map data; and provide the set of features as input to the first neural-network-based language model to generate the first tokenized description.” This relates to a human extracts specifics from a map and determines a textual description thereof. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
With respect to claim 15 and 20, the claim relates to “wherein the processor is comprised in at least one of: a system for performing simulation operations; a system for performing simulation operations to test or validate autonomous machine applications; a system for performing digital twin operations; a system for performing light transport simulation; a system for rendering graphical output; a system for performing deep learning operations; a system for performing generative AI operations using a large language model (LLM);a system implemented using an edge device; a system for generating or presenting virtual reality (VR) content; a system for generating or presenting augmented reality (AR) content; a system for generating or presenting mixed reality (MR) content; a system incorporating one or more Virtual Machines (VMs);a system implemented at least partially in a data center; a system for performing hardware testing using simulation; a system for performing generative operations using a language model (LM);a system for synthetic data generation; a collaborative content creation platform for 3D assets; or a system implemented at least partially using cloud computing resources.” This relates to a human for generating of generative operations using a LM. The claims include The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Note: Only one of these are required by the claims and therefore only one has been interpreted.
With respect to claim 17, the claim relates to “wherein a difference between the first level of detail and the second level of detail corresponds to at least one of a lane or complex topography inferred from the map at the second level of detail, one or more gaps or omissions inferred from the map at the second level of detail, or at least one additional object identified for inclusion in the map at the second level of detail and one or more real-world relationships learned by the LLM.” This relates to a human of using a set of features that is used to generate a second level of detail and providing that to a LLM. The claim does not include additional elements except for a LLM addressed earlier that are sufficient to amount to significantly more than the judicial exception.
With respect to claim 18, the claim relates to “extract a set of features from the map at the second level of detail; and provide the set of features as input to the LLM to generate the output” This relates to a human of using a set of features that is used to generate a second level of detail and providing that to a LLM, where the second level of detail is an object the human determines should be placed on a map due to the inclusion of a new street lamp. The claim does not include additional elements except for a LLM addressed earlier that are sufficient to amount to significantly more than the judicial exception.
With respect to claim 19, the claim relates to “wherein the representation of the map at the first level of detail and the representation of the map at the second level of detail both correspond to tokenized descriptions of at least a portion of an environment” This relates to a human providing textual descriptions but the second level of detail being more specific. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
With respect to claim 21, the claim relates to “wherein generating the first tokenized description comprises generating a sequence of tokens using an auto-regressive language model.” This relates to determining which model to use such as a LLM. The claim does not include additional elements except for a LLM addressed earlier that are sufficient to amount to significantly more than the judicial exception.
These claims further do not remedy the judicial exception being integrated into a practical application and further fail to include additional elements that are sufficient to amount to significantly more than the judicial exception.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1, 3-4, 6-8, 10-20 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Joshi (WO 2024/182721 A1, provisional support verified in 63/449262).
As to claim 1 and 10, Joshi teaches a computer-implemented method, comprising:
generating, using a neural-network-based language model trained using map data for at least a portion of an environment, a first tokenized description of at least a portion of an environment (see Figure 5, step 508, where vision language model receives vision data capturing environment of the robot (step 506) and generates a task conditioned description of the environment and see para [0075] where map data is used to determine object descriptors relevant to a task); and
generating, using the neural-network-based language model and the first tokenized description, a second tokenized description for at least the portion of the environment, the second tokenized description including additional detail relating to at least one gap or omission in the map data, with respect to the environment, inferred in part using at least one of semantic, topological, geometric, kinematic, or relational information represented in the first tokenized description (see Figure 5, step 510, where based on the task conditioned description to generate one or more subtasks performing the task which is based on semantic and relational information that is known, and where [0104], where sub-tasks provide a more detailed output as compared to the first tokenized description as noted in [0103]) (The examiner interprets the vision language model and LLM as the language model as a whole).
As to claim 10, apparatus claim 10 and method claim 1 are related as apparatus and the method of using same, with each claimed element's function corresponding to the claimed method step. Accordingly claim 10 is similarly rejected under the same rationale as applied above with respect to method claim. Furthermore, Joshi teaches a processor and circuit (see Figure 7, processor 714)
As to claim 3, Joshi teaches wherein the additional detail relates to one or more lanes or complex topographies inferred from the first tokenized description (see [0076]-[0079], where birds eye provides information of the features in the environment which are captured by the vision data instance and used by the VLM)(e.g. Due to the alternative language the Examiner is interpreting the complex topography and is interpreted as features present in an environment).
As to claim 4 and 12, Joshi teaches wherein the additional detail relates to one or more objects identified for inclusion in the environment based at least on the first tokenized description and one or more real-world relationships learned by the neural-network-based language model (see [0079], where for example the refined LLM output indicates one or more refined subtask such as “navigate to the table”, “pick up the pear”, where such provides more details of the actual object such as table and pear based on VLM and LLM).
As to claim 6 and 13, Joshi teaches extracting a set of features from the map data; and providing the set of features as input to the neural-network-based language model to generate the first tokenized description (see Figure 5, step 508, where vision language model receives vision data capturing environment of the robot (step 506) and generates a task conditioned description of the environment and see para [0075] where map data is used to determine object descriptors relevant to a task).
As to claim 7, Joshi teaches providing, as additional input to the neural-network-based language model, at least one of geographic location information or contextual information to be used to determine the additional detail (see [0076], and [0104], where contextual information such as natural language instruction and the task condition description used) .
As to claim 8 and 14, Joshi teaches wherein the first tokenized description and the second tokenized description are each a tokenized text string representative of at least the portion of the environment, the tokenized text string including a sequence of tokens associated with objects in the environment (see [0103]-[0104], where first and second tokenized string is representative of the environment as explained in the examples provided in the paragraphs and the objects the environment contains).
As to claim 11, Joshi teaches wherein the first neural- network-based language model ((see Figure 5, step 508, where vision language model receives vision data capturing environment of the robot (step 506) and generates a task conditioned description of the environment) and the second neural-network-based language model correspond to different portions (see Figure 5, step 510, where based on the task conditioned description to generate one or more subtasks performing the task which is based on semantic and relational information that is known, and where [0104], where sub-tasks provide a more detailed output as compared to the first tokenized description as noted in [0103] using a LLM) or different instances of a single language model.
As to claim 15 and 20, Joshi teaches wherein the processor is comprised in at least one of: a system for performing simulation operations;
a system for performing simulation operations to test or validate autonomous machine applications;
a system for performing digital twin operations;
a system for performing light transport simulation;
a system for rendering graphical output;
a system for performing deep learning operations (see Figure 5, step 508 and 510 make use of VLM and LLM which are deep learning operations);
a system for performing generative AI operations using a large language model (LLM) (see Figure 5, step 510, LLM);
a system implemented using an edge device;
a system for generating or presenting virtual reality (VR) content;
a system for generating or presenting augmented reality (AR) content;
a system for generating or presenting mixed reality (MR) content;
a system incorporating one or more Virtual Machines (VMs);
a system implemented at least partially in a data center;
a system for performing hardware testing using simulation;
a system for performing generative operations using a language model (LM) (see Figure 5, step 510, making use of LLM and step 508, making use of a VLM);
a system for synthetic data generation;
a collaborative content creation platform for 3D assets; or
a system implemented at least partially using cloud computing resources (see [0030], where cloud based communication framework used).
As to claim 16, Joshi teaches a system comprising:
one or more processors to generate, using a neural-network-based large language model (LLM) trained using map data (see Figure 5, step 508, where vision language model receives vision data capturing environment of the robot (step 506) and generates a task conditioned description of the environment and see para [0075] where map data is used to determine object descriptors relevant to a task), an output corresponding to a map at a first level of detail, the output generated based at least on the neural-network-based LLM processing a representation corresponding to the map at a second level of detail less than the first level of detail, the representation comprising a machine-readable sequential text string generated based at least one of semantic, topological, geometric, kinematic, or relational information extracted from the map, the output including additional detail relating to at least one gap or omission in the map data with respect to the map (see Figure 5, step 510, where based on the task conditioned description to generate one or more subtasks performing the task which is based on semantic and relational information that is known, and where [0104], where sub-tasks provide a more detailed output as compared to the first tokenized description as noted in [0103]) (The examiner interprets the vision language model and LLM as the language model as a whole).
As to claim 17, Joshi teaches wherein the one or more processors are further to: extract a set of features from the map at the second level of detail; and provide the set of features as input to the LLM to generate the output (see [0079], where for example the refined LLM output indicates one or more refined subtask such as “navigate to the table”, “pick up the pear”, where such provides more details of the actual object such as table and pear based on VLM and LLM).
As to claim 18, Joshi teaches wherein a difference between the first level of detail and the second level of detail corresponds to at least one of a lane or complex topography inferred from the map at the second level of detail (see [0076]-[0079], where birds eye provides information of the features in the environment which are captured by the vision data instance and used by the VLM)(e.g. Due to the alternative language the Examiner is interpreting the complex topography and is interpreted as features present in an environment), one or more gaps or omissions inferred from the map at the second level of detail, or at least one additional object identified for inclusion in the map at the second level of detail and one or more real-world relationships learned by the LLM (see [0079], where for example the refined LLM output indicates one or more refined subtask such as “navigate to the table”, “pick up the pear”, where such provides more details of the actual object such as table and pear based on VLM and LLM).
As to claim 19, Joshi teaches wherein the representation of the map at the first level of detail and the representation of the map at the second level of detail both correspond to tokenized descriptions of at least a portion of an environment (see [0103]-[0104], where first and second tokenized string is representative of the environment as explained in the examples provided in the paragraphs and the objects the environment contains).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 5 are rejected under 35 U.S.C. 103 as being unpatentable over Joshi in view of Mi et al. (“HDMapGen: A Hierarchical Graph Generative Model of High Definition Maps”, 2021).
As to claim 5, Joshi teaches usage of map data. However Joshi does not specifically teach wherein the map data corresponds to a standard definition (SD) map representation of at least the portion of the environment, and wherein the second tokenized description contains information corresponding to a high definition (HD) map version of at least the portion of the environment.
Mi does teach wherein the map data corresponds to a standard definition (SD) map representation of at least the portion of the environment, and wherein the second tokenized description contains information corresponding to a high definition (HD) map version of at least the portion of the environment (see Abstract, where high quality diverse maps are generated based on a hierarchical graph generation model via a coarse to fine approach, and see Figure 5, where left side figure shows a standard definition and right sight shows the traffic light text along with the lanes).
Therefore, it would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed inventions to have modified the map as taught by Joshi with the tokenized description corresponding to a HD map as taught by Mi to create a model that provides high scalability and efficiency (see Mi Abstract).
Claim(s) 9 is rejected under 35 U.S.C. 103 as being unpatentable over Joshi in view of Dodampegama et al. (“Collaborating with Ad-Hoc Agents through Reasoning and Learning”, 2022).
As to claim 9, Joshi teaches all of the limitations as in claim 8, above.
However Joshi does not specifically teach wherein the tokenized text string is written in a road topology language (RTL) or a domain specific language (DSL)
Dodampegama does teach wherein the tokenized text string is written in a road topology language (RTL) or a domain specific language (DSL) (see page 5, sect. 3.3, 1st paragraph,. where LLMs used to translate natural language to domain specific language to generate guidance that is implemented using subroutines).
Therefore, it would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed inventions to have modified the output subroutines as taught by Joshi with the DSL as taught by Dodampegama to enable determination of a high level future task (see Dodampegama page 5, sect. 3.3, 1st paragraph)
Claim(s) 21 is rejected under 35 U.S.C. 103 as being unpatentable over Joshi in view of Eichenberg (“MAGMA-Multimodal Augmentation of Generative Models through Adapter based Finetuning”, 2022).
As to claim 21, Joshi teaches all of the limitations as in claim 1, above.
Joshi teaches usage of various types of VLMs as in para [0034].
However, Joshi does not specifically wherein generating the first tokenized description comprises generating a sequence of tokens using an auto-regressive language model.
Eichenberg does teach wherein generating the first tokenized description comprises generating a sequence of tokens using an auto-regressive language model (see page 2417, left column, bullet i, MAGMA, is an autoregressive VL model that generates text from visual and textual input).
It would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed invention to have substituted the VLM as described by Joshi with the autoregressive model VLM as taught by Eichenberg in order to create an efficient and naturally retaining encyclopedic knowledge and incontext learning abilities (see Eichenberg page 2417, left column, bullet i).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PARAS D SHAH whose telephone number is (571)270-1650. The examiner can normally be reached Monday-Thursday 7:30AM-2:30PM, 5PM-7PM (EST), Friday 8AM-noon (EST).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, PARAS D SHAH can be reached at 571-270-1650. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Paras D Shah/ Supervisory Patent Examiner, Art Unit 2653
09/29/2026
