DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 03/18/2026 has been entered.
Response to Amendment
This action is in response to amendments and remarks filed on 03/18/2026. Claims 1-10, 13-16, and 18-23 are pending. Claims 11, 12, and 17 are cancelled. Claims 1, 13, 15, and 20 have been amended. The objections to the claims have been withdrawn in light of the instant amendments.
Response to Arguments
Applicant presents the following arguments regarding the previous office action:
Yakov’s images are supplied by a search engine instead of being generated by a machine-learned semantic routing model. Therefore, Yakov fails to teach claim 1.
Yakov does not disclose any type of “simulation” or “virtual space”. Therefore, Yakov fails to teach claim 1.
Regarding argument A, the argument has been fully considered but is not persuasive. Yakov teaches, “Alternatively, the image search engine 130 uses any type of generative machine-trained model to transform input information fed to it to a synthesized output image” (par. 127). Yakov uses a generative machine-trained model to synthesize images as output. Furthermore, generating an image using a generative machine-trained model is already well-known in the art, and most publicly available generative AI tools are able to do this already.
Regarding argument B, the argument has been fully considered but is not persuasive. Yakov teaches, “According to some implementations of any of the methods of A1-A14, the intent is a travel intent or an atlas intent that expresses an intent to retrieve an image. The operation of performing further processing includes retrieving an image from a data store that pertains to a topic expressed by the query. The operation of generating includes presenting an interactive map pertaining to the query, and presenting the image that has been retrieved in prescribed proximity to the interactive map” (par. 175). The generated image is presented along with an interactive map, so that the image can be used in conjunction with the map to provide information to the user (par. 90 describes various ways in which the image is presented with the map).
A “simulation of the one or more suggested route segments traversing a virtual space” is broad. A simple map such as shown in Fig. 7 could be considered a simulation traversing a virtual space. The virtual space is the map, and the dotted lines represent a simulation of the route segments. Since the route depicted on the map is generated by the model, Examiner believes it could be considered that the image of the map with the route is implicitly generated by the model, since the image would not exist without the generated route.
Even if the image of the map is not considered being generated by the model, since the generated and presented image is used along with the interactive map, it could be considered that the image is generated so as to be combined with the map. Therefore, the generated image could be considered to aid in the simulation of the route along a virtual space.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 8-10, 15-16, and 20-23 is/are rejected under 35 U.S.C. 103 as being unpatentable over Dong (US 20180300641) in view of Yankov (US 20250076059 A1).
Regarding claim 1, Dong teaches a computer-implemented method, comprising: obtaining, by a computing system comprising one or more computing devices (par. 20 Fig. 1, computer system 12), training data (par. 55 Fig. 9, history route data 910) comprising:
route information indicative of a route from a starting location to a destination location (par. 55 Fig. 9, history route data 910), wherein the route comprises a plurality of route segments comprising a first subset of route segments (par. 55 Fig. 9, training data 920) and a second subset of route segments (par. 55 Fig. 9, testing data 811); and
route characteristic information descriptive of one or more route characteristics (par. 54 Fig. 9, route profile 243);
processing, by the computing system, at least the first subset of route segments and a portion of the route characteristic information associated with the first subset of route segments (par. 55 Fig. 9, training data 920) with a machine-learned semantic routing model (par. 52 Fig. 8, hybrid model 200) to obtain one or more predicted route segments for the second subset of route segments (par. 54 Fig. 8,“The intermediate prediction results 813, the credibility parameters 814, and the trained decision rule 815 together with the route profile 243 of the particular target object may be provided to the prediction decision engine 250 to determine the prediction result 260”);
adjusting, by the computing system, one or more parameters of the machine-learned semantic routing model based on an optimization function that evaluates a difference between the one or more predicted route segments and the second subset of route segments (par. 52, “the object-specific prediction model 221 and the object group-specific prediction model 222 may be trained and optimized according to history route data, and thus improving their prediction performance”);
Dong fails to teach obtaining, by the computing system, one or more inputs comprising request information indicative of requested route segments; and generating, by the computing system, based on inputting the one or more inputs into the machine-learned semantic routing model, multimodal model output comprising image output and one or more textual descriptions of one or more suggested route segments, wherein the image output is generated by the machine-learned semantic routing model and comprises one or more simulations of the one or more suggested route segments traversing a virtual space.
However, Yankov teaches obtaining, by the computing system, one or more inputs comprising request information indicative of requested route segments (abstract, “the technique uses the machine-trained language model to assess at least one intent associated with the query”; par. 57, “The language model 104 may invoke both the routing intent and the travel intent for a query that reads, “I want to see beautiful scenery on my way from St. Regis, Montana to Kalispell, Montana, even if takes me a bit out of the way”);
and generating, by the computing system, based on inputting the one or more inputs into the machine-learned semantic routing model, multimodal model output comprising image output (par. 69, “a travel intent expresses a request for the image search engine 130 to supply one or more travel-related images that illustrate a route selected by the routing engine 138”) and one or more textual descriptions of one or more suggested route segments (Fig. 7, textual description 714 and 724 in response to a route-finding intent query), wherein the image output is generated by the machine-learned semantic routing model and comprises one or more simulations of the one or more suggested route segments traversing a virtual space (par. 127, “the image search engine 130 uses any type of generative machine-trained model to transform input information fed to it to a synthesized output image. For the case in which a travel intent is detected, the output of the image search engine 130 is least one travel-related image”; par. 69, “For example, a travel intent expresses a request for the image search engine 130 to supply one or more travel-related images that illustrate a route selected by the routing engine 138”; par. 175, “According to some implementations of any of the methods of A1-A14, the intent is a travel intent or an atlas intent that expresses an intent to retrieve an image. The operation of performing further processing includes retrieving an image from a data store that pertains to a topic expressed by the query. The operation of generating includes presenting an interactive map pertaining to the query, and presenting the image that has been retrieved in prescribed proximity to the interactive map”).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Dong to incorporate the teachings of Yankov because, as Yankov states, it “successfully interprets complex map-related queries, without demanding that a user express the query in a predetermined manner…The technique is also readily extensible because it can be applied to many different environments without significant (or any) revision to its basic processing framework” (par. 13). Yankov also states that the image generation feature can be implemented using “any type of generative machine-trained model to transform input information fed to it to a synthesized output image” (par. 127), therefore could use Dong to implement it. Additionally, adding this feature would increase the invention’s practical applications for the user by giving it another input/output option. As can be seen with other popular AI models such as ChatGPT, versatility is an obvious desirable trait.
Regarding claim 8, the combination of Dong in view of Yankov teaches the computer-implemented method of claim 1. Dong further teaches the method further comprises: processing, by the computing system, input data (par. 27 Fig. 2, first route 210) with the machine-learned semantic routing model to obtain a model output, wherein the input data comprises:
route information indicative of a one or more first route segments of an incomplete route (par. 27 Fig. 2, first route 210);
and a prompt indicative of a request to generate one or more second route segments with requested route characteristics for the incomplete route (par. 29 Fig. 2, route profile 243);
and wherein the model output comprises the one or more second route segments with the requested route characteristics (par. 29, “the route profile 243 may be prior information of a particular object regarding its trajectories. Therefore, the adaptive hybrid model 200 may also consider general characteristics of the trajectories of a particular target object when determining the prediction result 260 for the target object”).
Dong fails to explicitly teach a prompt indicative of a request to describe the second route.
However, Yankov teaches a prompt indicative of a request to describe the second route (abstract, “the technique uses the machine-trained language model to assess at least one intent associated with the query”; par. 57, “The language model 104 may invoke both the routing intent and the travel intent for a query that reads, “I want to see beautiful scenery on my way from St. Regis, Montana to Kalispell, Montana, even if takes me a bit out of the way.”)
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Dong in view of Yankov to further incorporate the teachings of Yankov. Feeding an AI model a prompt is well-known in the art and would be a trivial addition to Dong.
Regarding claim 9, the combination of Dong in view of Yankov teaches the computer-implemented method of claim 1. Dong further teaches the method further comprises: processing, by the computing system, input data (par. 27 Fig. 2, first route 210) with the machine-learned semantic routing model to obtain a model output, wherein the input data comprises:
route information indicative of one or more example routes (par. 55 Fig. 9, history route data 910) and a second route comprising a plurality of second route segments (par. 38 Fig. 4a, route 410);
and a prompt indicative of a request to generate one or more alternate route segments for one or more respective second route segments of the second route based on the one or more example routes (par. 55 Fig. 9, history route data 910);
and wherein the model output comprises the one or more alternate route segments of the second route (par. 38 Fig. 4a, route 421).
Dong fails to explicitly teach a prompt indicative of a request to describe the second route, feeding an AI model a prompt is well-known in the art and would be a trivial addition to Dong.
However, Yankov teaches a prompt indicative of a request to describe the second route (abstract, “the technique uses the machine-trained language model to assess at least one intent associated with the query”; par. 57, “The language model 104 may invoke both the routing intent and the travel intent for a query that reads, “I want to see beautiful scenery on my way from St. Regis, Montana to Kalispell, Montana, even if takes me a bit out of the way.”)
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Dong in view of Yankov to further incorporate the teachings of Yankov. Feeding an AI model a prompt is well-known in the art and would be a trivial addition to Dong.
Regarding claim 10, the combination of Dong in view of Yankov teaches the computer-implemented method of claim 1. Dong further teaches the method further comprises: processing, by the computing system, input data with the machine-learned semantic routing model to obtain a model output, wherein the input data comprises route information descriptive of a second route comprising a plurality of second route segments (par. 55 Fig. 9, history route data 910), and wherein the model output comprises classification information that classifies the route as a first route type of a plurality of route types, and wherein the classification information further classifies a second route segment of the plurality of second route segments as a first route segment type of a plurality of route segment types (par. 57, “With the route profile 243, the intermediate prediction results and credibility parameters 951, 952 and 953, as well as the actual result 954, the decision rule 251 may be trained, for example, in terms of a classification problem”—although there is no explicit mention of both route types and route segment types, it would be a trivial addition to classify both).
Regarding claim 15, Dong teaches a computing system (par. 80 Fig. 1, computer system/server 12), comprising: (par. 80 Fig. 1, processing units 16); a memory (par. 80 Fig. 1, system memory 28), comprising:
a machine-learned semantic routing model (par. 27 Fig. 2, adaptive hybrid model 200), wherein the machine-learned semantic routing model is trained to process mapping information (par. 27 Fig. 2, first route 210) to generate a model output comprising suggested route segments or information associated with route segments (par. 54 Fig. 8, “The intermediate prediction results 813, the credibility parameters 814, and the trained decision rule 815 together with the route profile 243 of the particular target object may be provided to the prediction decision engine 250 to determine the prediction result 260”);
one or more tangible, non-transitory computer readable media storing computer-readable instructions that when executed by one or more processors cause the one or more processors to perform operations (par. 83, “memory 28 may include a computer program product storing one or program modules 42 comprising computer readable instructions configured to carry out one or more features of the present invention”), the operations comprising:
obtaining, from a client computing device, one or more inputs for the machine- learned semantic routing model, wherein the one or more inputs comprises at least one of:
processing, by the machine-learned semantic routing model, the one or more inputs to obtain a multimodal model output (see Fig. 2), wherein the multimodal model output comprises:
routing information indicative of a route that comprises one or more suggested route segments (par. 28, “prediction result 260 for the second route”); and
and providing the model output to the client computing device (see Fig. 10).
Dong fails to teach request information indicative of a requested route segment or a request for mapping-related information; or route characteristic information indicative of one or more route characteristics; and semantic mapping information comprising image output and one or more textual descriptions of one or more suggested route segments, wherein the image output is generated by the machine-learned semantic routing model and comprises one or more simulations of the one or more suggested route segments traversing a virtual space.
However, Yankov teaches request information indicative of a requested route segment or a request for mapping-related information; or route characteristic information indicative of one or more route characteristics (abstract, “the technique uses the machine-trained language model to assess at least one intent associated with the query”; par. 57, “The language model 104 may invoke both the routing intent and the travel intent for a query that reads, “I want to see beautiful scenery on my way from St. Regis, Montana to Kalispell, Montana, even if takes me a bit out of the way.”);
and semantic mapping information comprising image output (par. 69, “a travel intent expresses a request for the image search engine 130 to supply one or more travel-related images that illustrate a route selected by the routing engine 138”) and one or more textual descriptions of one or more suggested route segments (Fig. 7, textual description 714 and 724 in response to a route-finding intent query), wherein the image output is generated by the machine-learned semantic routing model and comprises one or more simulations of the one or more suggested route segments traversing a virtual space (par. 127, “the image search engine 130 uses any type of generative machine-trained model to transform input information fed to it to a synthesized output image. For the case in which a travel intent is detected, the output of the image search engine 130 is least one travel-related image”; par. 69, “For example, a travel intent expresses a request for the image search engine 130 to supply one or more travel-related images that illustrate a route selected by the routing engine 138”; par. 175, “According to some implementations of any of the methods of A1-A14, the intent is a travel intent or an atlas intent that expresses an intent to retrieve an image. The operation of performing further processing includes retrieving an image from a data store that pertains to a topic expressed by the query. The operation of generating includes presenting an interactive map pertaining to the query, and presenting the image that has been retrieved in prescribed proximity to the interactive map”).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Dong to incorporate the teachings of Yankov because, as Yankov states, it “successfully interprets complex map-related queries, without demanding that a user express the query in a predetermined manner…The technique is also readily extensible because it can be applied to many different environments without significant (or any) revision to its basic processing framework” (par. 13). Yankov also states that the image generation feature can be implemented using “any type of generative machine-trained model to transform input information fed to it to a synthesized output image” (par. 127), therefore could use Dong to implement it. Additionally, adding this feature would increase the invention’s practical applications for the user by giving it another input/output option. As can be seen with other popular AI models such as ChatGPT, versatility is an obvious desirable trait.
Regarding claim 16, the combination of Dong in view of Yankov teaches the computing system of claim 15. Dong further teaches obtaining the one or more inputs for the machine-learned semantic routing model comprises:
obtaining, from the client computing device, example route information indicative of one or more example routes (par. 55 Fig. 9, history route data 910) and a second route comprising a plurality of second route segments (par. 38 Fig. 4a, route 410), and a prompt indicative of a request to generate one or more alternate route segments for one or more respective second route segments of the second route based on the one or more example routes (par. 55 Fig. 9, history route data 910);
and wherein processing the one or more inputs to obtain the model output comprises: processing the example route information and the prompt to obtain the model output, wherein the model output comprises routing information indicative of the one or more alternate route segments for the one or more respective second route segments (par. 38 Fig. 4a, route 421).
Dong fails to explicitly teach a prompt indicative of a request to describe the second route.
However, Yankov teaches a prompt indicative of a request to describe the second route (abstract, “the technique uses the machine-trained language model to assess at least one intent associated with the query”; par. 57, “The language model 104 may invoke both the routing intent and the travel intent for a query that reads, “I want to see beautiful scenery on my way from St. Regis, Montana to Kalispell, Montana, even if takes me a bit out of the way.”)
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Dong in view of Yankov to further incorporate the teachings of Yankov. Feeding an AI model a prompt is well-known in the art and would be a trivial addition to Dong.
Regarding claim 20, Dong teaches one or more tangible, non-transitory computer readable media (par. 80 Fig. 1, system memory 28) storing computer-readable instructions that when executed by one or more processors (par. 80 Fig. 1, processing units 16) cause the one or more processors to perform operations (par. 83, “memory 28 may include a computer program product storing one or program modules 42 comprising computer readable instructions configured to carry out one or more features of the present invention”), the operations comprising: obtaining training data comprising:
route information indicative of a route from a starting location to a destination location (par. 55 Fig. 9, history route data 910), wherein the route comprises a plurality of route segments comprising a first subset of route segments (par. 55 Fig. 9, training data 920) and a second subset of route segments (par. 55 Fig. 9, testing data 811); and
route characteristic information descriptive of one or more route characteristics (par. 54 Fig. 9, route profile 243);
processing at least the first subset of route segments and a portion of the route characteristic information associated with the first subset of route segments (par. 55 Fig. 9, training data 920) with a machine-learned semantic routing model (par. 52 Fig. 8, hybrid model 200) to obtain one or more predicted route segments for the second subset of route segments (par. 54 Fig. 8,“The intermediate prediction results 813, the credibility parameters 814, and the trained decision rule 815 together with the route profile 243 of the particular target object may be provided to the prediction decision engine 250 to determine the prediction result 260”);
and adjusting one or more parameters of the machine-learned semantic routing model based on an optimization function that evaluates a difference between the one or more predicted route segments and the second subset of route segments (par. 52, “the object-specific prediction model 221 and the object group-specific prediction model 222 may be trained and optimized according to history route data, and thus improving their prediction performance”);
Dong fails to teach obtaining one or more inputs comprising request information indicative of requested route segments; and generating, based on inputting the one or more inputs into the machine-learned semantic routing model, multimodal model output comprising image output and one or more textual descriptions of one or more suggested route segments, wherein the image output is generated by the machine-learned semantic routing model and comprises one or more simulations of the one or more suggested route segments traversing a virtual space.
However, Yankov teaches obtaining one or more inputs comprising request information indicative of requested route segments (abstract, “the technique uses the machine-trained language model to assess at least one intent associated with the query”; par. 57, “The language model 104 may invoke both the routing intent and the travel intent for a query that reads, “I want to see beautiful scenery on my way from St. Regis, Montana to Kalispell, Montana, even if takes me a bit out of the way.”);
and generating, based on inputting the one or more inputs into the machine-learned semantic routing model, multimodal model output comprising image output (par. 69, “a travel intent expresses a request for the image search engine 130 to supply one or more travel-related images that illustrate a route selected by the routing engine 138”) and one or more textual descriptions of one or more suggested route segments (Fig. 7, textual description 714 and 724 in response to a route-finding intent query), wherein the image output is generated by the machine-learned semantic routing model and comprises one or more simulations of the one or more suggested route segments traversing a virtual space (par. 127, “the image search engine 130 uses any type of generative machine-trained model to transform input information fed to it to a synthesized output image. For the case in which a travel intent is detected, the output of the image search engine 130 is least one travel-related image”; par. 69, “For example, a travel intent expresses a request for the image search engine 130 to supply one or more travel-related images that illustrate a route selected by the routing engine 138”; par. 175, “According to some implementations of any of the methods of A1-A14, the intent is a travel intent or an atlas intent that expresses an intent to retrieve an image. The operation of performing further processing includes retrieving an image from a data store that pertains to a topic expressed by the query. The operation of generating includes presenting an interactive map pertaining to the query, and presenting the image that has been retrieved in prescribed proximity to the interactive map”).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Dong to incorporate the teachings of Yankov because, as Yankov states, it “successfully interprets complex map-related queries, without demanding that a user express the query in a predetermined manner…The technique is also readily extensible because it can be applied to many different environments without significant (or any) revision to its basic processing framework” (par. 13). Yankov also states that the image generation feature can be implemented using “any type of generative machine-trained model to transform input information fed to it to a synthesized output image” (par. 127), therefore could use Dong to implement it. Additionally, adding this feature would increase the invention’s practical applications for the user by giving it another input/output option. As can be seen with other popular AI models such as ChatGPT, versatility is an obvious desirable trait.
Regarding claim 21, the combination of Dong in view of Yankov teaches the one or more tangible, non-transitory computer readable media of claim 20. Dong fails to teach the one or more suggested route segments comprise a simulated route traveled by a simulated user.
However, Yankov teaches the one or more suggested route segments comprise a simulated route traveled by a simulated user (par. 69, “a travel intent expresses a request for the image search engine 130 to supply one or more travel-related images that illustrate a route selected by the routing engine 138”; Fig. 7, textual description 714 and 724 in response to a route-finding intent query, maps 710 and 720 indicative of the requested routes in response to a route-finding intent query—output ‘simulates’ the route the user would take).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Dong in view of Yankov to further incorporate the teachings of Yankov because, as Yankov states, it “successfully interprets complex map-related queries, without demanding that a user express the query in a predetermined manner…The technique is also readily extensible because it can be applied to many different environments without significant (or any) revision to its basic processing framework” (par. 13). Yankov also states that the image generation feature can be implemented using “any type of generative machine-trained model to transform input information fed to it to a synthesized output image” (par. 127), therefore could use Dong to implement it. Additionally, adding this feature would increase the invention’s practical applications for the user by giving it another input/output option. As can be seen with other popular AI models such as ChatGPT, versatility is an obvious desirable trait.
Regarding claim 22, the combination of Dong in view of Yankov teaches the computing system of claim 15. Dong fails to teach the one or more suggested route segments comprise a simulated route traveled by a simulated user.
However, Yankov teaches the one or more suggested route segments comprise a simulated route traveled by a simulated user (par. 69, “a travel intent expresses a request for the image search engine 130 to supply one or more travel-related images that illustrate a route selected by the routing engine 138”; Fig. 7, textual description 714 and 724 in response to a route-finding intent query, maps 710 and 720 indicative of the requested routes in response to a route-finding intent query—output ‘simulates’ the route the user would take).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Dong in view of Yankov to further incorporate the teachings of Yankov because, as Yankov states, it “successfully interprets complex map-related queries, without demanding that a user express the query in a predetermined manner…The technique is also readily extensible because it can be applied to many different environments without significant (or any) revision to its basic processing framework” (par. 13). Yankov also states that the image generation feature can be implemented using “any type of generative machine-trained model to transform input information fed to it to a synthesized output image” (par. 127), therefore could use Dong to implement it. Additionally, adding this feature would increase the invention’s practical applications for the user by giving it another input/output option. As can be seen with other popular AI models such as ChatGPT, versatility is an obvious desirable trait.
Regarding claim 23, Dong teaches computer-implemented method of claim 1. Dong fails to teach the one or more suggested route segments comprise a simulated route traveled by a simulated user.
However, Yankov teaches the one or more suggested route segments comprise a simulated route traveled by a simulated user (par. 69, “a travel intent expresses a request for the image search engine 130 to supply one or more travel-related images that illustrate a route selected by the routing engine 138”; Fig. 7, textual description 714 and 724 in response to a route-finding intent query, maps 710 and 720 indicative of the requested routes in response to a route-finding intent query—output ‘simulates’ the route the user would take).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Dong in view of Yankov to further incorporate the teachings of Yankov because, as Yankov states, it “successfully interprets complex map-related queries, without demanding that a user express the query in a predetermined manner…The technique is also readily extensible because it can be applied to many different environments without significant (or any) revision to its basic processing framework” (par. 13). Yankov also states that the image generation feature can be implemented using “any type of generative machine-trained model to transform input information fed to it to a synthesized output image” (par. 127), therefore could use Dong to implement it. Additionally, adding this feature would increase the invention’s practical applications for the user by giving it another input/output option. As can be seen with other popular AI models such as ChatGPT, versatility is an obvious desirable trait.
Claim(s) 2-5 and 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Dong in view of Yankov, and further in view of Laprise (US 20240419907).
Regarding claim 2, the combination of Dong in view of Yankov teaches the computer-implemented method of claim 1. Dong fails to teach processing the at least the first subset of route segments and the portion of the route characteristic information associated with the first subset of route segments with the machine-learned semantic routing model comprises: processing, by the computing system, the at least the first subset of route segments and the portion of the route characteristic information associated with the first subset of route segments with a first portion of the machine-learned semantic routing model to obtain a latent representation of the first subset of route segments
However, Laprise teaches processing the at least the first subset of route segments and the portion of the route characteristic information associated with the first subset of route segments with the machine-learned semantic routing model comprises: processing, by the computing system, the at least the first subset of route segments and the portion of the route characteristic information associated with the first subset of route segments with a first portion of the machine-learned semantic routing model to obtain a latent representation of the first subset of route segments (par. 50 “For example, a model (as part of the LLM or a separate model) can analyze the sensor data 104 for the environment and encode features of the sensor data into a latent space (or other embedding). The LLM can then take a feature vector as input that is a function of these individual latent space encodings, and can directly generate the tokenized text string representation of the environment. The features extracted can include semantic, relationship, and geometry features, among other such options”).
Dong and Laprise are analogous art because both are related to utilizing artificial intelligence to make sense of map data. Dong deals with generating routes and training a model on route segments. Laprise deals with generating maps and training a model on sensor data. An accurate understanding of maps and roads is crucial to both references.
It would have been prima facie obvious to one of ordinary skill in the art before
the effective filing date of the claimed invention to have modified the combination of Dong in view of Yankov to incorporate the teachings of Laprise to add the at least the first subset of route segments and the portion of the route characteristic information associated with the first subset of route segments with a first portion of the machine-learned semantic routing model to obtain a latent representation of the first subset of route segments. Laprise states that generating a latent representation can “prevent this information from being discarded early in the generation process, and allow for more accurate representations or reconstructions to be generated” (par. 50), and “These semantic-aware embeddings can also support arithmetic operations, for example, which allows for the identification of patterns, relationships, and other aspects of the underlying map data” (par. 92). One would be able to come to the conclusion that generating a latent representation can aid in getting a deeper understanding of the data, which would make training more efficient. Laprise also states that “Such an approach can be used for a wide variety of geospatial information processing and autonomous driving tasks (such as map building, map editing, map-based navigation, planning and driving) by representing those tasks as document manipulation tasks” (par. 39).
Regarding claim 3, the combination of Dong in view of Yankov and Laprise teaches the computer-implemented method of claim 2. Dong fails to teach the first portion of the machine-learned semantic routing model comprises an encoder or decoder portion of a pre-trained Large Language Model (LLM).
However, Laprise teaches the first portion of the machine-learned semantic routing model comprises an encoder or decoder portion of a pre-trained Large Language Model (LLM) (par. 50 “For example, a model (as part of the LLM or a separate model) can analyze the sensor data 104 for the environment and encode features of the sensor data into a latent space (or other embedding). The LLM can then take a feature vector as input that is a function of these individual latent space encodings, and can directly generate the tokenized text string representation of the environment. The features extracted can include semantic, relationship, and geometry features, among other such options”).
Dong and Laprise are analogous art because both are related to utilizing artificial intelligence to make sense of map data. Dong deals with generating routes and training a model on route segments. Laprise deals with generating maps and training a model on sensor data. An accurate understanding of maps and roads is crucial to both references.
It would have been prima facie obvious to one of ordinary skill in the art before
the effective filing date of the claimed invention to have modified the combination of Dong in view of Laprise to further incorporate the teachings of Laprise to add the first portion of the machine-learned semantic routing model comprises an encoder and/or decoder portion of a pre-trained Large Language Model (LLM). Laprise states that generating a latent representation can “prevent this information from being discarded early in the generation process, and allow for more accurate representations or reconstructions to be generated” (par. 50), and “These semantic-aware embeddings can also support arithmetic operations, for example, which allows for the identification of patterns, relationships, and other aspects of the underlying map data” (par. 92). One would be able to come to the conclusion that generating a latent representation can aid in getting a deeper understanding of the data, which would make training more efficient. Laprise also states that “Such an approach can be used for a wide variety of geospatial information processing and autonomous driving tasks (such as map building, map editing, map-based navigation, planning and driving) by representing those tasks as document manipulation tasks” (par. 39).
Regarding claim 4, the combination of Dong in view of Yankov and Laprise teaches the computer-implemented method of claim 3. Dong fails to teach the method further comprises: processing, by the computing system, input data with the machine-learned semantic routing model to obtain a model output, wherein the input data comprises: route information indicative of a second route from a second starting location to a second destination location; and a prompt indicative of a request to describe the second route; and wherein the model output comprises textual content descriptive of the second route.
However, Laprise teaches the method further comprises: processing, by the computing system, input data with the machine-learned semantic routing model to obtain a model output, wherein the input data comprises:
route information indicative of a second route from a second starting location to a second destination location (par. 135 Fig. 5G, “In this example process 580, input data corresponding to a set of observations for an environment can be provided 582 as input to a language model, such as a trained LLM. The set of observations can correspond to raw sensor data, features extracted from the raw sensor data, a set of feature vectors or embeddings, or an object map, among other such options”);
and wherein the model output comprises textual content descriptive of the second route (par. 135, Fig. 5G, “A text string can be generated and received 584 as output from the language model. The text string in this example can be a single, tokenized text string that comprises a tokenized description of the environment determine in part upon aspects and relationships determined or inferred for the set of observations, including semantic, geometric, and topological aspects of those observations”).
Although Laprise does not explicitly teach a prompt indicative of a request to describe the second route, feeding an AI model a prompt is well-known in the art and would be a trivial addition to Laprise.
Dong and Laprise are analogous art because both are related to utilizing artificial intelligence to make sense of map data. Dong deals with generating routes and training a model on route segments. Laprise deals with generating maps and training a model on sensor data. An accurate understanding of maps and roads is crucial to both references.
It would have been prima facie obvious to one of ordinary skill in the art before
the effective filing date of the claimed invention to have modified the combination of Dong in view of Yankov and Laprise to further incorporate the teachings of Laprise to add the input data comprises: route information indicative of a second route from a second starting location to a second destination location; and a prompt indicative of a request to describe the second route; and wherein the model output comprises textual content descriptive of the second route. Laprise states that the textual description would contain semantic topology, and geometry data (par. 37). Additionally, it would increase the invention’s practical applications for the user by giving it another input/output option. As can be seen with other popular AI models such as ChatGPT, versatility is an obvious desirable trait.
Regarding claim 5, the combination of Dong in view of Yankov and Laprise teaches the computer-implemented method of claim 3. Dong fails to teach the method further comprises: processing, by the computing system, input data with the machine-learned semantic routing model to obtain a model output, wherein the input data comprises textual content descriptive of a requested route, and wherein the model output comprises route information indicative of the requested route.
However, Yankov teaches the method further comprises: processing, by the computing system, input data with the machine-learned semantic routing model to obtain a model output, wherein the input data comprises textual content descriptive of a requested route (Fig. 7, textual description 714 and 724 in response to a route-finding intent query), and wherein the model output comprises route information indicative of the requested route (Fig. 7, maps 710 and 720 indicative of the requested routes in response to a route-finding intent query).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Dong in view of Yankov and Laprise to further incorporate the teachings of Yankov because, as Yankov states, it “successfully interprets complex map-related queries, without demanding that a user express the query in a predetermined manner…The technique is also readily extensible because it can be applied to many different environments without significant (or any) revision to its basic processing framework” (par. 13). Yankov also states that the image generation feature can be implemented using “any type of generative machine-trained model to transform input information fed to it to a synthesized output image” (par. 127), therefore could use Dong to implement it. Additionally, adding this feature would increase the invention’s practical applications for the user by giving it another input/output option. As can be seen with other popular AI models such as ChatGPT, versatility is an obvious desirable trait.
Regarding claim 7, the combination of Dong in view of Yankov and Laprise teaches the computer-implemented method of claim 2. Dong fails to teach processing the at least the first subset of route segments and the portion of the route characteristic information associated with the first subset of route segments with the first portion of the machine-learned semantic routing model further comprises: processing, by the computing system, the latent representation with a graph-based portion of the machine-learned semantic routing model to obtain a graph output, wherein the graph output comprises: a plurality of nodes representative of the starting location, the destination location, and intermediate locations between the starting location and the destination location; and a plurality of edges representative of a plurality of route segments.
However, Laprise teaches processing the at least the first subset of route segments and the portion of the route characteristic information associated with the first subset of route segments with the first portion of the machine-learned semantic routing model further comprises:
processing, by the computing system, the latent representation with a graph-based portion of the machine-learned semantic routing model to obtain a graph output (par. 117, “an important entity in a language-based representation—such as an RTL document—may be implemented as a directed graph describing a portion of an environment such as a road network. Such a graph can be used to express the connectivity between road features (topology) and may be similar to a knowledge graph”; see Fig. 5D), wherein the graph output comprises:
a plurality of nodes representative of the starting location, the destination location, and intermediate locations between the starting location and the destination location (par. 117, “The graph nodes can then correspond to landmark features”);
and a plurality of edges representative of a plurality of route segments (par. 117, “The edges of the graph can correspond to the relationships between road features”; par. 118, “an edge sequence can be used to express the path on the underlying graph of the map”).
Dong and Laprise are analogous art because both are related to utilizing artificial intelligence to make sense of map data. Dong deals with generating routes and training a model on route segments. Laprise deals with generating maps and training a model on sensor data. An accurate understanding of maps and roads is crucial to both references.
It would have been prima facie obvious to one of ordinary skill in the art before
the effective filing date of the claimed invention to have modified the combination of Dong in view of Yankov and Laprise to further incorporate the teachings of Laprise to add processing, by the computing system, the latent representation with a graph-based portion of the machine-learned semantic routing model to obtain a graph output. Laprise states that “Using a form of edge sequences can allow for a more compact representation of the RTL documents. Moreover, integer IDs used to refer to the features can be eliminated completely. Since there can be a linear path in the structure, the nodes and properties around that path can be expressed in an appropriate fashion” (par. 120). One could easily come to the conclusion that processing the latent representation into a graph output would make it easier to train the model, as the graph output would be more efficient and compact.
Claim(s) 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Dong, Yankov, and Laprise, and further in view of Rattan (Rattan, Puru et al. “Artificial Intelligence and Machine Learning: What You Always Wanted to Know but Were Afraid to Ask.” Gastro hep advances vol. 1,1 70-78. 3 Feb. 2022).
The combination of Dong in view of Yankov and Laprise teach the computer-implemented method of claim 5. Dong and Yankov fail to teach prior to processing the input data with the machine-learned semantic routing model to obtain the model output, the method comprises: processing, by the computing system, second training data comprising a training route with the machine-learned semantic routing model to obtain a textual description of the training route; and adjusting, by the computing system, one or more parameters of the machine-learned semantic routing model based on a loss function that evaluates the textual description of the training route and a corresponding ground-truth textual description of the training route.
However, Laprise teaches prior to processing the input data with the machine-learned semantic routing model to obtain the model output, the method comprises:
processing, by the computing system, second training data comprising a training route with the machine-learned semantic routing model to obtain a textual description of the training route (par. 135 Fig. 5G, “A text string can be generated and received 584 as output from the language model. The text string in this example can be a single, tokenized text string that comprises a tokenized description of the environment determine in part upon aspects and relationships determined or inferred for the set of observations, including semantic, geometric, and topological aspects of those observations”);
It would have been prima facie obvious to one of ordinary skill in the art before
the effective filing date of the claimed invention to have modified the combination of Dong in view of Yankov and Laprise to further incorporate the teachings of Laprise to add the input data comprises: route information indicative of a second route from a second starting location to a second destination location; and a prompt indicative of a request to describe the second route; and wherein the model output comprises textual content descriptive of the second route. Laprise states that the textual description would contain semantic topology, and geometry data (par. 37). One would easily be able to see that this textual description would be useful, so it would be an obvious conclusion to try and train it to get a more accurate output.
Laprise fails to teach adjusting, by the computing system, one or more parameters of the machine-learned semantic routing model based on a loss function that evaluates the textual description of the training route and a corresponding ground-truth textual description of the training route.
However, training a model based on a loss function using ground-truth data is well-known and routine in the field. Rattan teaches adjusting, by the computing system, one or more parameters of the machine-learned semantic routing model based on a loss function (pg. 73 column 2, “During training, an ML model learns by varying its parameters, adjustable values specific to each feature and analogous to coefficients in regression models, to minimize its loss function.”) that evaluates the textual description of the training route and a corresponding ground-truth textual description of the training route (pg. 71 column 2, “The “learning” aspect of ML corresponds to the initial training phase of building a model, where an ML model is trained on, or “learns” from, a representative data set. This can occur in two main frameworks: supervised learning or unsupervised learning. Data that have been labeled with an output variable using a predetermined gold or reference standard, known as the ground truth, by subject matter experts or through actual measurement can be used to train a supervised model”).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Dong in view of Yankov and Laprise to incorporate the teachings of Rattan to add adjusting, by the computing system, one or more parameters of the machine-learned semantic routing model based on a loss function that evaluates the textual description of the training route and a corresponding ground-truth textual description of the training route. Laprise states that the textual description would contain semantic topology, and geometry data (par. 37). One would easily be able to see that this textual description would be useful, so it would be an obvious conclusion to try and train it to get a more accurate output.
Claim(s) 13 and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Dong in view of Yankov, and further in view of Abhishek (US 20220373346).
Regarding claim 13, the combination of Dong in view of Yankov teaches the computer-implemented method of claim 1. Dong fails to teach the method further comprises: processing, by the computing system, input data with the machine-learned semantic routing model to obtain a model output, wherein the input data comprises: route request information descriptive of a route from a requested starting location to a requested destination location; and preferred route characteristic information descriptive of a preferred route characteristic for the route; and wherein the model output comprises route information indicative of a route from the requested starting location to the requested destination location that comprises the preferred route characteristic.
However, Abhishek teaches the method further comprises: processing, by the computing system, input data with the machine-learned semantic routing model to obtain a model output, wherein the input data comprises:
route request information descriptive of a route from a requested starting location to a requested destination location (par. 38, “the processor 102 can execute instructions 132 to receive a user request 182 for a personalized route”);
and preferred route characteristic information descriptive of a preferred route characteristic for the route (par. 38, “The user request 182 can include certain request attributes 184…The request attributes may optionally include the mode of transport that the user proposes to employ, and certain route constraints that may be placed by the user i.e., whether the user would prefer the shortest/fastest route or a scenic route, or preferred stops along the route, or specific locations to avoid, etc.”);
and wherein the model output comprises route information indicative of a route from the requested starting location to the requested destination location that comprises the preferred route characteristic (par. 53, “The routing engine 204 provides the optimal routes given a set of constraints provided by the recommender systems 206”).
Dong and Abhishek are analogous art because both are related to utilizing artificial intelligence to generate a route for a user (Abhishek specifies using AI at par. 73).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Dong in view of Yankov to incorporate the teachings of Abhishek to add the input data comprises: route request information descriptive of a route from a requested starting location to a requested destination location; and preferred route characteristic information descriptive of a preferred route characteristic for the route; and wherein the model output comprises route information indicative of a route from the requested starting location to the requested destination location that comprises the preferred route characteristic. Abhishek states that doing so allows a user to have a more personalized navigation experience (par. 27). Additionally, it would increase the invention’s practical applications for the user by giving it another input/output option. As can be seen with other popular AI models such as ChatGPT, versatility is an obvious desirable trait.
Regarding claim 18, Dong teaches computing system of claim 15. Dong fails to teach obtaining the one or more inputs for the machine-learned semantic routing model comprises: obtaining, from the client computing device: route request information descriptive of a route from a requested starting location to a requested destination location; and preferred route characteristic information descriptive of a preferred route characteristic for the route; and wherein processing the one or more inputs to obtain the model output comprises: processing the information indicative of the route request information and the preferred route characteristic information to obtain the model output, wherein the model output comprises routing information indicative of a route from the requested starting location to the requested destination location that comprises the preferred route characteristic.
However, Abhishek teaches obtaining the one or more inputs for the machine-learned semantic routing model comprises: obtaining, from the client computing device:
route request information descriptive of a route from a requested starting location to a requested destination location (par. 38, “the processor 102 can execute instructions 132 to receive a user request 182 for a personalized route”);
and preferred route characteristic information descriptive of a preferred route characteristic for the route (par. 38, “The user request 182 can include certain request attributes 184…The request attributes may optionally include the mode of transport that the user proposes to employ, and certain route constraints that may be placed by the user i.e., whether the user would prefer the shortest/fastest route or a scenic route, or preferred stops along the route, or specific locations to avoid, etc.”);
and wherein processing the one or more inputs to obtain the model output comprises: processing the information indicative of the route request information and the preferred route characteristic information to obtain the model output, wherein the model output comprises routing information indicative of a route from the requested starting location to the requested destination location that comprises the preferred route characteristic (par. 53, “The routing engine 204 provides the optimal routes given a set of constraints provided by the recommender systems 206”).
Dong and Abhishek are analogous art because both are related to utilizing artificial intelligence to generate a route for a user (Abhishek specifies using AI at par. 73).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Dong in view of Yankov to incorporate the teachings of Abhishek to add the input data comprises: route request information descriptive of a route from a requested starting location to a requested destination location; and preferred route characteristic information descriptive of a preferred route characteristic for the route; and wherein the model output comprises route information indicative of a route from the requested starting location to the requested destination location that comprises the preferred route characteristic. Abhishek states that doing so allows a user to have a more personalized navigation experience (par. 27). Additionally, it would increase the invention’s practical applications for the user by giving it another input/output option. As can be seen with other popular AI models such as ChatGPT, versatility is an obvious desirable trait.
Claim(s) 14 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Dong in view of Yankov, and further in view of Jung (US 20230152113).
Regarding claim 14, the combination of Dong in view of Yankov teaches the computer-implemented method of claim 1. Dong fails to teach the method further comprises: processing, by the computing system, input data with the machine-learned semantic routing model to obtain a model output, wherein the input data comprises one or more images of one or more locations; and wherein the model output comprises itinerary information indicative of a proposed route that includes at least one of the one or more locations.
However, Jung teaches the method further comprises: processing, by the computing system, input data with the machine-learned semantic routing model to obtain a model output, wherein the input data comprises one or more images of one or more locations (par. 69, “The controller 150 according to various exemplary embodiments of the present disclosure may determine the recommended destination based on the output of the neural network for the image selected by the user among the images stored in the user terminal 20”; see Fig. 3); and wherein the model output comprises itinerary information indicative of a proposed route that includes at least one of the one or more locations (par. 69, “control the user interface 110 to perform a route guide to the recommended destination”; see Fig. 3).
Dong and Jung are analogous art because both are related to utilizing artificial intelligence to generate a route for a user.
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Dong in view of Yankov to incorporate the teachings of Jung to add the input data comprises one or more images of one or more locations; and wherein the model output comprises itinerary information indicative of a proposed route that includes at least one of the one or more locations. Doing so would allow the user to get directions to a destination without actually knowing the address (par. 4-5). Additionally, it would increase the invention’s practical applications for the user by giving it another input/output option. As can be seen with other popular AI models such as ChatGPT, versatility is an obvious desirable trait.
Regarding claim 19, the combination of Dong in view of Yankov teaches the computing system of claim 15. Dong fails to teach obtaining the one or more inputs for the machine-learned semantic routing model comprises: obtaining, from the client computing device, the information indicative of the requested route segment, wherein the information comprises one or more images of one or more locations; and wherein processing the one or more inputs to obtain the model output comprises: processing the one or more images of the one or more locations to obtain the model output, wherein the model output comprises routing information indicative of a proposed route that includes at least one of the one or more locations.
However, Jung teaches obtaining the one or more inputs for the machine-learned semantic routing model comprises: obtaining, from the client computing device, the information indicative of the requested route segment, wherein the information comprises one or more images of one or more locations (par. 69, “The controller 150 according to various exemplary embodiments of the present disclosure may determine the recommended destination based on the output of the neural network for the image selected by the user among the images stored in the user terminal 20”; see Fig. 3); and wherein processing the one or more inputs to obtain the model output comprises: processing the one or more images of the one or more locations to obtain the model output, wherein the model output comprises routing information indicative of a proposed route that includes at least one of the one or more locations (par. 69, “control the user interface 110 to perform a route guide to the recommended destination”; see Fig. 3).
Dong and Jung are analogous art because both are related to utilizing artificial intelligence to generate a route for a user.
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Dong in view of Yankov to incorporate the teachings of Jung to add the input data comprises one or more images of one or more locations; and wherein the model output comprises itinerary information indicative of a proposed route that includes at least one of the one or more locations. Doing so would allow the user to get directions to a destination without actually knowing the address (par. 4-5). Additionally, it would increase the invention’s practical applications for the user by giving it another input/output option. As can be seen with other popular AI models such as ChatGPT, versatility is an obvious desirable trait.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MINATO LEE HORNER whose telephone number is (571)272-5425. The examiner can normally be reached M-F 8-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Christian Chace can be reached at (571) 272-4190. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/M.L.H./Examiner, Art Unit 3665 /CHRISTIAN CHACE/Supervisory Patent Examiner, Art Unit 3665
Prosecution Insights