DETAILED ACTION
In response to communication filed on 17 February 2026, claims 1, 3, 9, 14, 16, 18 and 20 are amended. Claims 1-20 are pending.
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments, see “Objections”, filed 17 February 2026, have been carefully considered and based on the claim amendments, the objections are withdrawn.
Applicant’s arguments, see “Section 103 Rejections”, filed 17 February 2026, have been carefully considered but are not considered to be persuasive. The arguments are related to newly added limitations and are addressed in the rejection below.
Claim Interpretation
Claim 1 recites “for generating multi-modal response to a query using a generative machine learning model”. These claim limitations appear to be citing intended use in terms of what the generative machine learning model is used for. Examiner suggests amending the claim to recite the functionality performed by the claimed method, instead of reciting what the claim elements are used for.
Claim 14 recites “for generating textual answer to a query using a generative machine learning model”. These claim limitations appear to be citing intended use in terms of what the generative machine learning model is used for. Examiner suggests amending the claim to recite the functionality performed by the claimed method, instead of reciting what the claim elements are used for.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-8, 10-11 and 13-20 are rejected under 35 U.S.C. 103 as being unpatentable over Baldua et al. (US 12,298,975 B2, hereinafter “Baldua”) in view of Gruber et al. (US 2013/0275164 A1, hereinafter “Gruber”) further in view of Mana (US 2016/0147878 A1, hereinafter “Mana”).
Regarding claim 1, Baldua teaches
A method for generating multi-modal response to a query using a generative machine learning model, the method comprising: (see Baldua, [col 13 lines 55-60] “one or more types of neural network-based machine learning model architectures include or are based on one or more multimodal neural networks capable of outputting different modalities (e.g., text, image, sound, etc.) separately and/or in combination based on textual input”; [col 4 lines 41-44] “A generative artificial intelligence (GAI) model or generative model uses artificial intelligence technology, e.g., neural networks, to machine-generate new digital content based on model inputs”; [col 14 lines 33-34] “The dynamic query planning system 110 receives the first query 106”; [col 8 line 1] “the method is performed by”).
receiving, from a client device, a query data object… (see Baldua, [col 9 lines 15-17] “the user input (e.g., first query 106 received via user interface mechanism 104 of application 102)”; [col 18 lines 33-34] “receives user input 202 and context data 204 from an application or client device (e.g., application 102)”).
providing the query data object and a first prompt to a machine learning system, wherein the first prompt includes (see Baldua, [col 47 lines 31-35] “receiving, via a user interface of an application, a first query including a user request for information retrievable using a first set of data resources, the first query including at least one first query term; configuring at least one prompt to cause a large language model to”; [col 13 lines 9-10] “Large language model 116 includes one or more neural network-based machine learning models”; [col 35 lines 4-6] “the processing device applies the large language model to the first prompt configured at operation 706 to obtain, from the large language model, the intent”) a set of functions and a description of each function of the set of functions, and (see Baldua, [col 4 lines 31-40] “the instructions set forth in the prompt, the large language model is to generate a query execution plan that includes a set of functions, where the set of functions are executable using a set of data resources to create a modified version of the initial query… instructions set forth in the prompt, the large language model is to select the set of functions in accordance with the user's explicit and/or implicit signals, e.g., the query input by the user and/or the user's history of interactions with the user interface”; [col 19 lines 44-47] “Function library 310 stores N templatized functions, and associated metadata, such as intents with which each function is associated. Function library 310 can include an index that maps functions with associated intents”) wherein each function of the set of functions is a link to at least one set of data stored in a data source; (see Baldua, [col 22 lines 57-64] “select resources 412 may include one or more examples of the types of criteria that can be used to select resources to which the selected set of functions are to be applied, along with an instruction to cause the large language model 404 to select resources to include in the plan 422 based on the examples provided in the select resources 412 portion of the prompt 402”; [col 39 line 64 – col 40 line 1] “(i) select a data resource of the at least first and second data resources based on the weights, and (ii) configure at least one function of the set of functions to obtain the at least one second query term from the selected data resource; [col 29 lines 17-20] “one or more sources of information, such as user connection network 636, entity graph 632, knowledge graph 634, one or more data stores of data storage system 660, or one or more data resources 650”).
receiving, from the machine learning system, a function, from the set of functions, wherein the received function is… (see Baldua, [col 47 lines 30-36] “receiving, via a user interface of an application, a first query including a user request for information retrievable using a first set of data resources, the first query including at least one first query term; configuring at least one prompt to cause a large language model to (i) translate the at least one first query term into a set of functions that can be executed”; [col 10 lines 50-51] “include a function to retrieve entity data related to the first query 106”; [col 35 lines 35-37] “where the processing device may simply execute the first query using a function based on the intent”) the query data object… (see Baldua, [col 9 lines 15-17] “the user input (e.g., first query 106 received via user interface mechanism 104 of application 102)”; [col 18 lines 33-34] “receives user input 202 and context data 204 from an application or client device (e.g., application 102)”) the set of functions… (see Baldua, [col 4 lines 37-38] “is to select the set of functions”; [col 11 line 40] “which includes the selected set of functions”) the description of each function of the set of functions… (see Baldua, [col 4 lines 31-40] “the instructions set forth in the prompt, the large language model is to generate a query execution plan that includes a set of functions, where the set of functions are executable using a set of data resources to create a modified version of the initial query… instructions set forth in the prompt, the large language model is to select the set of functions in accordance with the user's explicit and/or implicit signals, e.g., the query input by the user and/or the user's history of interactions with the user interface”; [col 19 lines 44-47] “Function library 310 stores N templatized functions, and associated metadata, such as intents with which each function is associated. Function library 310 can include an index that maps functions with associated intents”) the query data object; (see Baldua, [col 9 lines 15-17] “the user input (e.g., first query 106 received via user interface mechanism 104 of application 102)”; [col 18 lines 33-34] “receives user input 202 and context data 204 from an application or client device (e.g., application 102)”).
receiving, from the machine learning system, an output format; (see Baldua, [col 6 lines 62-67] “the disclosed technologies are not limited to generative models that receive text input and produce text output… the disclosed technologies can be used to receive input and/or generate output that includes non-text forms of content, such as digital imagery, videos, multimedia, audio, hyperlinks, and/or platform-independent file formats”).
providing at least one set of data mapped to the function, (see Baldua, [col 10 lines 35-46] “a plan generation prompt 122 based on the input classification 118 produced by the large language model 116… a query execution plan can include a set of functions that retrieve data from multiple different data resources 134 and incorporate at least some of that retrieved data into the modified version of the user input”’ [col 22 lines 57-60] “cause the large language model 404 to select one or more resources from the possible resources 414 for inclusion in the plan 422. For example, select resources 412 may include one or more examples of the types of criteria that can be used to select resources to which the selected set of functions are to be applied”) the query data object, and a second prompt to the machine learning system, (see col 36 lines 35-49] “configuring at least one second prompt based on the context data, the at least one second prompt including at least one instruction to cause the large language model to generate and output a second plan executable to formulate at least one recommendation for modifying the first query; applying the large language model to the at least one second prompt to obtain, from the large language model, the second plan including a second set of functions configured by the large language model… the large language model executes the instructions contained in the second prompt to select the set of functions, map the corresponding portions of the user input and/or context data to the respective functions (e.g., as parameters or arguments), determine the order of operation for the set of functions, arrange the functions according to the order of operation, and output the plan”).
receiving, from the machine learning system, a response to the query data object, (see Baldua, [col 37 lines 58-60] “a large language model is used to generate a query execution plan for processing a user input including a search query”; [col 39 lines 8-18] “the processing device executes the modified version of the first query created at operation 758 based on the at least one second query term to provide, via the user interface, a response to the first query… applies the modified version of the first query to one or more data resources (e.g., one or more of data resources 134 or data resources 176) to obtain a result set, and then formulates the response based on the result set”) wherein the response is formatted based on the output format; and (see Baldua, [col 6 lines 62-67] “the disclosed technologies are not limited to generative models that receive text input and produce text output… the disclosed technologies can be used to receive input and/or generate output that includes non-text forms of content, such as digital imagery, videos, multimedia, audio, hyperlinks, and/or platform-independent file formats”).
outputting the response to one or more users (see Baldua, [col 23 lines 50-52] “a user interface 500 includes a display of search results 506 that have been returned for a user's query 502”).
Baldua does not explicitly teach a query data object related to a sporting event; the received function is most correlated with the query data object out of the set of functions based on comparing the description of each function of the set of functions to the data query object.
However, Gruber discloses natural language dialog and teaches
query related to a sporting event; (see Gruber, [1086] “suppose the user says, "Who is playing the Lakers tonight?"… the digital assistant recognizes the sports-related vocabulary "the Lakers" and the sports-related language pattern "Who is playing [a sports team] . . . "… uses the context information (e.g., the current date) to determine which date the user is referring to by the word "tonight" in the speech input. After the digital assistant has fully disambiguated the user's speech input, the digital assistant proceeds to perform a search to retrieve the requested information. Specifically, the digital assistant retrieves the name of the team that is playing against the Lakers in the evening of the current date”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include the functionality of query related to a sporting event, player, current match state, historical team function, internet browser, textual response, preferences, as being disclosed and taught by Gruber, in the system taught by Baldua to yield the predictable results of providing improved efficiency for the user (see Gruber, [0136]-[0138] “better interpretation of user input (e.g., using personal history and physical context when interpreting language)… more personalized results ( e.g., that bias toward preferences or recent selections)… improved efficiency for the user ( e.g., by automating steps involving the signing up to services or filling out forms)”).
The proposed combination of Baldua and Gruber teaches the received function is most correlated with the query data object out of the set of functions based on comparing the description of each function of the set of functions to the data query object.
However, Mana discloses retrieving information and teaches
selecting response R that is most correlated with query… out of the plurality of content… based on comparing plurality of content C… to query Q (see Mana, [0047] “The semantic search engine returns to the user 10 a response R as the result of a matching process consisting in comparing the natural language query Q with a plurality of contents C, formed of phrases or expressions obtained from a contents database 6, and selecting the response R as being the contents corresponding to the comparison having a best semantic matching degree”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include the functionality of comparing to query in order to determine most correlated information as being disclosed and taught by Mana, in the system taught by the proposed combination of Baldua and Gruber to yield the predictable results of providing easy and effective recognition when it comes to retrieve actual and meaningful information when performing searching work (see Mana, [0015] “provides a tool for an easy and effective recognition when it comes to retrieve actual and meaningful information when performing searching work”).
Claims 14 and 18 incorporate substantively all the limitations of claim 1 in a system (see Baldua, [col 43 lines 23-24] “computer system 800 includes a processing device 802, a main memory 804”; [col 44 lines 33-35] “The received code can be executed by processing device 802 as it is received, and/or stored in data storage system 840, or other non-volatile storage for later execution”; see Gruber, [0153]-[0154] “Examples of different types of output data/information which may be generated by intelligent automated assistant 1002 may include, but are not limited to, one or more of the following (or combinations thereof)…Text output sent directly to an output device and/or to the user interface of a device”) and computer-readable medium form (see Baldua, [col 50 lines 63-67] “at least one non-transitory machine-readable storage medium including at least one instruction that, when executed by at least one processor, causes the at least one processor to perform at least one operation including”) and are rejected under the same rationale.
Regarding claim 2, the proposed combination of Baldua, Gruber and Mana teaches
wherein the query data object is (see Baldua, [col 9 lines 15-17] “the user input (e.g., first query 106 received via user interface mechanism 104 of application 102)”; [col 18 lines 33-34] “receives user input 202 and context data 204 from an application or client device (e.g., application 102)”) a query related to a player, (see Gruber, [1082] “users often ask questions related to game scores and player statistics”). The motivation for the proposed combination is maintained.
Claims 15 and 19 incorporate substantively all the limitations of claim 2 in a system and computer-readable medium form and are rejected under the same rationale.
Regarding claim 3, the proposed combination of Baldua, Gruber and Mana teaches
wherein the first prompt includes: (see Baldua, [col 47 lines 31-35] “receiving, via a user interface of an application, a first query including a user request for information retrievable using a first set of data resources, the first query including at least one first query term; configuring at least one prompt to cause a large language model to”; [col 13 lines 9-10] “Large language model 116 includes one or more neural network-based machine learning models”; [col 35 lines 4-6] “the processing device applies the large language model to the first prompt configured at operation 706 to obtain, from the large language model, the intent”).
a machine readable request instructing the machine learning system (see Baldua, [col 53 lines 60-61] “configuring a first instruction to cause the generative machine learning model to”) to associate the query data object with a function from the set of functions (see Baldua, [col 51 lines 25-31] “identify a plurality of functions related to context data associated with at least one of the user interface or the application and cause the large language model to select the set of functions from the plurality of functions based on context data associated with at least one of the user interface, the application, the set of second data resources, or the first query”) based on the description of each function (see Baldua, [col 3 lines 45-49] “select functions, such as functions for determining filters and facets, to be included in a plan for executing a search query based on the user's natural language input, without requiring the user to explicitly select those facets, filters, or other functions”; [col 19 lines 44-47] “Function library 310 stores N templatized functions, and associated metadata, such as intents with which each function is associated. Function library 310 can include an index that maps functions with associated intents”).
Claims 16 and 20 incorporate substantively all the limitations of claim 3 in a system and computer-readable medium form and are rejected under the same rationale.
Regarding claim 4, the proposed combination of Baldua, Gruber and Mana teaches
wherein the first prompt includes: (see Baldua, [col 47 lines 31-35] “receiving, via a user interface of an application, a first query including a user request for information retrievable using a first set of data resources, the first query including at least one first query term; configuring at least one prompt to cause a large language model to”; [col 13 lines 9-10] “Large language model 116 includes one or more neural network-based machine learning models”; [col 35 lines 4-6] “the processing device applies the large language model to the first prompt configured at operation 706 to obtain, from the large language model, the intent”).
a set of output formats; (see Baldua, [col 6 lines 62-67] “the disclosed technologies are not limited to generative models that receive text input and produce text output… the disclosed technologies can be used to receive input and/or generate output that includes non-text forms of content, such as digital imagery, videos, multimedia, audio, hyperlinks, and/or platform-independent file formats”).
a description of each output format; and (see Baldua, [col 39 line 58 – col 40 line 5] “includes assigning different weights to at least first and second data resources of the second set of data resources based on at least one of metadata.. metadata associated with a data resource can include an indication of the type of information that can be retrieved from the data resource or the format of the information stored in the data resource ( e.g., text, images, video, audio, etc.)”).
a machine readable request instructions the machine learning system (see Baldua, [col 53 lines 60-61] “configuring a first instruction to cause the generative machine learning model to”) to associate the query data object with the output format from the set of output formats (see Baldua, [col 47 line 61 – col 48 line 7] “assigning different weights to at least first and second data resources of the second set of data resources based on at least one of metadata… associated with the at least first and second data resources, and configuring the at least one prompt to cause the large language model to (i) select a data resource of the at least first and second data resources based on the weights, and (ii) configure at least one function of the set of functions to obtain the at least one second query term from the selected data resource… determining whether the first query translates to a plurality of functions; and responsive to determining that the first query translates to the plurality of functions”)..
Claim 17 incorporates substantively all the limitations of claim 4 in a system form and is rejected under the same rationale.
Regarding claim 5, the proposed combination of Baldua, Gruber and Mana teaches
wherein the set of output formats include audio, images, videos (see Baldua, [col 6 lines 62-67] “the disclosed technologies are not limited to generative models that receive text input and produce text output… the disclosed technologies can be used to receive input and/or generate output that includes non-text forms of content, such as digital imagery, videos, multimedia, audio, hyperlinks, and/or platform-independent file formats”; [col 39 line 58 – col 40 line 5] “includes assigning different weights to at least first and second data resources of the second set of data resources based on at least one of metadata.. metadata associated with a data resource can include an indication of the type of information that can be retrieved from the data resource or the format of the information stored in the data resource ( e.g., text, images, video, audio, etc.)”).
Regarding claim 6, the proposed combination of Baldua, Gruber and Mana teaches
wherein the set of functions are each mapped to respective data sources and types of information (see Baldua, [col 22 lines 36-53] “where the set of functions are executable using a set of data resources… determine function parameters 410 may include one or more examples of the types of parameter values that are applicable to certain functions along with an instruction to cause the large language model 404 to determine the function parameters 410 based on the examples provided in the determine function parameters 410 portion”).
Regarding claim 7, the proposed combination of Baldua, Gruber and Mana teaches
wherein the set of functions include: (see Baldua, [col 10 lines 35-46] “The plan generation prompt generator 120 configures a plan generation prompt 122… can include a set of functions that retrieve data from multiple different data resources 134 and incorporate at least some of that retrieved data into the modified version of the user input”).
a non-sports question (see Baldua, [col 3 lines 44-54] “a large language model to automatically select functions… if the user requests "fortune 500 companies on the west coast… determine the names of the fortune 500 companies located in Washington, Oregon, and California and add them as facets without requiring the user to explicitly select those company names as query terms”).
Regarding claim 8, the proposed combination of Baldua, Gruber and Mana teaches
wherein if the received function, from the set of functions, (see Baldua, [col 10 lines 35-46] “The plan generation prompt generator 120 configures a plan generation prompt 122… can include a set of functions that retrieve data from multiple different data resources 134 and incorporate at least some of that retrieved data into the modified version of the user input”) is the current match state function, (see Gruber, [1084] “queries related to current games or season”; [0223] “the various types of functions, operations, actions, and/or other features”) then the second prompt includes: (see Baldua, [col 50 line 50] “the at least one second prompt including”).
a machine readable request instructing the machine learning system (see Baldua, [col 53 lines 60-61] “configuring a first instruction to cause the generative machine learning model to”) to answer the query data object based on (see Baldua, [col 5 lines 17-20] “A large language model can be configured to perform one or more natural language processing (NLP) tasks, such as generating text, classifying text, answering questions in a conversational manner”) the current match state function (see Gruber, [1084] “queries related to current games or season”; [0223] “the various types of functions, operations, actions, and/or other features”). The motivation for the proposed combination is maintained.
Regarding claim 10, the proposed combination of Baldua, Gruber and Mana teaches
wherein if the received function from the set of functions, (see Baldua, [col 10 lines 35-46] “The plan generation prompt generator 120 configures a plan generation prompt 122… can include a set of functions that retrieve data from multiple different data resources 134 and incorporate at least some of that retrieved data into the modified version of the user input”) is a non-sports question, (see Baldua, [col 3 lines 44-54] “a large language model to automatically select functions… if the user requests "fortune 500 companies on the west coast… determine the names of the fortune 500 companies located in Washington, Oregon, and California and add them as facets without requiring the user to explicitly select those company names as query terms”) then the method further includes: (see Baldua, [col 8 line 1] “the method is performed by”).
performing a search for the query data object (see Baldua, [col 12 lines 23-24] “as a result of executing the plan 124 include job search results related to the first query 106”) through an internet browser; (see Gruber, [0591]-[0597] “Output data may include, for example, communication in natural language between the intelligent automated assistant and the user; data about domain entities, such as properties of restaurants, movies, products, and the like; domain specific data results from information services… use different output processing layouts and formats depending on which web browser and/or device is being used… Send a stream of output packages to a modality, showing intermediate status, feedback, or results throughout phases of interaction with assistant 1002”; [0010] “by activating and/or interfacing with any applications or services that may be available on an electronic device, as well as services that are available over an electronic network such as the Internet”).
saving results from (see Baldua, [col 23 lines 55-57] “that enable the user viewing the result set 506 to perform actions in relation to the search result, such as storing the result for future use”) the internet browser; and (see Gruber, [0624]-[0626] “Store the history of the dialog and user interactions in a database on the client, the server in a user-specific session, or in client session state such as web browser cookies or RAM used by the client; [0625] Store the list of recent user requests; [0626] Store the sequence of results of recent user requests”).
updating the second prompt to include a machine readable request to (see Baldua, [col 48 lines 37-40] “modifying the at least one prompt based on the context data to include at least one instruction to cause the large language model to generate and output a modified version of the plan”; [col 50 line 50] “the at least one second prompt including”) respond to the query data object and (see Baldua, [col 37 lines 58-60] “a large language model is used to generate a query execution plan for processing a user input including a search query”; [col 39 lines 8-18] “the processing device executes the modified version of the first query created at operation 758 based on the at least one second query term to provide, via the user interface, a response to the first query… applies the modified version of the first query to one or more data resources (e.g., one or more of data resources 134 or data resources 176) to obtain a result set, and then formulates the response based on the result set”) form a textual response (see Gruber, [0153]-[0154] “Examples of different types of output data/information which may be generated by intelligent automated assistant 1002 may include, but are not limited to, one or more of the following (or combinations thereof)…Text output sent directly to an output device and/or to the user interface of a device”) based on the results (see Baldua, [col 12 lines 23-24] “as a result of executing the plan 124 include job search results related to the first query 106”) from the internet browser (see Gruber, [0591]-[0597] “Output data may include, for example, communication in natural language between the intelligent automated assistant and the user; data about domain entities, such as properties of restaurants, movies, products, and the like; domain specific data results from information services… use different output processing layouts and formats depending on which web browser and/or device is being used… Send a stream of output packages to a modality, showing intermediate status, feedback, or results throughout phases of interaction with assistant 1002”; [0010] “by activating and/or interfacing with any applications or services that may be available on an electronic device, as well as services that are available over an electronic network such as the Internet”). The motivation for the proposed combination is maintained.
Regarding claim 11, the proposed combination of Baldua, Gruber and Mana teaches
wherein if the received function from the set of functions, (see Baldua, [col 10 lines 35-46] “The plan generation prompt generator 120 configures a plan generation prompt 122… can include a set of functions that retrieve data from multiple different data resources 134 and incorporate at least some of that retrieved data into the modified version of the user input”) is the graphic function, (see Baldua, [col 26 lines 47] “a graphical display such as a web page or mobile device screen, into which digital content such as search results, feed items, chat boxes, or threads, can be loaded for display to the use”) then the method further includes: (see Baldua, [col 8 line 1] “the method is performed by”).
a machine readable request instructing the machine learning system to (see Baldua, [col 53 lines 60-61] “configuring a first instruction to cause the generative machine learning model to”) provide an image related to the query data object (see Baldua, [col 13 lines 57-64] “multimodal neural networks capable of outputting different modalities (e.g., text, image, sound, etc.) separately and/or in combination based on textual input. Accordingly, in some examples, a multimodal neural network implemented in the dynamic query planning system is capable of outputting digital content that includes a combination of two or more of text, images, video or audio”) based on the graphic function (see Baldua, [col 26 lines 47] “a graphical display such as a web page or mobile device screen, into which digital content such as search results, feed items, chat boxes, or threads, can be loaded for display to the use”).
Regarding claim 13, the proposed combination of Baldua, Gruber and Mana teaches
wherein the query data object (see Baldua, [col 9 lines 15-17] “the user input (e.g., first query 106 received via user interface mechanism 104 of application 102)”; [col 18 lines 33-34] “receives user input 202 and context data 204 from an application or client device (e.g., application 102)”) query related to a sporting event includes (see Gruber, [1086] “suppose the user says, "Who is playing the Lakers tonight?"… the digital assistant recognizes the sports-related vocabulary "the Lakers" and the sports-related language pattern "Who is playing [a sports team] . . . "… uses the context information (e.g., the current date) to determine which date the user is referring to by the word "tonight" in the speech input. After the digital assistant has fully disambiguated the user's speech input, the digital assistant proceeds to perform a search to retrieve the requested information. Specifically, the digital assistant retrieves the name of the team that is playing against the Lakers in the evening of the current date”) preferences for format, (see Gruber, [0591]-[0595] “Format output data that is represented in a uniform internal data structure into forms and layouts that render it appropriately on different modalities… Dynamically render data for different graphical user interface display engines based on the request. For example, use different output processing layouts and formats depending on which web browser and/or device is being used… Dynamically render to specified modalities based on user preferences”) the method further comprising (see Baldua, [col 8 line 1] “the method is performed by”) providing the preferences to the assistant (see Gruber, [0693] “assistant 1002 helps the user explore the space of possible choices, eliciting the user's constraints and preferences”; [1047] “if the user has specified a party-size requirement in the restaurant search request, or if the digital assistant has inferred the party-size requirement based on context information (e.g., prior user input, prior user interactions, or default user preferences, etc.)”) the machine learning system (see Baldua, [col 10 lines 15-16] “configured to formulate a feature set for input to a classical machine learning model”). The motivation for the proposed combination is maintained.
Claims 9 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Baldua, Gruber and Mana in view of Shachaf et al. (US 12,353,407 B1, hereinafter “Shachaf”).
Regarding claim 9, the proposed combination of Baldua, Gruber and Mana teaches
wherein if the received function from the set of functions, (see Baldua, [col 10 lines 35-46] “The plan generation prompt generator 120 configures a plan generation prompt 122… can include a set of functions that retrieve data from multiple different data resources 134 and incorporate at least some of that retrieved data into the modified version of the user input”) is the historical team function, (see Gruber, [1082] “for the sports domain, users often ask questions related to… player and/or team history”; [0223] “the various types of functions, operations, actions, and/or other features”) method further includes: (see Baldua, [col 8 line 1] “the method is performed by”).
accessing a database; (see Baldua, [col 30 lines 10-12] “Each data resource 650 enables dynamic query planning system 110 to access the data resource”; [col 30 lines 4-6] “Examples of data resources 650 include entity graphs, knowledge graphs, indexes, databases”).
requesting historical information from the database; (see Baldua, [col 40 lines 8-10] “Feedback can include historical data about user reactions to presentations of information retrieved from the data source (e.g., likes, comments, shares, etc.)”; [col 30 lines 4-6] “Examples of data resources 650 include entity graphs, knowledge graphs, indexes, databases”).
obtaining the historical information in a structured query language (SQL) Query; and (see Baldua, [col 40 lines 8-10] “Feedback can include historical data about user reactions to presentations of information retrieved from the data source (e.g., likes, comments, shares, etc.)”; [col 30 lines 4-6] “Examples of data resources 650 include entity graphs, knowledge graphs, indexes, databases”; [col 31 lines 66-67] “Data can be written to and read from data stores using query technologies, e.g., SQL”).
updating the second prompt to include a machine readable request to… (see Baldua, [col 48 lines 37-40] “modifying the at least one prompt based on the context data to include at least one instruction to cause the large language model to generate and output a modified version of the plan”; [col 50 line 50] “the at least one second prompt including”) that responds to the query data object and form a response to the query data object (see Baldua, [col 37 lines 58-60] “a large language model is used to generate a query execution plan for processing a user input including a search query”; [col 39 lines 8-18] “the processing device executes the modified version of the first query created at operation 758 based on the at least one second query term to provide, via the user interface, a response to the first query… applies the modified version of the first query to one or more data resources (e.g., one or more of data resources 134 or data resources 176) to obtain a result set, and then formulates the response based on the result set”).
The motivation for the proposed combination of Baldua, Gruber and Mana does not explicitly teach the second prompt to include a machine readable request to adapt the SQL query to extract data.
However, Shachaf discloses wrapped prompts and teaches
a prompt to adapt the SQL query to extract data (see Shachaf, [col 6 lines 41-49] “may generate, by a large language model (LLM), a query based on the wrapped prompt, where the query may describe or include a plurality database operations or commands… desired output from an LLM may be for example an SQL query including one or more SQL commands, which may for example select two columns from a database, which may be used to extract a plurality of data entries or data items from a database”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include the functionality of SQL query to extract data and second machine learning system as being disclosed and taught by Shachaf, in the system taught by the proposed combination of Baldua, Gruber and Mana to yield the predictable results of improving data analysis technologies (see Shachaf, [col 22 lines 58-64] “may improve data analysis technologies by providing a user friendly, natural language based approach for producing database queries and corresponding data driven insights without requiring technical expertise (e.g., in SQL and/or additional relevant programming languages and tools) from a human user”).
Regarding claim 12, the proposed combination of Baldua, Gruber and Mana teaches
wherein if the received function from the set of functions, (see Baldua, [col 10 lines 35-46] “The plan generation prompt generator 120 configures a plan generation prompt 122… can include a set of functions that retrieve data from multiple different data resources 134 and incorporate at least some of that retrieved data into the modified version of the user input”) is the generation function, (see Baldua, [col 10 lines 47-51] “functions that can be included in the set of functions selected by the large language model 116 to be included in the plan 124, based on the plan generation prompt 122, include a function to retrieve entity data related to the first query 106”) then the method further includes: (see Baldua, [col 8 line 1] “the method is performed by”).
sending a machine readable request instructing a model to provide a response to the query data object (see Baldua, [col 53 lines 60-61] “configuring a first instruction to cause the generative machine learning model to”; [col 4 lines 28-30] “to include instructions to cause one or more generative artificial intelligence models (e.g., one or more large language models)” – there are plurality of models; [col 37 lines 58-60] “a large language model is used to generate a query execution plan for processing a user input including a search query”; [col 39 lines 8-18] “the processing device executes the modified version of the first query created at operation 758 based on the at least one second query term to provide, via the user interface, a response to the first query… applies the modified version of the first query to one or more data resources (e.g., one or more of data resources 134 or data resources 176) to obtain a result set, and then formulates the response based on the result set”) based on the generation function; and (see Baldua, [col 10 lines 47-51] “functions that can be included in the set of functions selected by the large language model 116 to be included in the plan 124, based on the plan generation prompt 122, include a function to retrieve entity data related to the first query 106”).
receiving the response from models (see Baldua, [col 53 lines 60-61] “configuring a first instruction to cause the generative machine learning model to”; [col 4 lines 28-30] “to include instructions to cause one or more generative artificial intelligence models (e.g., one or more large language models)” – there are plurality of models; [col 37 lines 58-60] “a large language model is used to generate a query execution plan for processing a user input including a search query”; [col 39 lines 8-18] “the processing device executes the modified version of the first query created at operation 758 based on the at least one second query term to provide, via the user interface, a response to the first query… applies the modified version of the first query to one or more data resources (e.g., one or more of data resources 134 or data resources 176) to obtain a result set, and then formulates the response based on the result set”).
The motivation for the proposed combination of Baldua, Gruber and Mana does not explicitly teach a second machine learning system to provide a response; receiving the response from the second machine learning system.
However, Shachaf discloses wrapped prompts and teaches
a second machine learning system that generates metadata based on the query (see Shachaf, [col 5 lines 35-38] “a second machine learning model (which may be, e.g., separate and distinct from the first machine learning model) may be used for generating or producing chart metadata or metadata items based on a query”).
the second machine learning system (see Shachaf, [col 5 lines 35-38] “a second machine learning model (which may be, e.g., separate and distinct from the first machine learning model) may be used for generating or producing chart metadata or metadata items based on a query”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include the functionality of SQL query to extract data and second machine learning system as being disclosed and taught by Shachaf, in the system taught by the proposed combination of Baldua, Gruber and Mana to yield the predictable results of improving data analysis technologies (see Shachaf, [col 22 lines 58-64] “may improve data analysis technologies by providing a user friendly, natural language based approach for producing database queries and corresponding data driven insights without requiring technical expertise (e.g., in SQL and/or additional relevant programming languages and tools) from a human user”).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to VAISHALI SHAH whose telephone number is (571)272-8532. The examiner can normally be reached Monday - Friday (7:30 AM to 4:00 PM).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, AJAY BHATIA can be reached at (571)272-3906. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/VAISHALI SHAH/Primary Examiner, Art Unit 2156
Prosecution Insights