DETAILED ACTION
Claims 1-20 of the instant application are pending and have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 09/26/2024 was filed. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Specification
The lengthy specification has not been checked to the extent necessary to determine the presence of all possible minor errors. Applicant’s cooperation is requested in correcting any errors of which applicant may become aware in the specification.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 6, 14, and 20 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 6, 14, and 20 recites the limitation "the existence of the search method" in line 6 of claim 6, line 5 of claim 14, and line 4 of claim 20. There is insufficient antecedent basis for this limitation in the claim.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim(s) 1-20 rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. More specifically directed to the abstract idea grouping of: mental process.
The independent claim(s) 1 and 17 recite(s):
1. One or more non-transitory computer-readable media storing one or more computer programs, the one or more computer programs configured to cause at least one processor to:
search for and classify a task that a user intends to complete based on natural language content of a message from the user;
responsive to the task being found and classified, provide input based on the content of the message to a large language model (LLM) and execute the LLM to understand and extract one or more parameter values from the natural language content;
receive output from the LLM as a result of the execution thereof; and
based on the received output from the LLM, generate a configuration file and perform one or more actions pertinent to the task using the generated configuration file.
17. A computer-implemented method, comprising:
searching for and classifying, by a computing system, a task that a user intends to complete based on natural language content of a message from the user;
responsive to the task being found and classified, providing input based on the content of the message, by the computing system, to a large language model (LLM) and executing the LLM, by the computing system or another computing system, to understand and extract one or more parameter values from the natural language content;
receiving output from the LLM as a result of the execution thereof, by the computing system; and
based on the received output from the LLM, generating a configuration file and performing one or more actions pertinent to the task using the generated configuration file, by the computing system.
This reads on a human (e.g., mentally and/or using pen and paper):
Analyzing (i.e., searching and classifying) a task based on a request (e.g., verbal or written) from a second human;
After the analysis, use the results (i.e., from searching and classifying) to follow a predetermined set of steps or rules and extract parameter values from the request;
Obtaining results from the predetermined set of steps or rules;
Based on results from the predetermined set of steps or rules, write down information and respond to the request.
The independent claim(s) 12 recite(s):
12. One or more computing systems, comprising:
memory storing computer program instructions;
and at least one processor configured to execute the computer program instructions, wherein the computer program instructions are configured to cause the at least one processor to:
search for and classify a task that a user intends to complete based on natural language content of a message from the user;
responsive to the task being found and classified, provide input based on the content of the message to a large language model (LLM) and execute the LLM to understand and extract one or more parameter values from the natural language content;
receive output from the LLM as a result of the execution thereof; and
based on the received output from the LLM, generate a configuration file and perform one or more actions pertinent to the task using the generated configuration file,
wherein the understanding and extracting of the one or more parameter values from the natural language content comprises employing at least one of chain-of-thought prompting, prompt chaining, Extensible Markup Language (XML) tagging, few-shot learning, and mocked-exchange instructions to help balance between missed extractions and hallucinations, and
the performing of the one or more actions pertinent to the task comprises automatically interacting with backend systems.
This reads on a human (e.g., mentally and/or using pen and paper):
Analyzing (i.e., searching and classifying) a task based on a request (e.g., verbal or written) from a second human;
After the analysis, use the results (i.e., from searching and classifying) to follow a predetermined set of steps or rules and extract parameter values from the request;
Obtaining results from the predetermined set of steps or rules;
Based on results from the predetermined set of steps or rules, write down information and respond to the request;
Wherein the extracting parameter values consist of employing a predetermined set of rules;
Responding to the request comprises interaction with predefined information.
This judicial exception is not integrated into a practical application because for example: claim 1 recites “computer-readable media,” “one or more computer programs,” “at least one processor,” and “large language model”, while claim 12 additionally recites “one or more computing systems,” “memory storing program instructions,” and “backend systems”, similarly, claim 17 recites “computer-implemented method” and “computing system”. As an example, in [0102] of the as filed specification, it is disclosed: “The computer programs can be implemented in hardware, software, or a hybrid implementation. The computer programs can be composed of modules that are in operative communication with one another, and which are designed to pass information or instructions to display. The computer programs can be configured to operate on a general purpose computer, an ASIC, or any other suitable device.” Therefore, a general-purpose computer or computing device is described and mainly used as an application thereof. Accordingly, these additional elements do not integrate the abstract idea into a practical idea because it does not impose any meaningful limits on practicing the abstract idea.
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional elements of using a computer is listed as a general computing device as noted. The claim is not patent eligible.
With respect to claims 2, 13, and 18, the claim(s) recite:
automatically insert prompt template inputs based on the natural language content and one or more configured input parameters that are missing; and
prompt the user for input values comprising an option to search and select entities and the automatically inserted prompt template inputs.
This reads on a human (e.g., mentally and/or using pen and paper):
using templates based on the request from the second human and any missing data;
ask the second human to indicate any missing data, including an option to search and select entities/categories and the template inputs.
No additional limitations are present.
With respect to claim 3, the claim(s) recite:
3. The one or more non-transitory computer-readable media of claim 1,
wherein the performing of the one or more actions pertinent to the task comprises automatically interacting with backend systems.
This reads on a human (e.g., mentally and/or using pen and paper):
responding to the request comprises interaction with predefined information.
Additional limitation of “backend system” is present. Same analysis provided for independent claims 1 and 12 applies.
With respect to claim 4, the claim(s) recite:
4. The one or more non-transitory computer-readable media of claim 1, wherein
the one or more computer programs comprise a plurality of modules and use case names, and
the input to the LLM comprises at least one module of the plurality of modules and at least one of the use case names for intent classification.
This reads on a human (e.g., mentally and/or using pen and paper):
using a plurality of predefined modules/names;
using said predefined modules/names to classify the intent of the request.
No additional limitations are present.
With respect to claim 5 and 19, the claim(s) recite:
wherein the understanding and extracting of the one or more parameter values from the natural language content comprises employing at least one of chain-of-thought prompting, prompt chaining, Extensible Markup Language (XML) tagging, few-shot learning, and mocked-exchange instructions to help balance between missed extractions and hallucinations.
This reads on a human (e.g., mentally and/or using pen and paper):
wherein the understanding an extracting comprises performing a predetermined set of rules/steps.
No additional limitations are present.
With respect to claims 6, 14, and 20, the claim(s) recite:
check whether a defined entity search method exists based on one or more input parameters; and
responsive to the existence of the search method, trigger the defined entity search method, wherein the configuration file comprises at least one search method definition of options that are available for fetching entity objects from one or more backend systems.
This reads on a human (e.g., mentally and/or using pen and paper):
checking whether a name/entity/category exists;
based on the determination of whether the name/entity/category exists, perform the search, wherein there are predetermined set of steps for the search for fetching/obtaining data from predefined sources.
Additional limitation of “backend system” is present. Same analysis provided for independent claims 1 and 12 applies.
With respect to claim 7, the claim(s) recite:
7. The one or more non-transitory computer-readable media of claim 1, wherein the one or more computer programs are configured to handle a plurality of different application programming interface (API) structures.
This reads on a human (e.g., mentally and/or using pen and paper):
handling multiple predefined applications.
Additional limitation of “application programming interface (API) structures” is present. Same analysis provided for independent claims 1, 12, and 17 applies.
With respect to claims 8 and15, the claim(s) recite:
wherein the configuration file comprises criteria for sorting search results such that a dedicated searching or sorting application programming interface (API) is not required.
This reads on a human (e.g., mentally and/or using pen and paper):
using predetermined set of steps to sort search results.
Additional limitation of “application programming interface (API) structures” is present. Same analysis provided for independent claims 1, 12, and 17 applies.
With respect to claims 9 and 16, the claim(s) recite:
wherein the one or more actions pertinent to the task comprise at least one of an application programming interface (API) request, code execution, robotic process automation (RPA), and an external script.
This reads on a human (e.g., mentally and/or using pen and paper):
wherein the response comprises a request in a predefined application.
Additional limitations of “application programming interface (API) structures” and “robotic process automation (RPA)” are present. Same analysis provided for independent claims 1, 12, and 17 applies.
With respect to claims 10, the claim(s) recite:
10. The one or more non-transitory computer-readable media of claim 1, wherein the generation of the configuration file and the performing of the one or more actions pertinent to the task using the generated configuration file comprises constructing a Universal Resource Locator (URL) and a body based on the input and triggering a corresponding backend application programming interface (API).
This reads on a human (e.g., mentally and/or using pen and paper):
wherein the response comprises determining a website address or URL based on input and using a corresponding predefined application.
No additional limitations are present.
With respect to claims 11, the claim(s) recite:
11. The one or more non-transitory computer-readable media of claim 1, wherein
the one or more actions comprise a plurality of actions, and
the plurality of actions are chained, where execution results of a previous action are provided as input to a subsequent action in the chain.
This reads on a human (e.g., mentally and/or using pen and paper):
performing actions or responses comprise a plurality of actions, and
the plurality of actions are chained/connected/ordered/related.
No additional limitations are present.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Liu et al. (US 20250390525 A1).
As to independent claim 1, Liu et al. teaches:
1. One or more non-transitory computer-readable media storing one or more computer programs, the one or more computer programs configured to cause at least one processor (see ¶ [0353]: “… For example, a computer system or other data processing system, such as the computing system 100 or the computing system 700, can carry out the above-described computer-implemented methods in response to its processor executing a computer program (e.g., a sequence of instructions) contained in a memory or other non-transitory machine-readable storage medium…”) to:
search for and classify a task that a user intends to complete based on natural language content of a message from the user (see ¶ [0029, 0039-0040, and 0043]: “[0029] A generative artificial intelligence model, generative machine learning model, or generative model uses artificial intelligence technology to machine-generate digital content based on model inputs and data with which the model has been trained. A generative language model is a particular type of generative model that is capable of generating and outputting digital content in response to model input including a task description, also referred to as a prompt. [0039] Embodiments of the described approach can be used to improve, for example, generative model-based conversational agents. For example, general purpose-, task- and/or domain-specific generative model-based conversational agents can be configured and implemented at scale because the reliability, relevance, and accuracy of the model-generated content can be improved using the described approaches. [0040] Examples of conversational agents include search agents, assessment agents, navigation agents, and combinations of any of these and/or other agents. Search agents can perform information search and retrieval functions using a conversational dialog-based user interface. In the case of search agents, the model-generated content produced using COTRO can include a disambiguated search query. Examples of search agents include job search agents, people search agents, and other entity-based search agents (e.g., product search, content search, company search, etc.). [0043] Generative model-based navigation agents can also or alternatively use COTRO to generate a classification of an input or state, where the conversational agent can use the classification to make routing decisions among agents of a multi-agent system.”);
responsive to the task being found and classified (see ¶ [0029, 0039-0040, and 0043] citations as in limitation above, more specifically ¶ [0039]: “Embodiments of the described approach can be used to improve, for example, generative model-based conversational agents. For example, general purpose-, task- and/or domain-specific generative model-based conversational agents can be configured and implemented at scale…” and ¶ [0043]: “Generative model-based navigation agents can also or alternatively use COTRO to generate a classification of an input or state, where the conversational agent can use the classification to make routing decisions among agents of a multi-agent system.”.), provide input based on the content of the message to a large language model (LLM) and execute the LLM to understand and extract one or more parameter values from the natural language content (see Fig. 4A3 (422: “Am I a good fit for this job?”) and ¶ [0081, 0118-0119, 0122 and 0141]: “[0081] The input 161 is used to configure an input classification prompt 174. For example, the input 161 can be merged or combined with an input classification prompt template to create the input classification prompt 174, where the input classification prompt template contains one or more instructions to cause the classification machine learning model 168 to, e.g., extract one or more entities from the input 161 and use e.g., binary classification, to classify the input 161 based on the extracted one or more entities. The entities can include canonical words or phrases such as names, locations, skills, job titles, nouns, verbs, adjectives, adverbs, etc. In other implementations, the input 161 can include one or more features extracted from the input 161, such as vectors or embeddings… [0096] Each prompt is configured for its associated machine learning model. For example, if the portion of the agent's task (e.g., a first sub-task) performed at state 1 involves the use of a first type of large language model, the associated state 1 prompt is configured for input to that first type of large language model (e.g., an LLM that does not have reasoning capabilities). If the portion of the agent's task (e.g., a second sub-task) performed at state 2 involves the use of a second type of large language model (e.g., an LLM that does have reasoning capabilities), the associated state 2 prompt is configured for input to that second type of large language model. [0118] In FIG. 4A1, FIG. 4A2, FIG. 4A3, a user interface 400 of an application system includes input boxes 402, 404 and a search execution option 407. The input boxes 402, 406 can prompt the user to input structured search term such as job title, skill, company name, or geographic location. Once the user has input one or more structured search terms via one or more of the input boxes 402, 406, the user can select the search execution option 407 to cause the application system to execute a search query on, e.g., one or more databases or other corpus of digital content. [0119] In the example of FIG. 4A1, FIG. 4A2, FIG. 4A3, the application system has generated and presented via the user interface 400 a scrollable list of suggested job postings 406. For example, the application system executed a search query entered by the user or proactively generated the list 406 based on, e.g., aspects of the user's online profile and/or previous search history. [0122] The user interface 400 also includes a conversational agent panel 420. The conversational agent panel 420 provides a conversational dialog-based format for interacting with the application system, which does not require the user to input structured search terms. In the example of FIG. 4A1, FIG. 4A2, FIG. 4A3, the user has input a conversational natural language (i.e., unstructured) request 422 (“Am I a good fit for this job?”) or the user has selected the selectable option 416 and in response to the user's selection of the option 416, the application system has opened the conversational agent panel 420 and presented the request 422. In response to the request 422, the conversational agent initiates a job analysis process using a state machine as described, e.g., with reference to FIG. 2 and FIG. 3. For instance, the conversational agent or a state thereof formulates a COTRO prompt for job assessment generation using the request 422, the current application state (e.g., conversational agent panel 420 is open), context data, and the techniques described herein, and passes the COTRO prompt for job assessment generation to a machine learning model. The machine learning model processes the COTRO prompt for job assessment generation and outputs the job analysis 424. The MLM-generated job analysis includes highlighted portions 426, 428, 430. Highlighted portion 426 includes a machine learning model-generated summary of or conclusion regarding the extent to which the user's online profile matches the job description. Highlighted portions 428 and 430 indicate machine learning model-generated reasoning supporting the MLM-generated job assessment conclusion presented via highlighted portion 426.”);
receive output from the LLM as a result of the execution thereof (see ¶ [0081, 0118-0119, 0122] citations as in limitation above, more specifically: ¶ [0122]: “…The machine learning model processes the COTRO prompt for job assessment generation and outputs the job analysis 424…”); and
based on the received output from the LLM, generate a configuration file and perform one or more actions pertinent to the task using the generated configuration file (see ¶ [0047, 0061, 0065, and 0121]: “[0047] … An agent system as used here can refer to a system that is capable of being used to create an agent, configure an agent, and/or cause one or more agents to execute one or more actions, tasks, sub-actions, or sub-tasks. [0061] …An example of a COTRO prompt template is described with reference to FIG. 1B. The machine learning model 110 can be implemented using, e.g., a pre-trained generative machine learning model, such as an LLM, an LM, or another type of generative model. [0065] FIG. 1B is an example of a prompt template in accordance with some embodiments of the present disclosure. In FIG. 1B, an exemplary prompt template 120 includes a task practice instruction 122, a reasoning generation instruction 126, and a response generation instruction 130. Each of the task practice instruction 122, reasoning generation instruction 126, and response generation instruction 130 includes a respective set of instructions 124, 128, 132. In FIG. 1B, the sets of instructions 124, 128, 132 are in the form of natural language text. In other embodiments, one or more of the sets of instructions 124, 128, 132 can include non-text content or multimodal content, for example. [0121] In contrast, the content presented in connection with the MLM-generated options 416 (e.g., “Am I a good fit for this job?”), 418 is generated dynamically by a machine learning model using, e.g., current application state data, user input, interaction history, user profile data, and the COTRO-based content generation techniques described herein. For instance, the application system or a navigation agent of the application system detects a user selection of the Principal Product Manager job posting in the listing 406. The application system or navigation agent passes the user selection of that job posting, the current application state data (e.g., detailed job posting view), and context data if available to a COTRO prompt generator. The COTRO prompt generator generates and outputs a COTRO prompt for navigation option generation using approaches described, e.g., with reference to FIG. 1A and FIG. 1B. The application system or navigation agent passes the COTRO prompt to a machine learning model, and the machine learning model generates and outputs the selectable options 416, 418 as described, e.g., with reference to FIG. 1C and FIG. 1D. As a result, the content of the MLM-generated options 416, 418 changes in response to changes in the user input, application state, and/or context. Because the context is one of the inputs to the content generation prompt, the MLM-generated options 416, 418 can be customized according to user preferences such as language, style and tone.”).
As to independent claim 12, Liu et al. teaches:
12. One or more computing systems (see ¶ [0047]: “Agent as used herein can refer to an automated agent, a sub-agent, or a group of agents that programmatically execute one or more automated or semi-automated processes via a computer system…”), comprising:
memory storing computer program instructions (see ¶ [0047] citation as in limitation above and further ¶ [0340]: “The example computer system 900 includes a processing device 902, a main memory 904 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM) such as synchronous DRAM (SDRAM) or Rambus DRAM (RDRAM), etc.), a memory 903 (e.g., flash memory, static random access memory (SRAM), etc.), an input/output system 910, and a data storage system 940, which communicate with each other via a bus 930.”);
and at least one processor configured to execute the computer program instructions, wherein the computer program instructions are configured to cause the at least one processor (see ¶ [0047 and 0340] citations as in limitations above: “processing device” and “memory” and further ¶ [0341]: “Processing device 902 represents at least one general-purpose processing device such as a microprocessor, a central processing unit, or the like. More particularly, the processing device can be a complex instruction set computing (CISC) microprocessor, reduced instruction set computing (RISC) microprocessor, very long instruction word (VLIW) microprocessor, or a processor implementing other instruction sets, or processors implementing a combination of instruction sets. Processing device 902 can also be at least one special-purpose processing device such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), network processor, or the like. The processing device 902 is configured to execute instructions 912 for performing the operations and steps discussed herein.”) to:
search for and classify a task that a user intends to complete based on natural language content of a message from the user (see ¶ [0029, 0039-0040, and 0043] citations as in claim 1, above.);
responsive to the task being found and classified, provide input based on the content of the message to a large language model (LLM) and execute the LLM to understand and extract one or more parameter values from the natural language content (see ¶ [0029, 0039-0040, 0043, 0081, 0118-0119, and 0122] citations as in claim 1, above.);
receive output from the LLM as a result of the execution thereof (see ¶ [0081, 0118-0119, 0122] citations as in claim 1, above.); and
based on the received output from the LLM, generate a configuration file and perform one or more actions pertinent to the task using the generated configuration file (see ¶ [0047, 0061, 0065, and 0121] citations as in claim 1, above.),
wherein the understanding and extracting of the one or more parameter values from the natural language content comprises employing at least one of chain-of-thought prompting, prompt chaining, Extensible Markup Language (XML) tagging, few-shot learning, and mocked-exchange instructions to help balance between missed extractions and hallucinations (see ¶ [0037]: “Embodiments address these and/or other technical challenges. Embodiments provide in-prompt hallucination management by applying chain of thought reasoning to preliminary model-generated output. The described approach can be referred to herein as chain of thought reasoning on the output (COTRO). In contrast to COTRI, the described COTRO approach focuses on refining the model-generated output rather than on refining the model input (e.g., the examples included in the prompt)…”), and
the performing of the one or more actions pertinent to the task comprises automatically interacting with backend systems (see ¶ [0242]: “A request includes, for example, a network message such as an HTTP (HyperText Transfer Protocol) request for a transfer of data from an application front end to the application's back end, or from the application's back end to the front end, or, more generally, a request for a transfer of data between two different devices or systems, such as data transfers between servers and user systems. ”).
As to independent claim 17, Liu et al. teaches:
17. A computer-implemented method (see ¶ [0058]: “The method is performed by processing logic that includes hardware (e.g., processing device, circuitry, dedicated logic, programmable logic, microcode, hardware of a device, integrated circuit, etc.), software (e.g., instructions run or executed on a processing device), or a combination thereof. In some embodiments, the method is performed by components of distributed multi-agent system 105, including, in some embodiments, components or flows shown in FIG. 1A that may not be specifically shown in other figures and/or including, in some embodiments, components or flows shown in other figures that may not be specifically shown in FIG. 1A.”), comprising:
searching for and classifying, by a computing system, a task that a user intends to complete based on natural language content of a message from the user (see ¶ [0029, 0039-0040, and 0043] citations as in claim 1, above.);
responsive to the task being found and classified, providing input based on the content of the message, by the computing system, to a large language model (LLM) and executing the LLM, by the computing system or another computing system, to understand and extract one or more parameter values from the natural language content (see ¶ [0029, 0039-0040, 0043, 0081, 0118-0119, and 0122] citations as in claim 1, above.);
receiving output from the LLM as a result of the execution thereof, by the computing system (see ¶ [0081, 0118-0119, 0122] citations as in claim 1, above.); and
based on the received output from the LLM, generating a configuration file and performing one or more actions pertinent to the task using the generated configuration file, by the computing system (see ¶ [0047, 0061, 0065, and 0121] citations as in claim 1, above.).
Regarding claims 2, 13, and 18, Liu et al. teaches the limitations as in claims 1, 12, and 17, above.
Liu et al. further teaches:
2 and 13. The one or more non-transitory computer-readable media/ computer systems of claims 1 and 12, wherein the one or more computer programs are further configured to cause the at least one processor (see ¶ [0353]: “… For example, a computer system or other data processing system, such as the computing system 100 or the computing system 700, can carry out the above-described computer-implemented methods in response to its processor executing a computer program (e.g., a sequence of instructions) contained in a memory or other non-transitory machine-readable storage medium…”) to:
automatically insert prompt template inputs based on the natural language content and one or more configured input parameters that are missing (see ¶ [0223 and 0371]: “[0223] In some implementations, the prompt is to cause the first machine learning model to search the second natural language input for the data before searching the historical sequence or the user profile. In some implementations, the prompt is to cause the first machine learning model to search the historical sequence before searching the user profile. In some implementations, the prompt is to cause the first machine learning model use at least one entity extracted from the first natural language input to formulate the first search query. In some implementations, the prompt is to cause the first machine learning model to identify at least one query term missing from the first natural language input, obtain the at least one missing query term from at least one of the ranked plurality of data sources, and include the at least one missing query term in the first search query. [0371] In some aspects, the techniques described herein relate to a method, wherein the prompt is to cause the first machine learning model to identify at least one query term missing from the first natural language input, obtain the at least one missing query term from at least one of the ranked plurality of data sources, and include the at least one missing query term in the first search query.”); and
prompt the user for input values comprising an option to search and select entities and the automatically inserted prompt template inputs (see ¶ [0223 and 0371] citations as in limitation above and further ¶ [0118 and 0231]: “[0118] In FIG. 4A1, FIG. 4A2, FIG. 4A3, a user interface 400 of an application system includes input boxes 402, 404 and a search execution option 407. The input boxes 402, 406 can prompt the user to input structured search term such as job title, skill, company name, or geographic location. Once the user has input one or more structured search terms via one or more of the input boxes 402, 406, the user can select the search execution option 407 to cause the application system to execute a search query on, e.g., one or more databases or other corpus of digital content. [0231] User interface 712 can be used to interact with the agent system 780 and/or one or more application systems 730. For example, user interface 712 enables the user of a user system 710 to interact with an application system to create, edit, send, view, receive, process, and organize requests, search queries, search results, content items, news feeds, and/or portions of online dialogs. In some implementations, user interface 712 enables the user to input requests (e.g., queries) for various different types of information, to initiate user interface events, and to view or otherwise perceive output such as data and/or digital content produced by, e.g., an application system 730, agent system 780, content distribution service 738 and/or search engine 740. ”).
18. The computer-implemented method of claim 17, further comprising:
automatically inserting prompt template inputs, by the computing system, based on the natural language content and one or more configured input parameters that are missing (see ¶ [0223 and 0371] citations as in claim 2 and 13, above.); and
prompting the user for input values comprising an option to search and select entities and the automatically inserted prompt template inputs, by the computing system (see ¶ [0118, 0223, 0231, and 0371] citations as in claim 2 and 13, above.).
Regarding claim 3, Liu et al. teaches the limitations as in claims 1, above.
Liu et al. further teaches:
3. The one or more non-transitory computer-readable media of claim 1, wherein the performing of the one or more actions pertinent to the task comprises automatically interacting with backend systems (see ¶ [0047, 0061, 0065, and 0121] citations as in claim 1, above and further ¶ [0242]: “A request includes, for example, a network message such as an HTTP (HyperText Transfer Protocol) request for a transfer of data from an application front end to the application's back end, or from the application's back end to the front end, or, more generally, a request for a transfer of data between two different devices or systems, such as data transfers between servers and user systems. ”).
Regarding claim 4, Liu et al. teaches the limitations as in claims 1, above.
Liu et al. further teaches:
4. The one or more non-transitory computer-readable media of claim 1, wherein the one or more computer programs comprise a plurality of modules and use case names (see ¶ [0072 and 0262]: “[0072] In the example of FIG. 1C, a method 140 illustrates how a machine learning model 144 can process a COTRO prompt 142. The machine learning model 144 can be implemented using, e.g., a pre-trained or fine-tuned generative machine learning model, such as an LLM, an LM, or another type of generative model. [0262] Machine learning systems include components and processes that perform data generation, model training, model evaluation (e.g., calibration and validation), and application. Data preparation includes obtaining and aggregating model input data. The preparation of training data can include labeling the aggregated data. Training data can include structured data, unstructured data, text, multimodal data, or any combination of any of the foregoing. Model training can include configuring hyperparameters, determining performance metrics, and applying the machine learning model to the training data, evaluating the performance metrics, and parameter tuning. Application includes applying the trained machine learning model to the real-world environment, e.g., in a specific use case using data not included in the training data (e.g., unlabeled data). The application phase can be referred to as inferencing or inference time.”), and
the input to the LLM comprises at least one module of the plurality of modules and at least one of the use case names for intent classification (see ¶ [0072 and 0262] citations as in limitation above, more specifically: “[0072] …machine learning model, such as an LLM… [0262] Machine learning systems include components and processes that perform data generation, model training, model evaluation (e.g., calibration and validation), and application. Data preparation includes obtaining and aggregating model input data. ”).
Regarding claims 5 and 19, Liu et al. teaches the limitations as in claims 1 and 17, above.
Liu et al. further teaches:
5 and 19. The one or more non-transitory computer-readable media / computer implemented method of claims 1 and 17,
wherein the understanding and extracting of the one or more parameter values from the natural language content comprises employing at least one of chain-of-thought prompting, prompt chaining, Extensible Markup Language (XML) tagging, few-shot learning, and mocked-exchange instructions to help balance between missed extractions and hallucinations (see ¶ [0037]: “Embodiments address these and/or other technical challenges. Embodiments provide in-prompt hallucination management by applying chain of thought reasoning to preliminary model-generated output. The described approach can be referred to herein as chain of thought reasoning on the output (COTRO). In contrast to COTRI, the described COTRO approach focuses on refining the model-generated output rather than on refining the model input (e.g., the examples included in the prompt)…”).
Regarding claims 6, 14, and 20, Liu et al. teaches the limitations as in claims 1, 12, and 17, above.
Liu et al. further teaches:
6 and 14. The one or more non-transitory computer-readable media/ computer systems of claims 1 and 12, wherein the one or more computer programs are further configured to cause the at least one processor (see ¶ [0353]: “… For example, a computer system or other data processing system, such as the computing system 100 or the computing system 700, can carry out the above-described computer-implemented methods in response to its processor executing a computer program (e.g., a sequence of instructions) contained in a memory or other non-transitory machine-readable storage medium…”) to:
check whether a defined entity search method exists based on one or more input parameters (see ¶ [0243 and 0285]: “[0243] In the example of FIG. 7, application system 730 includes a search engine 740. Search engine 740 includes a software system designed to search for and retrieve information by executing queries on one or more data stores, such as databases, connection networks, and/or graphs. The queries are designed to find information that matches specified criteria, such as keywords and phrases contained in user input and/or system-generated queries. For example, search engine 740 is used to retrieve data in response to user input and/or system-generated queries, by executing queries on various data stores of data storage system 760 and/or data resources and tools 750, or by traversing entity graph 732, knowledge graph 734. [0285] A generative artificial intelligence (GAI) model or generative model uses artificial intelligence technology, e.g., machine learning, neural networks, to machine-generate digital content based on model inputs and the previously existing data with which the model has been trained.”); and
responsive to the existence of the search method, trigger the defined entity search method (see ¶ [0243 and 0285] citations as in limitation above and further ¶ [0102, 0216, and 0218-0219]: “[0102] Using a state machine architecture such as described with reference to FIG. 2, state 1 applies a machine learning model to input 302 to generate an input classification 306. The machine learning model used by state 1 can be, for example, a classification model or a large language model configured to perform a classification task (e.g., an LLM that does not have reasoning capabilities). State 1 executes transition logic to determine whether to proceed to state 2 of agent A or transition to a different agent, e.g., agent B. For example, if the input classification 306 matches a criterion, category or type that is related to the task that agent A is configured to perform, the state 1 transition logic initiates a transition to state 2 of agent A and passes the input classification 306 to state 2. If the input classification 306 does not match a criterion, category, or type related to the agent A task, the state 1 transition logic identifies an agent that matches the input classification 306 and initiates a transition to that agent, e.g., agent B. [0216] At operation 644, the processing device provides a prompt to a first machine learning model. The prompt includes at least one instruction configured using the COTRO approach described herein. The at least one instruction is to cause the first machine learning model to use at least the input to rank a plurality of data sources associated with the use of the conversational search system. For example, the at least one instruction includes a query disambiguation scheme such as described with reference to FIG. 5A1, FIG. 5A2, FIG. 5A3, FIG. 5B1, FIG. 5B2, FIG. 5B3, FIG. 5C1, FIG. 5C2, FIG. 5C3, and FIG. 5D1, FIG. 5D2, FIG. 5D3. [0218] The at least one instruction is to cause the first machine learning model to use the first search query and the reasoning to generate a second search query. For example, the second search query includes a second or subsequent disambiguated version of the first search query. The second search query is, for example, a hallucination-managed version of the first search query, which is produced using the COTRO approach. [0219] At operation 646, the processing device uses a second machine learning model to synthesize a response determined via execution of the second search query. For example, a query system is invoked to execute the second search query on one or more data sources, and results of the execution of the second search query are passed as input to the second machine learning model. The synthesized response is produced by the second machine learning model using the COTRO approach described herein. ”),
wherein the configuration file comprises at least one search method definition of options that are available for fetching entity objects from one or more backend systems (see ¶ [0102, 0216, 0218-0219, 0243 and 0285] citations as in limitations above and further ¶ [0116. 0242, and 0258]: “[0116] The user interface elements shown in FIG. 4A1, FIG. 4A2, FIG. 4A3 are presented to a user by an application system, such as a conversational agent. In some implementations, portions of the user interface elements are implemented as one or more web pages that are stored, e.g., at a user device, a server or in a cache of a user device, and then loaded into a display of a user device via the user device sending a page load request to the server or fetching data from the cache. [0242] A request includes, for example, a network message such as an HTTP (HyperText Transfer Protocol) request for a transfer of data from an application front end to the application's back end, or from the application's back end to the front end, or, more generally, a request for a transfer of data between two different devices or systems, such as data transfers between servers and user systems. [0258] In the embodiment of FIG. 9, portions of agent system 780 that may be implemented on a front end system, such as one or more user systems, and portions of agent system 780 that may be implemented on a back end system such as one or more servers, are collectively represented as agent system 950 for ease of discussion only.”).
20. The computer-implemented method of claim 17, further comprising:
checking, by the computing system, whether a defined entity search method exists based on one or more input parameters (see ¶ [0243 and 0285] citations as in claims 6 and 14, above.); and
responsive to the existence of the search method, triggering the defined entity search method, by the computing system (see ¶ [0102, 0216, 0218-0219, 0243 and 0285] citations as in claims 6 and 14, above.),
wherein the configuration file comprises at least one search method definition of options that are available for fetching entity objects from one or more backend systems (see ¶ [0102, 0116, 0216, 0218-0219, 0242-0243, 0258, and 0285] citations as in claims 6 and 14, above.).
Regarding claim 7, Liu et al. teaches the limitations as in claims 1, above.
Liu et al. further teaches:
7. The one or more non-transitory computer-readable media of claim 1, wherein the one or more computer programs are configured to handle a plurality of different application programming interface (API) structures (see ¶ [0073 and 0231]: “[0073] In some implementations, the COTRO prompt 142 is a multi-step prompt that is input to the machine learning model 144 via a single communication (e.g., a single application programming interface (API) call). [0231] … Examples of user interface 712 include web browsers, command line interfaces, and mobile app front ends. User interface 712 as used herein can include application programming interfaces (APIs).”).
Regarding claims 8 and 15, Liu et al. teaches the limitations as in claims 1 and 12, above.
Liu et al. further teaches:
8 and 15. The one or more non-transitory computer-readable media/ computer systems of claims 1 and 12, wherein the configuration file comprises criteria for sorting search results such that a dedicated searching or sorting application programming interface (API) is not required (see ¶ [0042 and 0141]: “[0042] Navigation agents can dynamically generate selectable navigation options to be presented via a user interface, based on the current state of a conversational agent, where each selectable option identifies an action that is statistically or probabilistically ranked as being of likely interest to a user. In the case of navigation agents, the model-generated content produced using COTRO can include descriptions or depictions of actions identified by or associated with the selectable options. [0141] In the example of FIG. 5A1, FIG. 5A2, FIG. 5A3, the application system has generated and presented via the user interface 500 scrollable lists of suggested job postings 508, 510. For example, the application system has proactively executed search queries that the application system has formulated using a machine learning model, one or more COTRO prompts, and information about the user, such as aspects of the user's online profile and/or previous search history. The lists of suggested job postings 508, 510 are grouped by criteria. For example, the list 508 includes search results returned by a query including search terms extracted from the user's profile and sorted by relevance while the list 510 highlights search results that have a social connection to the user.”).
Regarding claims 9 and 16, Liu et al. teaches the limitations as in claims 1 and 12, above.
Liu et al. further teaches:
9 and 16. The one or more non-transitory computer-readable media/ computer systems of claims 1 and 12,
wherein the one or more actions pertinent to the task comprise at least one of an application programming interface (API) request, code execution, robotic process automation (RPA), and an external script (see ¶ [0047, 0061, 0065, and 0121] citations as in claims 1 and 12, above and further ¶ [0073, 0231, and 0242]: “[0073] In some implementations, the COTRO prompt 142 is a multi-step prompt that is input to the machine learning model 144 via a single communication (e.g., a single application programming interface (API) call). [0231] … Examples of user interface 712 include web browsers, command line interfaces, and mobile app front ends. User interface 712 as used herein can include application programming interfaces (APIs). [0242] A request includes, for example, a network message such as an HTTP (HyperText Transfer Protocol) request for a transfer of data from an application front end to the application's back end, or from the application's back end to the front end, or, more generally, a request for a transfer of data between two different devices or systems, such as data transfers between servers and user systems. ”).
Regarding claim 10, Liu et al. teaches the limitations as in claim 1, above.
Liu et al. further teaches:
10. The one or more non-transitory computer-readable media of claim 1, wherein the generation of the configuration file and the performing of the one or more actions pertinent to the task using the generated configuration file comprises constructing a Universal Resource Locator (URL) and a body based on the input and triggering a corresponding backend application programming interface (API) (see ¶ [0047, 0061, 0065, and 0121] citations as in claims 1 and 12, above and further ¶ [0051 and 0231-0232]: “[0051] Certain aspects of the disclosed technologies are described in the context of generative artificial intelligence models that receive text input and output text. However, the disclosed technologies are not limited to generative models that receive text input and produce text output. For example, aspects of the disclosed technologies can be used to receive input and/or generate output that includes non-text forms of content, such as digital imagery, videos, multimedia, audio, hyperlinks, and/or platform-independent file formats. [0231] …Examples of user interface 712 include web browsers, command line interfaces, and mobile app front ends. User interface 712 as used herein can include application programming interfaces (APIs). [0232] Network 720 includes an electronic communications network. Network 720 can be implemented on any medium or mechanism that provides for the exchange of digital data, signals, and/or instructions between the various components of computing system 700. Examples of network 720 include, without limitation, a Local Area Network (LAN), a Wide Area Network (WAN), an Ethernet network or the Internet, or at least one terrestrial, satellite or wireless link, or a combination of any number of different networks and/or communication links.”).
Regarding claim 11, Liu et al. teaches the limitations as in claim 1, above.
Liu et al. further teaches:
11. The one or more non-transitory computer-readable media of claim 1,
wherein the one or more actions comprise a plurality of actions (see ¶ [0047, 0061, 0065, and 0121] citations as in claim 1, above, more specifically ¶ [0047]: “ … An agent system as used here can refer to a system that is capable of being used to create an agent, configure an agent, and/or cause one or more agents to execute one or more actions, tasks, sub-actions, or sub-tasks.”), and
the plurality of actions are chained, where execution results of a previous action are provided as input to a subsequent action in the chain (see ¶ [0037 and 0061]: “[0037] Embodiments address these and/or other technical challenges. Embodiments provide in-prompt hallucination management by applying chain of thought reasoning to preliminary model-generated output. The described approach can be referred to herein as chain of thought reasoning on the output (COTRO). [0061] Alternatively or in addition, the query system 104 can retrieve a chain of thought reasoning on the output (COTRO) prompt template 106 from a data store, e.g., a prompt library, and pass the COTRO prompt template 106 to prompt generator 108. The COTRO prompt template 106 includes one or more instructions that are configured to cause a machine learning model to perform chain of thought reasoning on the output as described herein.”).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Regarding performing tasks based on user request associated with LLM and/or AI-based agents (pertinent to claims 1-20):
Collins et al. (US 20260134493 A1, ¶ [0380, 0404, 0407, and 0433])
Pletner et al. (US 20250307095 A1, ¶ [0064])
Jeong et al. (US 20250278647 A1, ¶ [0064 and 0112])
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Keisha Y Castillo-Torres whose telephone number is (571)272-3975. The examiner can normally be reached Monday - Friday, 9:00 am - 4:00 pm (EST).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pierre-Louis Desir can be reached at (571)272-7799. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Keisha Y. Castillo-Torres
Examiner
Art Unit 2659
/Keisha Y. Castillo-Torres/Examiner, Art Unit 2659
Prosecution Insights