DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 4/14/2026 has been entered.
Response to Amendment
Claims 1, 14, and 21 are amended. Claims 25-26 are canceled. Claims 1-7 and 14-24 presented for examination.
Response to Arguments
Rejection under 35 U.S.C. 101
Applicant’s arguments have been fully considered and are persuasive. The amended independent claims recite using a machine learning execution chain containing machine learning blocks and a programmatic block to receive an input and prompt, generate a first model output containing a natural language claim based on the input and prompt, process an instruction of the programmatic block to retrieve grounding context corresponding to the claim from a database or website, generate a second model output including a factual accuracy for the claim using the grounding context, and providing an indication of the first model output and factual accuracy determination in response to the input. The claims recite an improvement to machine learning models by incorporating an execution chain of ML models containing a combination of machine learning blocks and programmatic blocks to perform a factual accuracy determination for a claim using additional retrieved grounding context, leading to less language model hallucinations and improved responses. Thus, the claims recite an improvement to the field of language models and integrate the judicial exception into a practical application under Step 2A, prong 2.
Rejection under 35 U.S.C. 102/103
Applicant’s arguments have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-7 and 14-24 are rejected under 35 U.S.C. 103 as being unpatentable over Cai et al. (US 20230112921 A1; hereinafter referred to as Cai) in view of Heller et al. (US 20240242037 A1; hereinafter referred to as Heller).
Regarding claim 1, Cai discloses: a system comprising: at least one processor; and memory storing instructions that, when executed by the at least one processor ([0110] The user computing device 102 includes one or more processors 112 and a memory 114), cause the system to perform a set of operations, the set of operations comprising: obtaining input to process ([0010] The operations include receiving an initial language input) according to a machine learning execution chain, wherein the machine learning execution chain includes a first machine learning block, a programmatic block ([0010] The model chain comprises a plurality of model instantiations of one or more machine-learned language models. Each model instantiation is configured to receive and process a model input to generate a model output), and a second machine learning block ([0019] the respective model input to at least one of the plurality of model instantiations in the model chain comprises: the model output from the sequentially previous model instantiation in the model chain; and the model output from one or more model instantiations in the model chain that each precede the sequentially previous model instantiation in the model chain);
generating, based on the input and a first prompt of the machine learning block, a first model output… ([0010] Each model instantiation is configured to receive and process a model input to generate a model output, wherein at least a portion of the model input comprises a model prompt descriptive of a task performed by the model instantiation on the model input to generate the model output);
processing an instruction of the programmatic block ([0041] the provided user interface can enable the user to: construct and/or edit a new or existing model chain and/or view and edit the inputs, outputs, and/or prompts for each instantiation within the chain. The user input can be an instruction.) to generate programmatic output for the programmatic block of the machine learning execution chain… ([0044] an interactive user interface can display a live visualization of the model chain structure and can allow users to customize chains at various levels. As examples, the user can be enabled to: iterate on or otherwise modify the local prompts per step; edit intermediate data transformed between steps; and/or reconstruct or modify the architecture of flow of the model chain);
generating, based on the programmatic output and the first model output ([0043] In a chain, a problem can be broken down into a number of smaller sub-tasks, each mapped to a distinct step with a corresponding prompt; results of one or more previous steps can be aggregated in the next step's input prompt. The output of the second model can use data from a previous programmatic output and first model output.), a second model output for the second machine learning block of the machine learning execution chain… ([0040] the present disclosure introduces the concept of chaining instantiations of machine-learned language models (e.g., LLMs) together, where the output of one instantiation becomes the input for the next, and so on, thus aggregating the gains per step).
Cai does not explicitly, but Heller teaches: including a natural language claim… ([0259, 0264] the request may be received as part of a chat flow. Alternatively, the request may be received as part of a correspondence generation flow. The request may, for instance, include a natural language instruction to generate a correspondence letter pertaining to a particular topic on behalf of a particular party… the text generation interface system determines a number of separate correspondence portions to generate. The correspondence portions are then generated at 1118 and 1120 and combined into a single correspondence at 1122. According to various embodiments, such operations may be performed by transmitting appropriate prompts to the text generation modeling system… The correspondence can be an example of a natural language claim.);
at least in part by programmatically retrieving, from a database or a website, additional grounding context ([0028] techniques and mechanisms described herein may be used to link a large language model with a legal research database, allowing the large language model to automatically determine appropriate searches to perform and then ground its responses to a source of truth (e.g., in actual law) so that it does not “hallucinate” a response that is inaccurate) corresponding to the natural language claim ([0069] a search of a database, set of documents, or other data source may be executed base at least in part on one or more search parameters determined based on a request received from a client machine. For instance, the request may identify one or more search terms and a set of documents to be searched using the one or more search terms), wherein the programmatic output ([0019] According to various embodiments, techniques and mechanisms described herein provide for retrieval augmented generation. A search is conducted base on a search query. Then, the search results are provided to an artificial intelligence system. The artificial intelligence system then further processes the search results to produce an answer based on that search results) includes the additional grounding context… ([0262] At 1108, one or more search queries are executed to determine search results. In some embodiments, one or more search queries may be executed against an external database such as a repository of case law, secondary sources, statutes, and the like. Alternatively, or additionally, one or more search queries may be executed against an internal database such as a repository of documents associated with the party generating the request at 1102. The search results can be the programmatic output including the grounding context.);
wherein the second model output includes a factual accuracy determination for the natural language claim ([0266] the identification of a factual claim may be associated with a respective set of search terms. The search terms may be used to search for evidence for or against the factual claims at 1126-1128. The results of these searches may then be provided in prompts to evaluate the factual claims sent to the text generation modeling system at 1130-1132. The text generation modeling system may complete the prompts by indicating whether the factual claims are accurate given the available search results);
and providing an indication of the first model output ([0267] the text generation interface system revises the correspondence by transmitting one or more prompts to the text generation modeling system. The requests may include the correspondence generated at 1122 as well as one or more results of the analysis of the factual claims. In this way, the text generation modeling system may revise the correspondence for accuracy, for instance by removing factual claims deemed to be inaccurate) and the factual accuracy determination ([0279] A determination is made at 1216 as to whether the factual assertion is accurate. In some embodiments, the determination may be made by parsing the response returned by the text generation modeling system at 1214. For instance, the text generation modeling system may complete the prompt by indicating whether the factual assertion is true, false, or uncertain based on the provided summary of search results) in response to the obtained input ([0031] The method 100 may be performed in order to generate new text based on input text provided by a client machine. For instance, the method 100 may be used to summarize a set of documents, generate correspondence, answer a search query, or the like).
Cai and Heller are considered analogous in the field of machine learning. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Cai to combine the teachings of Heller because doing so would allow for reduced input processing for improved prompt generation, and more accurate responses from LLMs by checking model response accuracy using an external database, leading to less model hallucinations and more useful responses when using multiple language models (Heller [0026, 0028] techniques and mechanisms described herein may be used to divide text into portions while respecting semantic boundaries and simultaneously reducing calls to the large language model. The cost of using many large language models depends on the amount of input and/or output text. Accordingly, techniques and mechanisms described herein provide for reduced overhead associated with prompt instructions while at the same time providing for improved model context to yield an improved response… techniques and mechanisms described herein may be used to link a large language model with a legal research database, allowing the large language model to automatically determine appropriate searches to perform and then ground its responses to a source of truth (e.g., in actual law) so that it does not “hallucinate” a response that is inaccurate).
Regarding claim 2, the combination of Cai and Heller teaches: the system of claim 1. Cai further teaches: wherein: the second model output is generated using a second prompt of the second machine learning block ([0058] example systems and methods of the present disclosure can assist in resolving this issue by chaining multiple prompts together, so that the problem is broken down into a number of smaller sub-tasks, each mapped to a distinct step with a corresponding prompt. In particular, in FIG. 1B, while the underlying LLM remains the same as was used in FIG. 1A, by splitting presentation problems and ideating suggestions per problem into distinct subtasks), thereby generating the second model output ([0019] the respective model input to at least one of the plurality of model instantiations in the model chain comprises: the model output from the sequentially previous model instantiation in the model chain; and the model output from one or more model instantiations in the model chain that each precede the sequentially previous model instantiation in the model chain).
Regarding claim 3, the combination of Cai and Heller teaches: the system of claim 1. Cai further teaches: wherein generating the first model output comprises populating the first prompt with at least a part of the obtained input ([0010] Each model instantiation is configured to receive and process a model input to generate a model output, wherein at least a portion of the model input comprises a model prompt descriptive of a task performed by the model instantiation on the model input to generate the model output. A user can also edit a prompt to contain part of the obtained input.), thereby generating a prompt template for processing by a machine learning model associated with the first machine learning block ([0085] Some example implementations define prompt templates per operation primarily with task descriptions. They specify the desired model functionality in an application-agnostic manner with key verbs and placeholders for application-specific details, improving model generalization).
Regarding claim 4, the combination of Cai and Heller teaches: the system of claim 1. Cai further teaches: wherein generating the first model output comprises: providing, to a machine learning service ([0113] one or more machine-learned models 140 can be included in or otherwise stored and implemented by the server computing system 130 that communicates with the user computing device 102 according to a client-server relationship. For example, the machine-learned models 140 can be implemented by the server computing system 140 as a portion of a web service (e.g., a model creation/interaction service)), an indication of the input and the first prompt ([0026] enables a user to edit, for each of the one or more of the model instantiations, a respective prompt that forms a portion of the respective model input of the one or more of the model instantiations);
and receiving, from the machine learning service, the first model output for the first machine learning block ([0127] The machine-learned model(s) can process the text or natural language data to generate an output).
Regarding claim 5, the combination of Cai and Heller teaches: the system of claim 1. Cai further teaches: wherein the programmatic block includes branching logic that corresponds to one or more additional blocks of the machine learning execution chain ([0134] The output may comprise a text output which is mapped to the spoken utterance. In some cases, the task comprises encrypting or decrypting input data. In some cases, the task comprises a microprocessor performance task, such as branch prediction or memory address translation. Also see Fig. 4, which includes branching logic.).
Regarding claim 6, the combination of Cai and Heller teaches: the system of claim 1. Cai further teaches: wherein the programmatic block includes looping logic that causes the first prompt of the first machine learning block ([0043] in a chain, a problem can be broken down into a number of smaller sub-tasks, each mapped to a distinct step with a corresponding prompt; results of one or more previous steps can be aggregated in the next step's input prompt. Prompts can be repeated for subsequent blocks.) to be processed in a subsequent iteration of at least a part of the machine learning execution chain ([0120] The training computing system 150 can include a model trainer 160 that trains the machine-learned models 120 and/or 140 stored at the user computing device 102 and/or the server computing system 130 using various training or learning techniques, such as, for example, backwards propagation of errors... Gradient descent techniques can be used to iteratively update the parameters over a number of training iterations).
Regarding claim 7, the combination of Cai and Heller teaches: the system of claim 1. Cai further teaches: wherein the programmatic block of the machine learning execution chain includes a reference to output generated by a previous block of the machine learning execution chain other than the first machine learning block ([0043] In a chain, a problem can be broken down into a number of smaller sub-tasks, each mapped to a distinct step with a corresponding prompt; results of one or more previous steps can be aggregated in the next step's input prompt. Thus, chaining enables users to run one or more language models (e.g., in some cases the same LLM) on multiple sub-tasks, with each sub-task having a higher probability of success (e.g., as opposed to solving the entire task in one go). Also see Fig. 4, which uses previous output of blocks in subsequent blocks.).
Regarding claim 14, it recites similar limitations as claim 1 and therefore is rejected similarly.
Regarding claim 15, it recites similar limitations as claim 2 and therefore is rejected similarly.
Regarding claim 16, it recites similar limitations as claim 3 and therefore is rejected similarly.
Regarding claim 17, it recites similar limitations as claim 4 and therefore is rejected similarly.
Regarding claim 18, it recites similar limitations as claim 5 and therefore is rejected similarly.
Regarding claim 19, it recites similar limitations as claim 6 and therefore is rejected similarly.
Regarding claim 20, it recites similar limitations as claim 7 and therefore is rejected similarly.
Regarding claim 21, Cai discloses: a method comprising: obtaining input to process ([0010] The operations include receiving an initial language input) according to a machine learning execution chain, wherein the machine learning execution chain includes a first machine learning block, a programmatic block ([0010] The model chain comprises a plurality of model instantiations of one or more machine-learned language models. Each model instantiation is configured to receive and process a model input to generate a model output), and a second machine learning block ([0019] the respective model input to at least one of the plurality of model instantiations in the model chain comprises: the model output from the sequentially previous model instantiation in the model chain; and the model output from one or more model instantiations in the model chain that each precede the sequentially previous model instantiation in the model chain);
generating, based on the input and a first prompt of the machine learning block, a first model output ([0010] Each model instantiation is configured to receive and process a model input to generate a model output, wherein at least a portion of the model input comprises a model prompt descriptive of a task performed by the model instantiation on the model input to generate the model output);
processing an instruction of the programmatic block ([0041] the provided user interface can enable the user to: construct and/or edit a new or existing model chain and/or view and edit the inputs, outputs, and/or prompts for each instantiation within the chain. The user input can be an instruction.) to generate programmatic output for the programmatic block of the machine learning execution chain… ([0044] an interactive user interface can display a live visualization of the model chain structure and can allow users to customize chains at various levels. As examples, the user can be enabled to: iterate on or otherwise modify the local prompts per step; edit intermediate data transformed between steps; and/or reconstruct or modify the architecture of flow of the model chain);
generating, based on the programmatic output, the first model output ([0043] In a chain, a problem can be broken down into a number of smaller sub-tasks, each mapped to a distinct step with a corresponding prompt; results of one or more previous steps can be aggregated in the next step's input prompt. The output of the second model can use data from a previous programmatic output and first model output.), and a second prompt of the second machine learning block ([0058] example systems and methods of the present disclosure can assist in resolving this issue by chaining multiple prompts together, so that the problem is broken down into a number of smaller sub-tasks, each mapped to a distinct step with a corresponding prompt), a second model output for the second machine learning block of the machine learning execution chain… ([0040] the present disclosure introduces the concept of chaining instantiations of machine-learned language models (e.g., LLMs) together, where the output of one instantiation becomes the input for the next, and so on, thus aggregating the gains per step).
Cai does not explicitly, but Heller teaches: including a natural language claim… ([0259, 0264] the request may be received as part of a chat flow. Alternatively, the request may be received as part of a correspondence generation flow. The request may, for instance, include a natural language instruction to generate a correspondence letter pertaining to a particular topic on behalf of a particular party… the text generation interface system determines a number of separate correspondence portions to generate. The correspondence portions are then generated at 1118 and 1120 and combined into a single correspondence at 1122. According to various embodiments, such operations may be performed by transmitting appropriate prompts to the text generation modeling system… The correspondence can be an example of a natural language claim.);
at least in part by programmatically retrieving, from a database or a website, additional grounding context ([0028] techniques and mechanisms described herein may be used to link a large language model with a legal research database, allowing the large language model to automatically determine appropriate searches to perform and then ground its responses to a source of truth (e.g., in actual law) so that it does not “hallucinate” a response that is inaccurate) corresponding to the natural language claim, wherein the programmatic output ([0019] According to various embodiments, techniques and mechanisms described herein provide for retrieval augmented generation. A search is conducted base on a search query. Then, the search results are provided to an artificial intelligence system. The artificial intelligence system then further processes the search results to produce an answer based on that search results) includes the additional grounding context… ([0262] At 1108, one or more search queries are executed to determine search results. In some embodiments, one or more search queries may be executed against an external database such as a repository of case law, secondary sources, statutes, and the like. Alternatively, or additionally, one or more search queries may be executed against an internal database such as a repository of documents associated with the party generating the request at 1102. The search results can be the programmatic output including the grounding context.);
wherein the second model output includes a factual accuracy determination for the natural language claim ([0266] the identification of a factual claim may be associated with a respective set of search terms. The search terms may be used to search for evidence for or against the factual claims at 1126-1128. The results of these searches may then be provided in prompts to evaluate the factual claims sent to the text generation modeling system at 1130-1132. The text generation modeling system may complete the prompts by indicating whether the factual claims are accurate given the available search results);
and providing an indication of the first model output ([0267] the text generation interface system revises the correspondence by transmitting one or more prompts to the text generation modeling system. The requests may include the correspondence generated at 1122 as well as one or more results of the analysis of the factual claims. In this way, the text generation modeling system may revise the correspondence for accuracy, for instance by removing factual claims deemed to be inaccurate) and the factual accuracy determination ([0279] A determination is made at 1216 as to whether the factual assertion is accurate. In some embodiments, the determination may be made by parsing the response returned by the text generation modeling system at 1214. For instance, the text generation modeling system may complete the prompt by indicating whether the factual assertion is true, false, or uncertain based on the provided summary of search results) in response to the obtained input ([0031] The method 100 may be performed in order to generate new text based on input text provided by a client machine. For instance, the method 100 may be used to summarize a set of documents, generate correspondence, answer a search query, or the like).
Regarding claim 22, it recites similar limitations as claim 3 and therefore is rejected similarly.
Regarding claim 23, it recites similar limitations as claim 5 and therefore is rejected similarly.
Regarding claim 24, it recites similar limitations as claim 7 and therefore is rejected similarly.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Nathan Tengbumroong whose telephone number is (703)756-1725. The examiner can normally be reached Monday - Friday, 11:30 am - 8:00 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hai Phan can be reached at 571-272-6338. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NATHAN TENGBUMROONG/Examiner, Art Unit 2654
/HAI PHAN/Supervisory Patent Examiner, Art Unit 2654
Prosecution Insights