DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The amendment filed 05/14/2026 has been entered. Claims 1 and 19 have been amended. Claims 1-19 remain pending in the application.
Response to Arguments
Claim Rejections - 35 USC §101
The 35 U.S.C. 101 rejections of claims 1-19 have been withdrawn in response to applicant's amendments.
Claim Rejections - 35 USC §102
In response to the applicant's amendments , the 35 U.S.C. 102 rejections of claims 1,16-17 and 19 have been withdrawn. However, Examiner relies on a combination of references.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5, 9 and 16-18 are rejected under 35 U.S.C. 103 as being unpatentable over Revach (US 2025/0117381) in view of Thompson IV (US 2025/0046409 Al)
Regarding claim 1, Revach discloses: A method, comprising: receiving, by one or more processors, input specifying a set of search criteria using natural language text; (Revach, [0095], e.g. in response to receiving the NL based input, determining, based on one or more criteria, whether to generate and execute multiple subqueries based on the NL based input. …the one or more criteria can additionally or alternatively include one or more search criteria; [0007], e.g. a NL based input ( e.g., a textual query submitted via a search system interface)
executing, by the one or more processors, one or more searches based on the set of search criteria specified in the input, the one or more searches comprising a search of at least one data source; (Revach, [0055] If, at block 354, the system determines the criteria are not satisfied, the system proceeds to block 356 and a search is performed based on the NL based input; [0012] search system resources (corresponding to “at least one data source”) are utilized; [0044] The search result engine 130, for each of the subqueries of the subset selected by subset selection engine 128, interacts with search system(s) 140 (corresponding to “at least one data source”) to obtain result(s) for the subquery)
obtaining, by the one or more processors, an initial set of search results based on the one or more searches; (Revach, Fig. 5A; [0076], e.g. an NL based input 501A has been provided…and includes two search results (A and B) for the first subquery; [0044] The search result engine 130, for each of the subqueries of the subset selected by subset selection engine 128, interacts with search system(s) 140 to obtain result(s) for the subquery)
providing, by the one or more processors, one or more prompts to one or more large language models (LLMs), (Revach, Fig. 3; [0057] At block 360, the system processes the subquery generation prompt, using an LLM, to generate LLM output; [0088], e.g. receiving natural language (NL) based input associated with a client device. The method further includes, in response to receiving the NL based input: generating a subquery generation prompt that includes the NL based input and additional NL content that promotes subquery generation;)
wherein the one or more prompts comprise information associated with the initial set of search results, the set of search criteria, or both; (Revach, Fig. 3, item 352 “IDENTIFY NL BASED INPUT”, item 354 “CRITERIA? YES”, item 358 “GENERATE SUBQUERY GENERATION PROMPT THAT INCLUDES NL BASED INPUT AND ADDITIONAL NL CONTENT THAT PROMOTES SUBQUERY GENERATION; item 360 “PROCESS SUBQUERY GENERATION PROMPT (corresponding to “prompts comprise… the set of search criteria”), USING LLM, TO GENERATE LLM OUTPUT”; [0088], e.g. receiving natural language (NL) based input associated with a client device. The method further includes, in response to receiving the NL based input: generating a subquery generation prompt that includes the NL based input and additional NL content that promotes subquery generation; [0057] At block 360, the system processes the subquery generation prompt, using an LLM, to generate LLM output.)
and outputting, by the one or more processors, a response to the input
(Revach, Fig. 3, item 352 “IDENTIFY NL BASED INPUT”, item 372 “GENERATE RESPONSE BASED ON SEARCH RESULTS)
based on content generated by the one or more LLMs,
(Fig. 3, item 372 “GENERATE RESPONSE BASED ON SEARCH RESULTS, item 362,
“GENERATE ONE OR MORE CANDIDATE SUBQUERIES BASED ON LLM OUTPUT (corresponding to ”content generated by the one or more LLMs”)
wherein the response is generated by the one or more LLMs based on the one or more prompts, (Revach, [0088], e.g. generating a response to the NL based input based on the corresponding search results for the candidate subqueries of the subset; [0010] the first subquery generation prompt can be processed, using the LLM, to generate first LLM output and the first LLM output utilized ( e.g., decoded) to determine multiple first candidate subqueries; [0016], e.g. the response can be a shortened summary of the top ranked search results, such as a shortened summary that is generated based on processing, using an LLM, each of the search results along with a summarization prompt ( e.g., "generate a summary of [ search results]").)
However Revach does not clearly disclose:
wherein the one or more LLMs are constrained to generate the response based on content of the initial set of search results, such that the response is grounded in the retrieved search results.
However Thompson IV discloses:
wherein the one or more LLMs are constrained to generate the response based on content of the initial set of search results, such that the response is grounded in the retrieved search results. (Thompson IV [0079] A query 510 is obtained (e.g., is obtained via a digital assistant or graphical user interface described herein) and snippets 512 relevant to the query 510 are retrieved from the datasets 508… the top k results are obtained and optionally ranked by the retriever component; [0080] The relevant snippets 512 are incorporated into a prompt 514 for an AI component 516; [0081] The AI component 516 provides results 518 responsive to the prompt 514. For example, the AI component 516 may identify members of a target population by analyzing the relevant snippets 512.)
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Revach with the teaching of Thompson IV to prevent hallucinations by the generative AI
component, (Thompson IV, [0125]) and also implementing a retrieval-augmented
generative (RAG) approach to identify relevant portions of text. A RAG approach proves to be more efficient and effective than providing the LLM with larger context windows, (Thompson IV, [0104]).
Regarding claim 2, Revach in view of Thompson IV discloses all of the features with respect to claim 1 as outlined above. Claim 2 further recites: wherein the initial set of search results comprise search results corresponding to different result types. (Revach, Fig. 5A, item 502A1 “SUBQUERY 1 (DERIVED FROM INPUT) - RESULT A - RESULT B, item 502AN, “SUBQUERY N (DERIVED FROM INPUT) - RESULT N”; [0008] generate a response to the NL based input based on the corresponding search results for the candidate subqueries of the subset, and cause the response to be rendered responsive to the NL based input; [0044] The search result engine 130, for each of the subqueries of the subset selected by subset selection engine 128, interacts with search system(s) 140 to obtain result(s) for the subquery. For example, the search result engine 130 can obtain a top result, the top N results, or any result(s) having a quality score (and/or other score(s)) above a threshold; [0050] The search result engine 130 interacts with search system(s) 140 to obtain, for each of the subqueries of the subset, one or more corresponding results, and provides the collective results 205 to the response engine 132;)
However Revach does not clearly disclose:
different result types
However Thompson IV discloses:
different result types ( Thompson IV [0095] the set of patients are identified from a patient database and/or a medical database (e.g., the medical databases 242 and/or 332 and/or the external database(s) I 08); [0079] snippets 512 relevant to the query 510 are retrieved from the datasets 508. In one example, the query 510 is a request to identify a target population having one or more predefined characteristics. In some embodiments, a predetermined number of snippets are retrieved; [0058]-[0059];[0077])
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Revach with the teaching of Thompson IV to prevent hallucinations by the generative AI component, (Thompson IV, [0125]) and also implementing a retrieval-augmented generative (RAG) approach to identify relevant portions of text. A RAG approach proves to be more efficient and effective than providing the LLM with larger context windows, (Thompson IV, [0104]).
Regarding claim 3, Revach in view of Thompson IV discloses all of the features with respect to claim 2 as outlined above. Claim 3 further recites: restricting a number of search results included in the initial set of search results for each of the different result types. (Revach, Fig. 5A, item 502A1 “SUBQUERY 1 (DERIVED FROM INPUT) - RESULT A - RESULT B, item 502AN, “SUBQUERY N (DERIVED FROM INPUT) - RESULT N” (corresponding to “different result types”) ; [0044] The search result engine 130, for each of the subqueries of the subset selected by subset selection engine 128, interacts with search system(s) 140 to obtain result(s) for the subquery. For example, the search result engine 130 can obtain a top result, the top N results, or any result(s) having a quality score (and/or other score(s)) above a threshold), (corresponding to “restricting a number of search results”); [0017]For example, generating and executing of multiple subqueries can occur for a given NL based input based on the given NL based input having a length that is greater than a threshold, being submitted less than a threshold frequency, and/or having results that are of low quality; [0050] The search result engine 130 interacts with search system(s) 140 to obtain, for each of the subqueries of the subset, one or more corresponding results, and provides the collective results 205 to the response engine 132; [0095], e.g. generating the subquery generation prompt, generating the plurality of candidate subqueries, selecting the subset of the candidate subqueries, obtaining the corresponding search results, generating the response to the NL based input based on the corresponding search results, and/or causing the response to be rendered at the client device responsive to the NL based input are only performed in response to determining to generate and execute the multiple subqueries based on the NL based input)
However Revach does not clearly disclose:
different result types
However Thompson IV discloses:
different result types (Thompson IV, [0095] the set of patients are identified from a patient database and/or a medical database (e.g., the medical databases 242 and/or 332 and/or the external database(s) I 08). [0079] snippets 512 relevant to the query 510 are retrieved from the datasets 508. In one example, the query 510 is a request to identify a target population having one or more predefined characteristics. In some embodiments, a predetermined number of snippets are retrieved;)
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Revach with the teaching of Thompson IV to prevent hallucinations by the generative AI component, (Thompson IV, [0125]) and also implementing a retrieval-augmented generative (RAG) approach to identify relevant portions of text. A RAG approach proves to be more efficient and effective than providing the LLM with larger context windows, (Thompson IV, [0104]).
Regarding claim 4, Revach in view of Thompson IV discloses all of the features with respect to claim 1 as outlined above. Revach does not clearly disclose:
wherein the one or more prompts comprises the input and portions of the initial set of search results identified as being relevant to the set of search criteria.
However Thompson IV discloses:
wherein the one or more prompts comprises the input and portions of the initial set of search results identified as being relevant to the set of search criteria.
(Thompson IV, [0052] prompt for the AI component ( e.g., the prompt 514) includes
retrieved patient context, inclusion/exclusion criteria, and a question to determine if the
patient satisfies the criteria; [0080] The relevant snippets 512 are incorporated into a prompt 514 for an AI component 516 …the relevant snippets 512 correspond to inclusion and/or exclusion criteria for a target population)
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Revach with the teaching of Thompson IV to prevent hallucinations by the generative AI component, (Thompson IV, [0125]) and also implementing a retrieval-augmented generative (RAG) approach to identify relevant portions of text. A RAG approach proves to be more efficient and effective than providing the LLM with larger context windows, (Thompson IV, [0104]).
Regarding claim 5, Revach in view of Thompson IV discloses all of the features with respect to claim 4 as outlined above. Claim 5 further recites: generating the response via an iterative process. (Revach, [0042] When multiple prompts are provided for an NL based input, the subquery generation engine 126 can perform multiple iterations of processing, each using the LLM and a different one of the prompts, generating multiple of the candidate subqueries based on the LLM output at each iteration; [0048] The subquery generation engine 126 performs three iterations of processing ( optionally in parallel), using the LLM, with each iteration processing a different one of the subquery generation prompts 202A, 202B, and 202C.)
Regarding claim 9, Revach in view of Thompson IV discloses all of the features with respect to claim 1 as outlined above. Revach does not clearly disclose:
identifying portions of each search result in the initial set of search results relevant to the set of search criteria.
However Thompson IV discloses:
identifying portions of each search result in the initial set of search results relevant to the set of search criteria. (Thompson IV, [0128] retriever component may
search a patient index to identify the data relevant to the task at hand… The top k results may be surfaced and potentially re-ranked to then be iteratively fed as context
in a prompt template for the AI component ; [0079] A query 510 is obtained (e.g., is obtained via a digital assistant or graphical user interface described herein) and snippets 512 relevant to the query 510 are retrieved from the datasets 508… the top k results are obtained and optionally ranked by the retriever component; [0080] The relevant snippets 512 are incorporated into a prompt 514 for an AI component 516; [0081] The AI component 516 provides results 518 responsive to the prompt 514. For example, the AI component 516 may identify members of a target population by analyzing the relevant snippets 512.)
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Revach with the teaching of Thompson IV to prevent hallucinations by the generative AI component, (Thompson IV, [0125]) and also implementing a retrieval-augmented generative (RAG) approach to identify relevant portions of text. A RAG approach proves to be more efficient and effective than providing the LLM with larger context windows, (Thompson IV, [0104]).
Regarding claim 16, Revach in view of Thompson IV discloses all of the features with respect to claim 1 as outlined above. Claim 16 further recites: wherein the response comprises a summary of one or more search results included in the initial set of search results. (Revach, [0016] the response can be a shortened summary of the top ranked search results, such as a shortened summary that is generated based on processing, using an LLM, each of the search results along with a summarization prompt ( e.g., "generate a summary of [ search results]").
Regarding claim 17, Revach in view of Thompson IV discloses all of the features with respect to claim 16 as outlined above. Claim 17 further recites: wherein the summary comprises information associated with negative treatment of at least one search result of the initial set of search results, information associated with fact patterns for at least one search result of the initial set of search results, information summarizing a portion of the initial set of search results, suggestions to expand a search based on the inputs, or a combination thereof. (Revach, [0016] the response can be a shortened summary of the top ranked search results (corresponding to “information summarizing a portion of the initial set of search results”), such as a shortened summary that is generated based on processing, using an LLM, each of the search results along with a summarization prompt ( e.g., "generate a summary of [ search results]").
Regarding claim 18, Revach in view of Thompson IV discloses all of the features with respect to claim 1 as outlined above. Revach does not clearly disclose:
training the one or more LLMs.
However Thompson IV discloses:
training the one or more LLMs. (Thompson IV, [0042] the server system 106 trains, publishes, and/or utilities one or more agents and/or language models.)
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Revach with the teaching of Thompson IV to prevent hallucinations by the generative AIcomponent, (Thompson IV, [0125]) and also implementing a retrieval-augmented generative (RAG) approach to identify relevant portions of text. A RAG approach proves to be more efficient and effective than providing the LLM with larger context windows, (Thompson IV, [0104]).
Claims 6 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Revach (US 2025/0117381) in view of Thompson IV (US 2025/0046409 Al) in view of Shao (“Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy”, (hereinafter “Shao”) )
Regarding claim 6, Revach in view of Thompson IV discloses all of the features with respect to claim 5 as outlined above. Revach in view of Thompson IV does not clearly disclose: wherein, during each iteration of the iterative process, a portion of the initial set of search results is presented to the one or more LLMs and an interim response is generated, and wherein the interim response and a next portion of the initial set of search results are provided as input to a next iteration of the iterative process until the response is output.
However Shao discloses:
wherein, during each iteration of the iterative process,
(Shao, page 3, section 3 Iterative Retrieval-Generation Synergy)
a portion of the initial set of search results is presented to the one or more LLMs
and an interim response is generated, (Shao, page 3, section 3 Iterative Retrieval-Generation Synergy, 3.1 Overview Given a question q and a retrieval corpus D = {d} where d is a paragraph, ITER-RETGEN repeats retrieval-generation for T iterations; in iteration t, we (1) leverage the generation yt−1 from the previous iteration, concatenated with q, to retrieve top-k paragraphs, and then (2) prompt an LLM M to produce an output yt, with both the retrieved paragraphs (denoted as Dyt−1||q) and q integrated into the prompt. Therefore, each iteration can be formulated as follows: yt = M(yt|prompt(Dyt−1||q,q)), ∀1 ≤ t ≤ T The last output yt will be produced as the final answer.)
and wherein the interim response and a next portion of the initial set of search results are provided as input to a next iteration of the iterative process until the response is output. (Shao, page 3, section 3 Iterative Retrieval-Generation Synergy, 3.1 Overview Given a question q and a retrieval corpus D = {d} where d is a paragraph, ITER-RETGEN repeats retrieval-generation for T iterations; in iteration t, we (1) leverage the generation yt−1 from the previous iteration, concatenated with q, to retrieve top-k paragraphs, and then (2) prompt an LLM M to produce an output yt, with both the retrieved paragraphs (denoted as Dyt−1||q) and q integrated into the prompt. Therefore, each iteration can be formulated as follows: yt = M(yt|prompt(Dyt−1||q,q)), ∀1 ≤ t ≤ T The last output yt will be produced as the final answer.
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Revach in view of Thompson IV with the teaching of Shao to improve relevance modeling by having large language models actively involved in retrieval, i.e., to improve retrieval with generation that it can flexibly leverage parametric knowledge and non-parametric knowledge, and is superior to or competitive with state-of-the-art retrieval-augmented baselines while causing fewer overheads of retrieval and generation and also to improve performance via generation augmented retrieval adaptation, (Shao, abstract).
Regarding claim 10, Revach in view of Thompson IV discloses all of the features with respect to claim 9 as outlined above. Claim 10 further recites: ranking or re-ranking each portion of the initial set of search results (Revach, [0016] Implementations obtain search results for only subqueries of the selected subset, and generate a response, to the NL based input, based on those search results. For example, top ranked search result(s) for each subquery can be obtained, and the response can be generated based on the top ranked search results for the subqueries.
Revach in view of Thompson IV does not clearly disclose:
ranking or re-ranking each portion of the initial set of search results identified as relevant to the set of search criteria.
However Shao discloses:
ranking or re-ranking each portion of the initial set of search results identified as relevant to the set of search criteria. (Shao, page 3, section 3 Iterative Retrieval-Generation Synergy, 3.1 Overview Given a question q and a retrieval corpus D = {d} where d is a paragraph, ITER-RETGEN repeats retrieval-generation for T iterations; in iteration t, we (1) leverage the generation yt−1 from the previous iteration, concatenated with q, to retrieve top-k paragraphs, and then (2) prompt an LLM M to produce an output yt, with both the retrieved paragraphs (denoted as Dyt−1||q) and q integrated into the prompt. Therefore, each iteration can be formulated as follows: yt = M(yt|prompt(Dyt−1||q,q)), ∀1 ≤ t ≤ T The last output yt will be produced as the final answer; page 4, section 3.4 , e.g. Re-ranker A re-ranker, parametrized by ϕ, outputs the probability of a paragraph being relevant to a query; we denote the probability as sϕ(q, d).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Revach in view of Thompson IV with the teaching of Shao to improve relevance modeling by having large language models actively involved in retrieval, i.e., to improve retrieval with generation that it can flexibly leverage parametric knowledge and non-parametric knowledge, and is superior to or competitive with state-of-the-art retrieval-augmented baselines while causing fewer overheads of retrieval and generation and also to improve performance via generation augmented retrieval adaptation, (Shao, abstract).
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Revach (US 2025/0117381) in view of Thompson IV (US 2025/0046409 Al) in view of Zangrilli (US 2025/0117863 Al)
Regarding claim 7, Revach in view of Thompson IV discloses all of the features with respect to claim 1 as outlined above. Revach in view of Thompson IV does not clearly disclose: outputting a question to the user, wherein the question is configured to obtain additional information related to the set of search criteria, and wherein the response is updated based on information received in response to the question.
However Zangrilli discloses:
outputting a question to the user, wherein the question is configured to obtain additional information related to the set of search criteria, and wherein the response is updated based on information received in response to the question. (Zangrilli [0032], e.g. the GLM 28 displayed a previous response 52B to ask if there was a specific question regarding the previous prompt 52A; [0017], e.g. a user requesting the system to generate a verbose tax category description can revise or fine tune the description through the use of multiple prompts in an interaction session, with later prompts building on or revising the output generated in response to earlier prompts.)
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Revach in view of Thompson IV with the teaching of Zangrilli to revise or fine tune the description through the use of multiple prompts in an interaction session, with later prompts building on or revising the output generated in response to earlier prompts (Zangrilli, [0017])
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Revach (US 2025/0117381) in view of Thompson IV (US 2025/0046409 Al) in view of Mukherjee (US 2024/0354436 Al)
Regarding claim 8, Revach in view of Thompson IV discloses all of the features with respect to claim 1 as outlined above. Revach in view of Thompson IV does not clearly disclose: identifying at least a portion of the initial set of search results based on outputs of a clustering algorithm.
However Mukherjee discloses:
identifying at least a portion of the initial set of search results based on outputs of a clustering algorithm. (Mukherjee [0036], e.g. the system may identify relevant portions of a set of documents based on the user query through chunking and vectorizing documents and executing similarity search on documents; [0041], e.g. The system may execute the similarity search using one of the cosine similarity search, approximate nearing neighbor (ANN) algorithms, k nearest neighbors (KNN) method, locality sensitive hashing (LSH), range queries, or any other vector clustering and/or similarity search algorithms.)
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Revach in view of Thompson IV with the teaching of Mukherjee to advantageously generate a prompt for the LLMs using a user query and portions of a set of documents that are more relevant or bear similarity to the user query, rather than including the set of documents in its entirety that might exceed a size limit on the prompt into the prompt. (Mukherjee, [0013])
Claims 11-13 are rejected under 35 U.S.C. 103 as being unpatentable over Revach (US 2025/0117381) in view of Thompson IV (US 2025/0046409 Al) in view of Khosla (US 2025/0005058 Al )
Regarding claim 11, Revach in view of Thompson IV discloses all of the features with respect to claim 1 as outlined above. Revach in view of Thompson IV does not clearly disclose:
evaluating an accuracy of the response to the set of search criteria.
However Khosla discloses:
evaluating an accuracy of the response to the set of search criteria. (Khosla, fig. 3, item 8) DETERMINE IF ANSWER(S) ARE HALLUCINATED; [0014], e.g. the natural language question answer service can utilize a verifier to verify the answer generated by the LLM to ensure it was not generated in error ( e.g., hallucinated). The verifier may utilize one or more modules to ensure the answer was not generated in error; [0066] At (8), the verifier component 108 determines if the answer was generated in error ( e.g., hallucinated). As stated above, the verifier component 108 may look for textual overlap between an answer and retrieved passages, determine whether there is a contradiction between the answers and the retrieved passages, use head/tail/relational triples to confirm faithfulness, use membership inference attacks techniques to confirm whether a question (e.g., or similar) is in a dataset, and/or a score of any of the four combined)
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Revach in view of Thompson IV with the teaching of Khosla to determine whether the answer generated from the LLM is not generated in error in relation to the natural language question by using head, tail, and relation triples (Khosla, [0014]) and also to provide reference links and titles to the retrieved passages used by the LLM component (e.g., retrieved passages used as context to generate the answer), which may allow the submitter of the question to get more details on the referenced passages, (Khosla, [0067]).
Regarding claim 12, Revach in view of Thompson IV in view of Khosla discloses all of the features with respect to claim 11 as outlined above. Revach in view of Thompson IV does not clearly disclose:
enhancing the response based at least in part on the evaluating.
However Khosla discloses:
enhancing the response based at least in part on the evaluating. (Khosla [0068] At (11), the watermarking component 110 adds patterns to the answer to make the answer proprietary to the natural language question answering service 102 and verifiable against subsequent copying; fig. 3, item 8) DETERMINE IF ANSWER(S) ARE HALLUCINATED; [0014], e.g. the natural language question answer service can utilize a verifier to verify the answer generated by the LLM to ensure it was not generated in error ( e.g., hallucinated). The verifier may utilize one or more modules to ensure the answer was not generated in error;)
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Revach in view of Thompson IV with the teaching of Khosla to determine whether the answer generated from the LLM is not generated in error in relation to the natural language question by using head, tail, and relation triples (Khosla, [0014]) and also to provide reference links and titles to the retrieved passages used by the LLM component (e.g., retrieved passages used as context to generate the answer), which may allow the submitter of the question to get more details on the referenced passages, (Khosla, [0067]).
Regarding claim 13, Revach in view of Thompson IV in view of Khosla discloses all of the features with respect to claim 12 as outlined above. Revach in view of Thompson IV does not clearly disclose:
wherein enhancing the response comprises determining one or more authorities to cite in the response, detecting negative treatment of one or more results included in the initial set of search results, altering a format of the response, incorporating treatment information into the response, or a combination thereof.
However Khosla discloses:
wherein enhancing the response comprises determining one or more authorities to cite in the response, detecting negative treatment of one or more results included in the initial set of search results, altering a format of the response, incorporating treatment information into the response, or a combination thereof.
(Khosla, [0067] At (10), the attribution component 109 may provide references to the retrieved passages, inline citations to sentences of retrieved passages used in the answer, or provide similar questions to the natural language question. For example, the attribution component 109 may provide reference links and titles to the retrieved passages used by the LLM component 106 (e.g., retrieved passages used as context to generate the answer), which may allow the submitter of the question to get more details on the referenced passages; [0068] At (11), the watermarking component 110 adds patterns to the answer to make the answer proprietary to the natural language question answering service 102 and verifiable against subsequent copying.)
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Revach in view of Thompson IV with the teaching of Khosla to determine whether the answer generated from the LLM is not generated in error in relation to the natural language question by using head, tail, and relation triples (Khosla, [0014]) and also to provide reference links and titles to the retrieved passages used by the LLM component (e.g., retrieved passages used as context to generate the answer), which may allow the submitter of the question to get more details on the referenced passages, (Khosla, [0067]).
Claims 14-15 are rejected under 35 U.S.C. 103 as being unpatentable over Revach (US 2025/0117381) in view of Thompson IV (US 2025/0046409 Al) in view of Wang (US 2023/0245651 Al)
Regarding claim 14, Revach in view of Thompson IV discloses all of the features with respect to claim 1 as outlined above. Revach in view of Thompson IV does not clearly disclose:
analyzing the input to determine a suitability of the input for LLM content generation.
However Wang discloses:
analyzing the input to determine a suitability of the input for LLM content generation. (Wang, [0414] In FIG. 19, the AI system is shown evaluating the user's inputs and contextual information to determine if any additional information is required to improve the accuracy of understanding the most likely intent and objective 1900; [0416] Next, the AI system determines whether any additional information is needed 1904. If the AI system determines that the available contextual information is insufficient or the AI system is unable to determine the user's intent and objective with a reasonable level of confidence, it may request additional information again or provide alternative options for the user to choose from;)
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Revach in view of Thompson IV with the teaching of Wang to enable contextually relevant conversational interaction and also to determine an understanding of a most relevant intent and a most relevant objective which is validated by the AI system with the user until the user agrees. The validated most relevant intent and the most relevant objective is utilized to facilitate the user-centered and contextually relevant conversational interaction, (Wang, abstract).
Regarding claim 15, Revach in view of Thompson IV in view of Wang discloses all of the features with respect to claim 14 as outlined above. Revach in view of Thompson IV does not clearly disclose:
prompting the user for additional information based on the analyzing.
However Wang discloses:
prompting the user for additional information based on the analyzing. (Wang [0414] In FIG. 19, the AI system is shown evaluating the user's inputs and contextual information to determine if any additional information is required to improve the accuracy of understanding the most likely intent and objective 1900; [0416] Next, the AI system determines whether any additional information is needed 1904. If the AI system determines that the available contextual information is insufficient or the AI system is unable to determine the user's intent and objective with a reasonable level of confidence, it may request additional information again or provide alternative options for the user to choose from; [0419] If the user responds to the AI system and provides additional information, the AI system retrieves the relevant information from the appropriate sources 1909 and integrates it with the available contextual information 1910.)
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Revach in view of Thompson IV with the teaching of Wang to enable contextually relevant conversational interaction and also to determine an understanding of a most relevant intent and a most relevant objective which is validated by the AI system with the user until the user agrees. The validated most relevant intent and the most relevant objective is utilized to facilitate the user-centered and contextually relevant conversational interaction, (Wang, abstract).
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Zangrilli (US 2025/0117863 Al) in view of Thompson IV (US 2025/0046409 Al)
Regarding claim 19, Zangrilli discloses: A method comprising: receiving, by one or more processors, a set of search criteria via a graphical user interface; (Zangrilli, [0017] A user may input into the prompt interface 34 an instruction requesting a verbose tax category description (corresponding to “a set of search criteria “)… The instruction includes instruction text 40 indicating a tax category for which the verbose tax category description is requested, such as a tax category name, a tax category type, a product, and/or a jurisdiction, for example.)
providing, by the one or more processors, the set of search criteria or information derived from the set of search criteria as one or more prompts to one or more large language models (LLMs); (Zangrilli [0018], e.g. the prompt generator 48 generates a prompt 50 for the GLM (corresponding to “to one or more large language models (LLMs)”) based on at least the matching source text data 46 and the instruction text 40 (corresponding to “a set of search criteria “); [0017] A user may input into the prompt interface 34 an instruction requesting a verbose tax category description (corresponding to “a set of search criteria “)… The instruction includes instruction text 40 indicating a tax category for which the verbose tax category description is requested, such as a tax category name, a tax category type, a product, and/or a jurisdiction, for example; [0011], e.g. GLMs with large model sizes such as these, are referred to as large language models (LLMs).)
generating, by the one or more LLMs, textual content based on the one or more prompts, (Zangrilli, [0020] The prompt 50 is input to the to the GLM 36, which is configured to output a verbose tax category description 60. The verbose tax category description 60 is output and displayed as verbose tax category description text 60A in the prompt interface 34 of the GUI 38.)
wherein the textual content comprises information associated with one or more legal issues associated with the set of search criteria. (Zangrilli, [0036], e.g. storing the text data associated with the defined tax category in a legal definition database and identifying at least one governing body for the defined tax category… The text data may include at least one of jurisdictional rules, jurisdictional regulations, industry bodies, and industry standards for defining the tax category; [0020] The prompt 50 is input to the to the GLM 36, which is configured to output a verbose tax category description 60. The verbose tax category description 60 is output and displayed as verbose tax category description text 60A in the prompt interface 34 of the GUI 38.)
However Zangrilli does not clearly disclose:
wherein the one or more LLMs are constrained to generate the textual content based on content of the one or more prompts, such that the textual content is grounded in the information derived from the set of search criteria.
However Thompson IV discloses:
wherein the one or more LLMs are constrained to generate the textual content based on content of the one or more prompts, such that the textual content is grounded in the information derived from the set of search criteria. (Thompson IV [0079] A query 510 is obtained (e.g., is obtained via a digital assistant or graphical user interface described herein) and snippets 512 relevant to the query 510 are retrieved from the datasets 508… the top k results are obtained and optionally ranked by the retriever component; [0080] The relevant snippets 512 are incorporated into a prompt 514 for an AI component 516; [0081] The AI component 516 provides results 518 responsive to the prompt 514. For example, the AI component 516 may identify members of a target population by analyzing the relevant snippets 512.)
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Zangrilli with the teaching of Thompson IV to prevent hallucinations by the generative AI
component, (Thompson IV, [0125]) and also implementing a retrieval-augmented
generative (RAG) approach to identify relevant portions of text. A RAG approach proves to be more efficient and effective than providing the LLM with larger context windows, (Thompson IV, [0104]).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action. Any inquiry concerning this communication or earlier communications from the examiner should be directed to Faezeh Forouharnejad whose telephone number is (571)270-7416. The examiner can normally be reached on Mondays, Wednesdays and Thursdays.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Shah Sanjiv can be reached on (571)272-4098. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of an application may be obtained from Patent Center and the Private Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from Patent Center or Private PAIR. Status information for unpublished applications is available through Patent Center and Private PAIR to authorized users only. Should you have questions about access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free)
/F.F. /
Examiner, Art Unit 2166
/KHANH B PHAM/Primary Examiner, Art Unit 2166
Prosecution Insights