Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
2. Claims 1-18 are presented for examination.
3. This office action is in response to the REM filed 01/20/2026.
4. Claims 1 and 17 are independent claims.
5. The office action is made Final.
Claim Rejections - 35 USC § 112
6. The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL. —The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
7. Claims 1-18 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for pre-AIA the inventor(s), at the time the application was filed, had possession of the claimed invention.
8 Claim 1 and 17 recite the limitation “the modification is based on an introspective function that analyzes prior hallucination scores, entity salience, and structural prompt components derived from prior iterations, there is no support for this language in Applicant's specification Paragraphs [0051-0056], [0078-0082]. Also, “the recursive prompt engineering selectively adjusts identified subcomponents of the initial query corresponding to entities, temporal constraints, or task objectives without full regeneration of the prompt”. there is no support for this language in applicant's specification Paragraphs [0026], [0040-0041], and [0052].
Examiner Note: per definition, an introspective function is the mental process of examining one’s own conscious thoughts, feelings, and motives to gain self-awareness and understanding. It involves looking inward to evaluate personal behaviors and mental states, serving as a foundational element of psychology, mindfulness, and personal growth.
There is no support for this language in Applicant's specification.
Dependent claims are rejected under 35 U.S.C. 112(a) due to their dependence on independent claims, carrying the same deficiencies.
9. Claims 1-18 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
10. Claims 1 and 17 recite ““the modification is based on an introspective function that analyzes prior hallucination scores, entity salience, and structural prompt components derived from prior iterations, and the recursive prompt engineering selectively adjusts identified subcomponents of the initial query corresponding to entities, temporal constraints, or task objectives without full regeneration of the prompt). One skilled in the art could not interpret the scope of this limitations in line with specification. As such, this renders the claims vague and indefinite.
Dependent claims are rejected under 35 U.S.C. 112(b) due to their dependence on independent claims, carrying the same deficiencies.
Examiner Note
11. The Examiner cites particular columns and line numbers in the references as applied to the claims below for the convenience of the Applicant(s). Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested that, in preparing responses, the Applicant fully consider the references in their entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the Examiner.
Claim Rejections - 35 USC § 103
9. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
10. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
a) A patent may not be obtained through the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made.
1. Claims 1-18 are rejected under 35 U.S.C.103 as being unpatentable over Gajek et al (US 11861320 B1) hereinafter as Gajek in view of Aggarwal et al (US 20240119220 A1) hereinafter as Aggarwal.
15. Regarding claim 1 (Currently amended), Gajek teaches A computing system: one or more processors (Fig 7, processor 201); a reference database comprising real-time content feeds and structured reference data (Fig 2, the database system 214, col 8, lines 62-67 & col 9, lines 1-12, “the database system 214 may be configured as a relational database system, a non-relational database system, or any other type of database system capable of supporting the storage and querying of information”, background, “question/answer system”, col 9, lines 40-46, the request may discuss the content of the desired email or other correspondence. “real-time content feeds”, col 29, lines 24-60, “Such databases may include, but are not limited to: public sources such as the internet, internal document databases, and external document databases”); and one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations (Fig 7, memory), the operations comprising:
receiving an initial query (Fig 1, col 5, lines 56-67, “A request to analyze a plurality of text portions based on a query is received at 102. In some embodiments, the request may be received as part of a chat session in which a text generation system automatically generates responses to text received from a client machine”. Fig 12 illustrates a hallucination detection method 1200, col 29, lines 1-3, “A request (an initial query) is received at 1202 to identify one or more hallucinations in novel text generated by a text generation model.”);
processing the initial query with an artificial intelligence (Al) model selected from an ensemble of language models to generate an output (Fig 12, col 29, lines 1-3, “A request is received at 1202 to identify one or more hallucinations in novel text generated by a text generation model.”, col 7, lines 23-24, “the text generation model 276 may be a large language model (an artificial intelligence (AI) model)”);
Gajek also implicitly teaches generating a plurality of metrics by comparing the output to the reference database according to a plurality of truthfulness and recency criterion (col 33, lines 1-29, “Fig 15 illustrates a cross-encoder modeling system that accepts as input both a query portion 1502 and a text portion 1504 and employs a number of layers of cross-linked neurons 1508 to produce a relevance score 1510 (a plurality of metrics) … The text embedding models may then be used to produce relevance scores for a number of different queries and text portions. The relevance scores may then be used to create a loss function for hyperparameter tuning of the number of layers of neurons and number of neurons per layer in a cross-encoder model. Then, the cross-encoder model may be used for future iterations without pre-training.”, Fig 12, steps 1204-1214, col 29, lines 24-60, “one or more searches may be executed against any suitable database. The one or more search results are summarized at 1212. At 1214, the factual assertion is evaluated against the one or more search results (a request to evaluate the factual assertion, information characterizing the factual assertion, and a summary of the one or more search results determined as discussed at 1212.). col 33, lines 30-48);
determining a hallucination score according to an evaluation of the plurality of metrics (Fig 12, “a hallucination detection method 1200”, step 1216, col 29, lines 61-67 & col 30, lines 1-9, “A determination is made at 1216 as to whether the factual assertion is accurate. the text generation modeling system may complete the prompt by indicating whether the factual assertion is true, false, or uncertain (score/rate) based on the provided summary of search results.”, col 3, lines 14-16, “Grounding the LLM's answers in a set of documents greatly reduces the hallucination rate.”, see more detail at col 21, lines 29-67 & col 1-50, “relevancy score”, “some references are more relevant than others, so we have assigned them relevancy scores of 1 to 5, with 1 being least relevant and 5 being most relevant. However, it's possible that some references may have been taken out of context. If a reference is missing context needed to determine whether it truly supports the response, subtract 1 point from its relevancy score. Then, rank each response from most-reliable to least-reliable, based on the adjusted relevancy scores and how well the references support the response.”); and
performing recursive prompt engineering by dynamically modifying the initial query to reduce the hallucination score wherein: the modification is based on an introspective function that analyzes prior hallucination scores, entity salience, and structural prompt components derived from prior iterations, and the recursive prompt engineering selectively adjusts identified subcomponents of the initial query corresponding to entities, temporal constraints, or task objectives without full regeneration of the prompt (col 3, lines 10-16, “techniques and mechanisms described herein can reduce or eliminate the hallucination problems to which LLMs are prone. LLMs tend to create misinformation or false knowledge when generation answers on the fly. Grounding the LLM's answers in a set of documents greatly reduces the hallucination rate.”, col 5, lines 1-9, “techniques and mechanisms described herein provide for reduced overhead associated with prompt instructions while at the same time providing for improved model context to yield an improved response.”, col 5, lines 22-35, “text chunking may reduce token overhead (initial query) and hence cost expended on large language model prompts. As yet another example, text chunking may reduce calls to a large language model, increasing response speed. As still another example, text chunking may increase and preserve context provided to a large language model by dividing text into chunks in semantically meaningful ways.”, col 14, lines 27-34, “text portions near to one another in the text itself may be assigned to the same text chunk where possible to reduce the number of divisions between semantically similar elements of the text (similarity clustering, entity extraction).”, Fig 14, steps 1408-1420, col 33, lines 1-29, “The relevance scores may then be used to create a loss function for hyperparameter tuning of the number of layers of neurons and number of neurons per layer in a cross-encoder model. Then, the cross-encoder model may be used for future iterations without pre-training.”, “The process may be repeated to iteratively bootstrap from both the bi-encoder and the cross-encoder.”, see more detail at the examples col 18, lines 37-67 & col 19, lines 1-11 and col 19, lines 50-67 & col 20, lines 1-30) an introspective function derived from prior iterations and embedded structural analysis of prompt components, and the recursive prompt engineering selectively adjusts subcomponents of the initial query without full regeneration of the prompt (col 32, lines 1-6, “A trained classification model is determined at 1316 based on the training data… the classification model may include a text embedding model that positions text in a vector space.”, col 33, lines 9-20, “one or more text embedding models may be created using a training data set. The text embedding models may then be used to produce relevance scores for a number of different queries and text portions. The relevance scores may then be used to create a loss function for hyperparameter tuning of the number of layers of neurons and number of neurons per layer in a cross-encoder model. Then, the cross-encoder model may be used for future iterations without pre-training.”).
However, for further support, Aggarwal explicitly teaches generating a plurality of metrics by comparing the output to the reference database according to a plurality of truthfulness and recency criterion ([0030], “receive the complex text and present the modified text and metrics based on the complex text and the modified text. In some cases, the metrics include faithfulness to the original complex text, readability of the modified text, and simplicity of the modified text.”, Fig 4, [0066], “Hallucinations are removed by pruning component 430 based on entailment scores.”, [0074], “metrics of modified text 520 are also presented as metrics 525. These metrics may include faithfulness to the original complex text, readability of the modified text, and simplicity of the modified text. For example, faithfulness may be measured by a combination of a semantic similarity score and a hallucination score (e.g., a degree of hallucination). Readability may be measured by one or more of the following: Automatic Readability Index (ARI), a Simple Measure of Gobbledygook (SMOG), and Flesch Kincaid. Simplicity may be measured in several ways, including dependency depth. Dependency depth is the depth(s) of dependency parse trees of sentences. A lower depth indicates sentences with less nesting of clauses and ideas and fewer long-range dependencies.” Fig 7, step 710, Fig 8, step 860, [0101], “the system additionally outputs metrics of the modified text including faithfulness, readability, and simplicity.”, [0120], “the unsupervised training generates a sequenced output from an input sequence, and evaluates the generation on metrics such as simplicity, fluency, and saliency.”);
determining a hallucination score according to an evaluation of the plurality of metrics ([0029], “compute a hallucination score between the complex text and the simplified text.”, Fig 4, “Pruning component 430”, [0066], “Pruning component 430 uses the entailment scores, hallucination scores, and semantic similarity scores to generate modified text 435.”, Fig 7, step 710, [0091], “At operation 710, the system computes an entailment score for each sentence of the simplified text using a neural network”, [0102], “computing a first hallucination score based on the complex text and the simplified text; computing a second hallucination score based on the complex text and the modified text; comparing the first hallucination score to the second hallucination score; and selecting the modified text based on the comparison.”); and
performing recursive prompt engineering by dynamically modifying the initial query to reduce the hallucination score wherein: the modification is based on an introspective function that analyzes prior hallucination scores, entity salience, and structural prompt components derived from prior iterations, and the recursive prompt engineering selectively adjusts identified subcomponents of the initial query corresponding to entities, temporal constraints, or task objectives without full regeneration of the prompt (Fig 11, [0054-0055], “pruning component 240 (recursive prompt engineering) filters sentences of the simplified text based on the entailment score to obtain a modified text. Pruning component 240 uses entailment scores from neural network 220, hallucination scores from hallucination scoring component 230, and semantic similarity scores from semantic similarity component 235 to generate a modified text with minimal hallucinations.”, [0067-0068], Fig 6, step 615, [0087], “at operation 615, the system prunes simplified text to generate modified text.”, Fig 7, step 715, [0092], “At operation 715, the system generates a modified text based on the entailment score, the simplified text, and the complex text, where the modified text includes the original information and excludes the additional information.”, Fig 8, “a method for pruning a simplified text”).
It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to incorporate the concept of teachings suggested in Aggarwal’s system into Gajek’s and by incorporating Aggarwal into Gajek because both systems are related to interactions with text generation modeling systems would provide systems and methods for text simplification that simplify text without hallucinations. (Aggarwal, [0002]).
16. Regarding claim 2, Gajek and Aggarwal teach the invention as claimed in claim 1 above and further Gajek teaches wherein the operations comprise:
rewriting the initial query to generate a new query having constrained entity references or temporal parameters (Fig 14, steps 1412, 1414, 1416, “A determination is made at 1408 as to whether the relevance score exceeds a designated threshold.”, excluding the selected text portion or including the selected text portion will generate a new query analysis); processing the new query with the AI model to generate a new output, generating a plurality of new metrics by comparing the new output to the reference database according to the plurality of criterion; and determining a new hallucination score according to an evaluation of the plurality of new metrics (same mapping as claim 1 using the new generated query, see also Fig 14, step 1418 (YES), repeat the steps 1404-1412 starting from step 1404).
Also, Aggarwal teaches the limitation at (same mapping to claim 1 applied and see [0044], “iterations over known data”, [0048], “Some examples of the transformer model are based on iterations of the transformer model such as GPT-2. In some cases, the transformer model is configured as an encoder-decoder model that receives data in sequence form and output data in sequence form, i.e. “seq2seq.” and Fig 11, [0122]).
17. Regarding claim 3, Gajek and Aggarwal teach the invention as claimed in claim 1 above and further Gajek teaches wherein the new query is generated as a function of a reduction target associated with the hallucination score (Fig 14, the new query generated from step 1418 (YES) is based on the new relevance score at step 1412).
Also, Aggarwal teaches the limitation at ([0029], [0050], [0046], “Supervised learning is a machine learning technique based on learning a function that maps an input to an output based on example input-output pairs. Supervised learning generates a function for predicting labeled data based on labeled training data consisting of a set of training examples.”).
18. Regarding claim 4, Gajek and Aggarwal teach the invention as claimed in claim 2 above and further Gajek teaches wherein the operations comprise: iteratively performing the rewriting, the processing, the generating, and the determining until the hallucination score satisfies a truthfulness threshold derived from the reference database (Fig 14, steps 1408-1420, col 33, lines 1-29, “The relevance scores may then be used to create a loss function for hyperparameter tuning of the number of layers of neurons and number of neurons per layer in a cross-encoder model. Then, the cross-encoder model may be used for future iterations without pre-training.”, “The process may be repeated to iteratively bootstrap from both the bi-encoder and the cross-encoder.”).
Also, Aggarwal teaches the limitation at ([0055], [0067-0068], [0087] and Fig 8, [0100]).
19. Regarding claim 5, Gajek and Aggarwal teach the invention as claimed in claim 2 above and further Gajek teaches wherein the hallucination score or the new hallucination score is acceptable as compared to a threshold (Fig 14, step 1412, “A determination is made at 1408 as to whether the relevance score exceeds a designated threshold.”).
Also, Aggarwal teaches the limitation at ([0055], [0067-0068], [0087] and Fig 8, [0100]).
20. Regarding claim 6, Gajek and Aggarwal teach the invention as claimed in claim 3 above and further Gajek teaches wherein the operations comprise: providing a user interface configured to receive a new query according to the hallucination score (Fig 2, Fig 7, element 711).
Also, Aggarwal teaches the limitation at ([0045], “tasks such as question answering”, [0067-0068]).
21. Regarding claim 7, Gajek and Aggarwal teach the invention as claimed in claim 6above and further Gajek teaches wherein the operations comprise: processing the new query with the AI model to generate a new output, generating a plurality of new metrics by comparing the new output to the reference database according to the plurality of criterion; and determining a new hallucination score according to an evaluation of the plurality of new metrics (same mapping as claim 2, see also FIG. 4, FIG. 8, FIG. 9, FIG. 10, and/or FIG. 11 to evaluate a response returned by the text generation modeling system.).
Also, Aggarwal teaches the limitation at (same mapping as claim 1 and see [0044], “iterations over known data”, [0048], “Some examples of the transformer model are based on iterations of the transformer model such as GPT-2. In some cases, the transformer model is configured as an encoder-decoder model that receives data in sequence form and output data in sequence form, i.e. “seq2seq.””).
22. Regarding claim 8, Gajek and Aggarwal teach the invention as claimed in claim 7 above and further Gajek teaches wherein the operations comprise: iteratively performing the rewriting, the processing, the generating, and the determining until the hallucination score or the new hallucination score is acceptable (Fig 14, steps 1408-1420, col 33, lines 1-29, “The relevance scores may then be used to create a loss function for hyperparameter tuning of the number of layers of neurons and number of neurons per layer in a cross-encoder model. Then, the cross-encoder model may be used for future iterations without pre-training.”, “The process may be repeated to iteratively bootstrap from both the bi-encoder and the cross-encoder.”).
Also, Aggarwal teaches the limitation at (same mapping as claim 1 and see [0044], “iterations over known data”, [0048], “Some examples of the transformer model are based on iterations of the transformer model such as GPT-2. In some cases, the transformer model is configured as an encoder-decoder model that receives data in sequence form and output data in sequence form, i.e. “seq2seq.” and ([0055], [0067-0068], [0087] and Fig 8, [0100]).
23. Regarding claim 9, Gajek and Aggarwal teach the invention as claimed in claim 8 above and further Gajek teaches wherein the hallucination score or the new hallucination score is acceptable as compared to a threshold (Fig 14, step 1412, “A determination is made at 1408 as to whether the relevance score exceeds a designated threshold.”).
Also, Aggarwal teaches the limitation at ([0055], [0067-0068], [0087], “at operation 615, the system prunes simplified text to generate modified text. In an example process, the system uses entailment scores to generate a new body of text, and then determines whether the new body of text is acceptable based on a hallucination score and a semantic similarity. If the new body of text passes those thresholds, then the new body of text is used as the modified text.”, and Fig 8, [0100]).
24. Regarding claim 10, Gajek and Aggarwal teach the invention as claimed in claim 1 above and further Gajek teaches wherein the operations comprise: providing a user interface configured to allow a user to change the output according to the hallucination score (Fig 7, element 711).
Also, Aggarwal teaches the limitation at ([0045], “tasks such as question answering”, [0041], “User interface 215 allows a user to input a selection or reference to a complex text, as well as optional configurable parameters”, [0053], [0067-0068]).
25. Regarding claim 11, Gajek and Aggarwal teach the invention as claimed in claim 10 above and further Gajek teaches wherein the operations comprise: generating a new query according to the hallucination score and the changed output (Fig 14, steps 1412, 1414, 1416, “A determination is made at 1408 as to whether the relevance score exceeds a designated threshold.”, excluding the selected text portion or including the selected text portion will generate a new query analysis).
Also, Aggarwal teaches the limitation at (Fig 11, [0054-0055], “pruning component 240 (recursive prompt engineering) filters sentences of the simplified text based on the entailment score to obtain a modified text. Pruning component 240 uses entailment scores from neural network 220, hallucination scores from hallucination scoring component 230, and semantic similarity scores from semantic similarity component 235 to generate a modified text with minimal hallucinations.”, [0067-0068], Fig 6, step 615, [0087], “at operation 615, the system prunes simplified text to generate modified text.”, Fig 7, step 715, [0092], “At operation 715, the system generates a modified text based on the entailment score, the simplified text, and the complex text, where the modified text includes the original information and excludes the additional information.”, Fig 8, “a method for pruning a simplified text”).
26. Regarding claim 12, Gajek and Aggarwal teach the invention as claimed in claim 11 above and further Gajek teaches wherein the operations comprise: processing the new query with the AI model to generate a new output, generating a plurality of new metrics by comparing the new output to the reference database according to the plurality of criterion; and determining a new hallucination score according to an evaluation of the plurality of new metrics (same mapping as claim 1 with new generated query).
Also, Aggarwal teaches the limitation at (same mapping as claim 1 with new generated query).
27. Regarding claim 13, Gajek and Aggarwal teach the invention as claimed in claim 12 above and further Gajek teaches wherein the operations comprise: iteratively performing the rewriting, the processing, the generating, and the determining until the hallucination score or the new hallucination score is acceptable (Fig 14, steps 1408-1420, col 33, lines 1-29, “The relevance scores may then be used to create a loss function for hyperparameter tuning of the number of layers of neurons and number of neurons per layer in a cross-encoder model. Then, the cross-encoder model may be used for future iterations without pre-training.”, “The process may be repeated to iteratively bootstrap from both the bi-encoder and the cross-encoder.”).
Also, Aggarwal teaches the limitation at (same mapping as claim 1 and see [0044], “iterations over known data”, [0048], “Some examples of the transformer model are based on iterations of the transformer model such as GPT-2. In some cases, the transformer model is configured as an encoder-decoder model that receives data in sequence form and output data in sequence form, i.e. “seq2seq.” and ([0055], [0067-0068], [0087] and Fig 8, [0100]).
28. Regarding claim 14, Gajek and Aggarwal teach the invention as claimed in claim 13 above and further Gajek teaches wherein the hallucination score or the new hallucination score is acceptable as compared to a threshold (Fig 14, step 1412, “A determination is made at 1408 as to whether the relevance score exceeds a designated threshold.”).
Also, Aggarwal teaches the limitation at ([0055], [0067-0068], [0087], “at operation 615, the system prunes simplified text to generate modified text. In an example process, the system uses entailment scores to generate a new body of text, and then determines whether the new body of text is acceptable based on a hallucination score and a semantic similarity. If the new body of text passes those thresholds, then the new body of text is used as the modified text.”, and Fig 8, [0100]).
29. Regarding claim 15, Gajek and Aggarwal teach the invention as claimed in claim 1 above and further Gajek teaches wherein the AI model is a large language model (LLM) (Fig 12, col 29, lines 1-3, “A request is received at 1202 to identify one or more hallucinations in novel text generated by a text generation model.”, col 7, lines 23-24, “the text generation model 276 may be a large language model (an artificial intelligence (AI) model)”).
Also, Aggarwal teaches the limitation at ([0045], “BERT (Bidirectional Encoder Representations from Transformers) is considered a Large Language Model (LLM)”, [0124], “Roberta-large model”).
30. Regarding claim 16, Gajek and Aggarwal teach the invention as claimed in claim 1 above and further Gajek teaches wherein the reference database comprises is a real-time content feed (Fig 12, steps 1204-1214, col 29, lines 24-60, “one or more searches may be executed against any suitable database. Such databases may include, but are not limited to: public sources such as the internet, internal document databases, and external document databases.).
Also, Aggarwal teaches the limitation at (database 105, [0031], [0035]).
31. Regarding claim 17, Gajek teaches A computing system: one or more processors (Fig 7, processor 201); a first reference database comprising real-time content feeds (Fig 2, the database system 214, col 8, lines 62-67 & col 9, lines 1-12, “the database system 214 may be configured as a relational database system, a non-relational database system, or any other type of database system capable of supporting the storage and querying of information”, background, “question/answer system”, col 9, lines 40-46, the request may discuss the content of the desired email or other correspondence. “real-time content feeds”, col 29, lines 24-60, “Such databases may include, but are not limited to: public sources such as the internet, internal document databases, and external document databases”); and one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations, the operations (Fig 7, memory) comprising:
receiving an initial query (Fig 1, col 5, lines 56-67, “A request to analyze a plurality of text portions based on a query is received at 102. In some embodiments, the request may be received as part of a chat session in which a text generation system automatically generates responses to text received from a client machine”. Fig 12 illustrates a hallucination detection method 1200, col 29, lines 1-3, “A request (an initial query) is received at 1202 to identify one or more hallucinations in novel text generated by a text generation model.”);
processing the initial query with an artificial intelligence (AI) model to generate a first output (Fig 12, col 29, lines 1-3, “A request is received at 1202 to identify one or more hallucinations in novel text generated by a text generation model.”, col 7, lines 23-24, “the text generation model 276 may be a large language model (an artificial intelligence (AI) model)”),
Gajek also implicitly teaches generating a plurality of metrics by comparing the second output to a second reference database according to a plurality of criterion (col 33, lines 1-29, “Fig 15 illustrates a cross-encoder modeling system that accepts as input both a query portion 1502 and a text portion 1504 and employs a number of layers of cross-linked neurons 1508 to produce a relevance score 1510 (a plurality of metrics) … The text embedding models may then be used to produce relevance scores for a number of different queries and text portions. The relevance scores may then be used to create a loss function for hyperparameter tuning of the number of layers of neurons and number of neurons per layer in a cross-encoder model. Then, the cross-encoder model may be used for future iterations without pre-training.”, Fig 12, steps 1204-1214, col 29, lines 24-60, “one or more searches may be executed against any suitable database. The one or more search results are summarized at 1212. At 1214, the factual assertion is evaluated against the one or more search results (a request to evaluate the factual assertion, information characterizing the factual assertion, and a summary of the one or more search results determined as discussed at 1212.). col 33, lines 30-48);
determining a hallucination score according to an evaluation of the plurality of metrics (Fig 12, “a hallucination detection method 1200”, step 1216, col 29, lines 61-67 & col 30, lines 1-9, “A determination is made at 1216 as to whether the factual assertion is accurate. the text generation modeling system may complete the prompt by indicating whether the factual assertion is true, false, or uncertain (score/rate) based on the provided summary of search results.”, col 3, lines 14-16, “Grounding the LLM's answers in a set of documents greatly reduces the hallucination rate.”, see more detail at col 21, lines 29-67 & col 1-50, “relevancy score”, “some references are more relevant than others, so we have assigned them relevancy scores of 1 to 5, with 1 being least relevant and 5 being most relevant. However, it's possible that some references may have been taken out of context. If a reference is missing context needed to determine whether it truly supports the response, subtract 1 point from its relevancy score. Then, rank each response from most-reliable to least-reliable, based on the adjusted relevancy scores and how well the references support the response.”); and
performing recursive prompt engineering by dynamically modifying the initial query to reduce the hallucination score, wherein: the modification is based on an introspective function derived from prior iterations and similarity-based validation against clustered reference records, and the recursive prompt engineering selectively adjusts subcomponents of the initial query without full regeneration of the prompt (col 3, lines 10-16, “techniques and mechanisms described herein can reduce or eliminate the hallucination problems to which LLMs are prone. LLMs tend to create misinformation or false knowledge when generation answers on the fly. Grounding the LLM's answers in a set of documents greatly reduces the hallucination rate.”, col 5, lines 1-9, “techniques and mechanisms described herein provide for reduced overhead associated with prompt instructions while at the same time providing for improved model context to yield an improved response.”, col 5, lines 22-35, “text chunking may reduce token overhead (initial query) and hence cost expended on large language model prompts. As yet another example, text chunking may reduce calls to a large language model, increasing response speed. As still another example, text chunking may increase and preserve context provided to a large language model by dividing text into chunks in semantically meaningful ways.”, col 14, lines 27-34, “text portions near to one another in the text itself may be assigned to the same text chunk where possible to reduce the number of divisions between semantically similar elements of the text (similarity clustering, entity extraction).”, Fig 14, steps 1408-1420, col 33, lines 1-29, “The relevance scores may then be used to create a loss function for hyperparameter tuning of the number of layers of neurons and number of neurons per layer in a cross-encoder model. Then, the cross-encoder model may be used for future iterations without pre-training.”, “The process may be repeated to iteratively bootstrap from both the bi-encoder and the cross-encoder.”, see more detail at the examples col 18, lines 37-67 & col 19, lines 1-11 and col 19, lines 50-67 & col 20, lines 1-30) an introspective function derived from prior iterations and embedded structural analysis of prompt components, and the recursive prompt engineering selectively adjusts subcomponents of the initial query without full regeneration of the prompt (col 32, lines 1-6, “A trained classification model is determined at 1316 based on the training data… the classification model may include a text embedding model that positions text in a vector space.”, col 33, lines 9-20, “one or more text embedding models may be created using a training data set. The text embedding models may then be used to produce relevance scores for a number of different queries and text portions. The relevance scores may then be used to create a loss function for hyperparameter tuning of the number of layers of neurons and number of neurons per layer in a cross-encoder model. Then, the cross-encoder model may be used for future iterations without pre-training.”).
However, for further support, Aggarwal explicitly teaches inputting the first output into a system which contains a large language model (LLM) to generate a second output ([0045], “BERT (Bidirectional Encoder Representations from Transformers) is considered a Large Language Model (LLM)”, [0124], “Roberta-large model”),
generating a plurality of metrics by comparing the second output to a second reference database according to a plurality of criterion ([0030], “receive the complex text and present the modified text and metrics based on the complex text and the modified text. In some cases, the metrics include faithfulness to the original complex text, readability of the modified text, and simplicity of the modified text.”, Fig 4, [0066], “Hallucinations are removed by pruning component 430 based on entailment scores.”, [0074], “metrics of modified text 520 are also presented as metrics 525. These metrics may include faithfulness to the original complex text, readability of the modified text, and simplicity of the modified text. For example, faithfulness may be measured by a combination of a semantic similarity score and a hallucination score (e.g., a degree of hallucination). Readability may be measured by one or more of the following: Automatic Readability Index (ARI), a Simple Measure of Gobbledygook (SMOG), and Flesch Kincaid. Simplicity may be measured in several ways, including dependency depth. Dependency depth is the depth(s) of dependency parse trees of sentences. A lower depth indicates sentences with less nesting of clauses and ideas and fewer long-range dependencies.” Fig 7, step 710, Fig 8, step 860, [0101], “the system additionally outputs metrics of the modified text including faithfulness, readability, and simplicity.”, [0120], “the unsupervised training generates a sequenced output from an input sequence, and evaluates the generation on metrics such as simplicity, fluency, and saliency.”),
determining a hallucination score according to an evaluation of the plurality of metrics ([0029], “compute a hallucination score between the complex text and the simplified text.”, Fig 4, “Pruning component 430”, [0066], “Pruning component 430 uses the entailment scores, hallucination scores, and semantic similarity scores to generate modified text 435.”, Fig 7, step 710, [0091], “At operation 710, the system computes an entailment score for each sentence of the simplified text using a neural network”, [0102], “computing a first hallucination score based on the complex text and the simplified text; computing a second hallucination score based on the complex text and the modified text; comparing the first hallucination score to the second hallucination score; and selecting the modified text based on the comparison.”); and
performing recursive prompt engineering by dynamically modifying the initial query to reduce the hallucination score, wherein: the modification is based on an introspective function derived from prior iterations and similarity-based validation against clustered reference records, and the recursive prompt engineering selectively adjusts subcomponents of the initial query without full regeneration of the prompt (Fig 11, [0054-0055], “pruning component 240 (recursive prompt engineering) filters sentences of the simplified text based on the entailment score to obtain a modified text. Pruning component 240 uses entailment scores from neural network 220, hallucination scores from hallucination scoring component 230, and semantic similarity scores from semantic similarity component 235 to generate a modified text with minimal hallucinations.”, [0067-0068], Fig 6, step 615, [0087], “at operation 615, the system prunes simplified text to generate modified text.”, Fig 7, step 715, [0092], “At operation 715, the system generates a modified text based on the entailment score, the simplified text, and the complex text, where the modified text includes the original information and excludes the additional information.”, Fig 8, “a method for pruning a simplified text”).
It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to incorporate the concept of teachings suggested in Aggarwal’s system into Gajek’s and by incorporating Aggarwal into Gajek because both systems are related to interactions with text generation modeling systems would provide systems and methods for text simplification that simplify text without hallucinations. (Aggarwal, [0002]).
32. Regarding claim 18, Gajek and Aggarwal teach the invention as claimed in claim 1 above and further Gajek teaches rewriting the initial query to generate a new query (Fig 14, steps 1412, 1414, 1416, “A determination is made at 1408 as to whether the relevance score exceeds a designated threshold.”, excluding the selected text portion or including the selected text portion will generate a new query analysis);
processing the new query with the AI model to generate a third output, inputting the third output into the system which contains an LLM to generate a fourth output, generating a plurality of new metrics by comparing the fourth output a third reference database according to the plurality of criterion, and determining a new hallucination score according to an evaluation of the plurality of new metrics (same mapping as claim 1 using the new generated query, see also Fig 14, step 1418 (YES), repeat the steps 1404-1412 starting from step 1404).
Also, Aggarwal teaches the limitation at (same mapping as claim 1 using the new generated query (text)).
Respond to Amendments and Arguments
33. In the remarks received 01/20/2026:
A- To advance prosecution Applicant has amended claims 1-4 and 17 to overcome the 112 rejections.
B- Applicant amended claim 1-4 and 17 to recite new features and argued that Gajek fails to teach or suggest determining a hallucination score based on comparison to real-time reference databases, performing recursive prompt engineering driven by prior hallucination metrics, or selectively modifying only subcomponents of a prompt without full regeneration. Gajek evaluates output quality but does not control or restructure prompt inputs based on introspective analysis of prior AI behavior. The claimed recursive introspection loop is absent from the cited art.
Examiner presents the following responses to Applicant’s arguments:
A- Applicant's arguments regarding 112 rejections have been fully considered but they are not persuasive. Claims 1-4 and 17 as amended does not render moot and therefore does not overcome the rejection under 35 U.S.C. 112.
B- Applicant’s arguments (see REM, filed 01/20/2026), with respect to the rejection(s) of claim(s) under 35 USC § 102 have been fully considered. However, upon further consideration, a new ground of rejection necessitated by applicant’s amendment of claims 1-20 are made under 35 USC § 103. The claims are rejected under 35 U.S.C.103 as being unpatentable over Gajek et al (US 11861320 B1) in view of Aggarwal et al (US 20240119220 A1).
CONCLUSION
34. The prior art made of record and not relied upon is considered pertinent to applicant s disclosure.
Sridhar et al (US 20250078818 A1) discloses generating and using unimodal/multimodal generative models that mitigate hallucinations.
Tunstall-Pedoe et al (US 12073180 B2) discloses a method of interacting with a large language model (LLM).
McInerney et al (US 11699026 B2) discloses methods and systems for summarizing multiple documents.
Choubey et al (US 20230119109 A1)
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HICHAM SKHOUN whose telephone number is (571)272-9466. The examiner can normally be reached Normal schedule: Mon-Fri 10am-6:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amy Ng can be reached at 5712701698. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HICHAM SKHOUN/Primary Examiner, Art Unit 2164