DETAILED ACTION
Claims 1-20 of the instant application are pending and have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claims 1, 10, 15, 16 and 20 objected to because of the following informalities: the limitation “generated by the chat bot…” should read “generated by the chat bot software…”. Appropriate correction is required.
Claims 5, 10, and 20 objected to because of the following informalities: the limitation “a language model neural network…” should read “[[a]] the language model neural network…”. Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim(s) 1-20 rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. More specifically directed to the abstract idea grouping of: mental process and/or mathematical concept.
The independent claim(s) 1, 15, and 16 recite(s):
1. A method performed by one or more computers, the method comprising:
receiving an input query and a plurality of candidate responses to the input query;
receiving a response generated by chat bot software for the input query that summarizes the candidate responses to the input query; and
processing a language model input that comprises (i) the input query, (ii) the plurality of candidate responses, and (ii) the response generated by the chat bot software using a language model neural network to generate a classification output that characterizes whether the response generated by the chat bot has an error of a first error type.
15. One or more non-transitory computer storage media storing instructions that when executed by one or more computers cause the one more computers to perform operations comprising:
[the limitations as in claim 1, above].
16. A system comprising one or more computers and one or more storage devices storing instructions that when executed by one or more computers cause the one more computers to perform operations comprising:
[the limitations as in claim 1, above].
This reads on a human (e.g., mentally and/or using pen and paper):
Receiving an input request and a plurality of candidate responses to said input request;
Receiving a response from another human following a predefined set of rules to summarize the responses to the input request; and
Using a predetermined set of rules (i.e., model) to analyze the input request, the candidate responses, and the summary to generate a classification to the output (e.g., summary) regarding if there is an error or not.
This judicial exception is not integrated into a practical application because for example: claim 1 recites “one or more computers,” “chat bot software,” “language model neural network”, claim 15 recites “one or more non-transitory computer storage media” and “one or more computers”, and claim 16 recites “one or more computers,” “one or more storage devices”. As an example, in page 15, lines 10-11 of the as filed specification, it is disclosed: “Computers suitable for the execution of a computer program can be based on general or special purpose microprocessors or both, or any other kind of central processing unit.” Therefore, a general-purpose computer or computing device is described and mainly used as an application thereof. Accordingly, these additional elements do not integrate the abstract idea into a practical idea because it does not impose any meaningful limits on practicing the abstract idea.
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional elements of using a computer is listed as a general computing device as noted. The claim is not patent eligible.
With respect to claims 2 and 17, the claim(s) recite:
2 and 17. The method/system of claims 1 and 16, (the operations – claim 17) further comprising:
classifying the response as either containing an error of the first error type or not containing an error of the first error type based on the classification output.
This reads on a human (e.g., mentally and/or using pen and paper):
Classifying a response as either containing an error or not based on a predetermined set of rules.
No additional limitations are present.
With respect to claims 3 and 18, the claim(s) recite:
3 and 18. The method/system of claims 1 and 16, (the operations – claim 17) further comprising:
determining whether to deploy the chat bot software for responding to user queries based at least in part on the classification output.
This reads on a human (e.g., mentally and/or using pen and paper):
Determining whether if to apply a predetermined set of rules or not for responding to the input request.
No additional limitations are present.
With respect to claims 4 and 19, the claim(s) recite:
4 and 19. The method/system of claims 1 and 16, wherein the classification output is a confidence score that represents a likelihood that the response has an error of the first error type.
This reads on a human (e.g., mentally and/or using pen and paper):
Wherein the classification involves determining a confidence score that represents the likelihood of the response containing an error (e.g., predetermined set of rules / mathematical concept).
No additional limitations are present.
With respect to claims 5 and 20, the claim(s) recite:
5 and 20. The method/system of claims 4 and 19, wherein processing an input that comprises (i) the input query, (ii) the plurality of candidate responses, and (ii) the response generated by the chat bot software using a language model neural network to generate a classification output that characterizes whether the response generated by the chat bot has an error comprises:
processing an input that comprises (i) the input query, (ii) the plurality of candidate responses, and (ii) the response generated by the chat bot software using the language model neural network to generate a first score for a first natural language label that indicates that the response contains an error of the first error type and a second score for a second natural language label that indicates that the response does not contain an error of the first error type; and
generating the confidence score from at least the first score and the second score.
This reads on a human (e.g., mentally and/or using pen and paper):
Using a predetermined set of rules (i.e., model) to analyze the input request, the candidate responses, and the summary to generate a classification to the output (e.g., summary) regarding if there is an error or not by:
Generating scores for labels indicating if there are errors present or not
Generating a confidence score based on the scores above.
No additional limitations are present.
With respect to claim 6, the claim(s) recite:
6. The method of claim 5, wherein the confidence score is a probability and wherein generating the confidence score comprises applying a softmax function to a set of scores that includes the first score and the second score.
This reads on a human (e.g., mentally and/or using pen and paper):
Generating the confidence score by applying a predetermined set of rules (e.g., softmax function – mathematical concept)
No additional limitations are present.
With respect to claim 7, the claim(s) recite:
7. The method of claim 1, wherein the first error type is a hallucination error that occurs when the response generated by the chat bot software references a candidate response that was not included in the plurality of candidate responses.
This reads on a human (e.g., mentally and/or using pen and paper):
Following a predetermined set of rules for the determination that the error occurs when the response is not included in candidate responses.
No additional limitations are present.
With respect to claim 8, the claim(s) recite:
8. The method of claim 1, wherein the first error type is a coverage error that occurs when the response generated by the chat bot software does not reference one or more of the candidate responses that were included in the plurality of candidate responses.
This reads on a human (e.g., mentally and/or using pen and paper):
Following a predetermined set of rules for the determination that the error occurs when the response does not reference a candidate response included in the candidate responses.
No additional limitations are present.
With respect to claim 9, the claim(s) recite:
9. The method of claim 1, wherein the language model input further comprises a first prompt corresponding to the first error type.
This reads on a human (e.g., mentally and/or using pen and paper):
Wherein the predetermined set of rules comprise a request corresponding to a predefined error type.
No additional limitations are present.
With respect to claim 10, the claim(s) recite:
10. The method of claim 9, further comprising:
processing a second language model input that comprises (i) the input query, (ii) the plurality of candidate responses, (iii) the response generated by the chat bot software, and (iv) a second prompt corresponding to a second, different error type using a language model neural network to generate a second classification output that characterizes whether the response generated by the chat bot has an error of the second error type.
This reads on a human (e.g., mentally and/or using pen and paper):
Using a predetermined set of rules (i.e., model) to analyze the input request, the candidate responses, and the summary to generate a classification to the output (e.g., summary) regarding if there is an error or not.
No additional limitations are present.
With respect to claim 11, the claim(s) recite:
11. The method of claim 9, wherein the first prompt is a prompt that has been learned through prompt tuning on a training data set that includes a plurality of first training examples, each first training example comprising: (i) a training query, (ii) a plurality of candidate responses to the training query, (iii) a training response to the training query, and (iv) a ground truth label indicating whether the training response contains an error of the first type.
This reads on a human (e.g., mentally and/or using pen and paper):
Using a predetermined set of rules (i.e., model) to analyze the input request, the candidate responses, the summary and labels to generate a classification to the output (e.g., summary) regarding if there is an error or not.
No additional limitations are present.
With respect to claim 12, the claim(s) recite:
12. The method of claim 10, wherein the second prompt is a prompt that has been learned through prompt tuning on a training data set that includes a plurality of second training examples, each second training example comprising: (i) a training query, (ii) a plurality of candidate responses to the training query, (iii) a training response to the training query, and (iv) a ground truth label indicating whether the training response contains an error of the second type.
This reads on a human (e.g., mentally and/or using pen and paper):
Using a predetermined set of rules (i.e., model) to analyze the input request, the candidate responses, the summary and labels to generate a classification to the output (e.g., summary) regarding if there is an error or not.
No additional limitations are present.
With respect to claim 13, the claim(s) recite:
13. The method of claim 1, wherein the chat bot software provides responses generated by one or more large language models (LLMs) in response to user queries.
This reads on a human (e.g., mentally and/or using pen and paper):
Generating and providing responses based on predetermined set of rules in response to input request.
No additional limitations are present.
With respect to claim 14, the claim(s) recite:
14. The method of claim 1, wherein the language model input further comprises (v) text referencing the first error type.
This reads on a human (e.g., mentally and/or using pen and paper):
Wherein the text references the error type.
No additional limitations are present.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-2 and 7-17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Manakul et al. ("Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models." Proceedings of the 2023 conference on empirical methods in natural language processing. 2023. https://arxiv.org/pdf/2303.08896) and further in view of Mondlock et al. (US 20250131247 A1).
As to independent claim 1, Manakul et al. teaches:
1. A method performed by one or more computers (see ¶ [ paragraph number ]: “[ related reference language ]”), the method comprising:
receiving an input query and a plurality of candidate responses to the input query (see Figure 1 (LLM e.g., GPT-3, LLM’s passage to be evaluated at sentence-level, Stochastically-generated responses, and LLM blocks.) and ¶ 1-2 of section 5. SelfCheckGPT: “SelfCheckGPT is our proposed black-box zero resource hallucination detection scheme, which operates by comparing multiple sampled responses and measuring consistency. Notation: Let R refer to an LLM response drawn from a given user query. SelfCheckGPT draws a further N stochastic LLM response samples {S1, S2, ..., Sn, ..., SN} using the same query, and then measures the consistency between the response and the stochastic samples…”);
receiving a response generated by chat bot software for the input query (see Figure 1 and ¶ 1-2 of section 5. SelfCheckGPT citations as in limitation(s) above: Stochastically-generated responses); and
processing a language model input that comprises (i) the input query, (ii) the plurality of candidate responses, and (ii) the response generated by the chat bot software using a language model neural network to generate a classification output that characterizes whether the response generated by the chat bot has an error of a first error type (see Figure 1 and ¶ 1-2 of section 5. SelfCheckGPT citations as in limitation(s) above and further see ¶ 2 of section 5. SelfCheckGPT: “…We design SelfCheckGPT to predict the hallucination score of the i-th sentence, S(i), such that S(i) ∈ [0.0, 1.0], where S(i) → 0.0 if the i-th sentence is grounded in valid information and S(i) → 1.0 if the i-th sentence is hallucinated.”).
However, Manakul et al. does not explicitly teach, but Mondlock et al. does teach:
receiving a response generated by chat bot software for the input query that summarizes the candidate responses to the input query (see ¶ [0149]: “The computer-implemented method 700 may include at block 736 sending an augmented user query to the LLM to generate an answer. The augmented user query may be sent to the LLM service 170 and an answer may be received from the LLM service 170 by the LLM interface module 132. The augmented user query may be generated and sent by the combining the relevant information and answering the user query block 388 of the RAG pipeline 300. The augmented user query may include each of the relevant information responses, the user query, and a prompt causing the LLM to generate an answer. The prompt may cause the LLM to generate the answer by combining each of the plurality of relevant information responses into a relevant response block and summarizing the relevant response block into an answer.”);
Manakul et al. and Mondlock et al. are considered to be analogous to the claimed invention because they are in the same field of endeavor in natural language processing associated with responses generated by chat bots. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Manakul et al. to incorporate the teachings of Mondlock et al. of receiving a response generated by chat bot software for the input query that summarizes the candidate responses to the input query which provides the benefit of improving the efficiency and accuracy of large language models in responding to user queries (abstract of Mondlock et al.).
As to independent claim 15, Manakul et al. further teaches:
15. One or more non-transitory computer storage media storing instructions that when executed by one or more computers cause the one more computers to perform operations (see footnote 1: “1Code and dataset can be found on the project page at https://github.com/potsawee/selfcheckgpt.” and ¶ 11 of section 7. Experiments: “The Impact of the Number of Samples: Although sample-based methods are expected to perform better when more samples are drawn, this has higher computational costs. Thus, we investigate performance as the number of samples is varied.”) comprising:
[the limitations as in claim 1, above].
As to independent claim 16, Manakul et al. further teaches:
16. A system comprising one or more computers and one or more storage devices storing instructions that when executed by one or more computers cause the one more computers to perform operations (see footnote 1: “1Code and dataset can be found on the project page at https://github.com/potsawee/selfcheckgpt.” and ¶ 11 of section 7. Experiments: “The Impact of the Number of Samples: Although sample-based methods are expected to perform better when more samples are drawn, this has higher computational costs. Thus, we investigate performance as the number of samples is varied.”) comprising:
[the limitations as in claim 1, above].
Regarding claims 2 and 17, Manakul et al. in combination with Mondlock et al. teaches the limitations as in claims 1 and 16, above.
Manakul et al. further teaches:
2 and 17. The method/system of claims 1 and 16, (the operations – claim 17) further comprising:
classifying the response as either containing an error of the first error type or not containing an error of the first error type based on the classification output (see Figure 1 (e.g., how often is the sentence supported by the samples: LLM block: “Does {sample} support (sentence)? Answer: [Yes/No]”) and Figure 3 (Non-factual 1 (major inaccurate), Non-factual 0.5 (minor inaccurate), and Factual 0 (accurate)) and ¶ 11 of section 5. SelfCheckGPT: “5.5. SelfCheckGPT with Prompt: … Context {} Sentence: {} Is the sentence supported by the context above? Answer Yes or No:”).
Regarding claim 7, Manakul et al. in combination with Mondlock et al. teaches the limitations as in claim 1, above.
Manakul et al. further teaches:
7. The method of claim 1, wherein the first error type is a hallucination error that occurs when the response generated by the chat bot software references a candidate response that was not included in the plurality of candidate responses (see Figure 1 and ¶ 1-2 of section 5. SelfCheckGPT citations as in limitation(s) above and further last ¶ of section 1. Introduction: “…Through analysis of annotated articles generated by GPT-3, we show that SelfCheckGPT is a highly effective hallucination detection method that can even outperform greybox methods, and serves as a strong first baseline for an increasingly important problem of LLMs.” and ¶ 1-2 of 5 SelfCheckGPT: “SelfCheckGPT is our proposed black-box zero resource hallucination detection scheme, which operates by comparing multiple sampled responses and measuring consistency… We design SelfCheckGPT to predict the hallucination score of the i-th sentence, S(i), such that S(i) ∈ [0.0, 1.0], where S(i) → 0.0 if the i-th sentence is grounded in valid information and S(i) → 1.0 if the i-th sentence is hallucinated…).
Regarding claim 8, Manakul et al. in combination with Mondlock et al. teaches the limitations as in claim 1, above.
Manakul et al. further teaches:
8. The method of claim 1, wherein the first error type is a coverage error that occurs when the response generated by the chat bot software does not reference one or more of the candidate responses that were included in the plurality of candidate responses (see Figure 1 and ¶ 1-2 of section 5. SelfCheckGPT citations as in limitation(s) above and further Figure 1 (“Does {sample 1} support {sentence}?”) and ¶ 1 of subsection 5.5. SelfCheckGPT with Prompt: “Thus, we query an LLM to assess whether the i-th sentence is supported by sample Sn (as the context) using the following prompt. Context: {} Sentence: {} Is the sentence supported by the context above? Answer Yes or No”).
Regarding claim 9, Manakul et al. in combination with Mondlock et al. teaches the limitations as in claim 1, above.
Manakul et al. further teaches:
9. The method of claim 1, wherein the language model input further comprises a first prompt corresponding to the first error type (see Figure 1 and ¶ 1-2 of section 5. SelfCheckGPT citations as in limitation(s) above and further last ¶ of section 1. Introduction: “…By sampling multiple responses from an LLM, one can measure information consistency between the different responses and determine if statements are factual or hallucinated…”).
Regarding claim 10, Manakul et al. in combination with Mondlock et al. teaches the limitations as in claim 1, above.
Manakul et al. further teaches:
10. The method of claim 9, further comprising:
processing a second language model input that comprises (i) the input query, (ii) the plurality of candidate responses, (iii) the response generated by the chat bot software, and (iv) a second prompt corresponding to a second, different error type using a language model neural network to generate a second classification output that characterizes whether the response generated by the chat bot has an error of the second error type (see Figure 1 (No (for sample1), Yes (for sample …), No (for sample N)) and ¶ 1-2 of section 5. SelfCheckGPT citations as in claim 1, above and further see ¶ 2 of section 5. SelfCheckGPT: “…We design SelfCheckGPT to predict the hallucination score of the i-th sentence, S(i), such that S(i) ∈ [0.0, 1.0], where S(i) → 0.0 if the i-th sentence is grounded in valid information and S(i) → 1.0 if the i-th sentence is hallucinated.”).
Regarding claim 11, Manakul et al. in combination with Mondlock et al. teaches the limitations as in claim 1, above.
Manakul et al. further teaches:
11. The method of claim 9, wherein the first prompt is a prompt that has been learned through prompt tuning on a training data set that includes a plurality of first training examples (see ¶ 5.3. SelfCheckGPT with n-gram: “…Consequently, we train a simple n-gram model using the samples {S1, ..., SN} as well as the main response R (which is assessed), where we note that including R can be considered as a smoothing method where the count of each token in R is increased by 1. We then compute the average of the log-probabilities of the sentence in response R, (Eq. 7) where ˜pij is the probability (of the j-th token of the i-th sentence) computed using the n-gram model.” and ¶ 5.5 SelfCheckGPT with Prompt: “… Context {} Sentence: {} Is the sentence supported by the context above? Answer Yes or No:”),
each first training example comprising: (i) a training query, (ii) a plurality of candidate responses to the training query, (iii) a training response to the training query, and (iv) a ground truth label indicating whether the training response contains an error of the first type (see ¶ 5.3. SelfCheckGPT with n-gram as in limitation above and further ¶ 6 Data and Annotation: “As, currently, there are no standard hallucination detection datasets available, we evaluate our hallucination detection approaches by 1) generating synthetic Wikipedia articles using GPT-3 on the individuals/concepts from the WikiBio dataset (Lebret et al., 2016); 2) manually annotating the factuality of the passage at a sentence level; 3) evaluating the system’s ability to detect hallucinations. WikiBio is a dataset where each input contains the first paragraph (along with tabular information) of Wikipedia articles of a specific concept. We rank the WikiBio test set in terms of paragraph length and randomly sample 238 articles from the top 20% of longest articles (to ensure no very obscure concept is selected). GPT-3 (text-davinci-003) is then used to generate Wikipedia articles on a concept, using the prompt "This is a Wikipedia passage about {concept}:". Table 1 provides the statistics of GPT-3 generated passages. #Passages #Sentences #Tokens/passage 238 1908 184.7±36.9 Table 1: The statistics of WikiBio GPT-3 dataset where the number of tokens is based on the OpenAI GPT-2 tokenizer. We then annotate the sentences of the generated passages using the guidelines shown in Figure 3 such that each sentence is classified as either: • Major Inaccurate (Non-Factual, 1): The sentence is entirely hallucinated, i.e. the sentence is unrelated to the topic. • Minor Inaccurate (Non-Factual, 0.5): The sentence consists of some non-factual information, but the sentence is related to the topic. • Accurate (Factual, 0): The information presented in the sentence is accurate. Of the 1908 annotated sentences, 761 (39.9%) of the sentences were labelled major-inaccurate, 631 (33.1%) minor-inaccurate, and 516 (27.0%) accurate. 201 sentences in the dataset had annotations from two different annotators. To obtain a single label for this subset, if both annotators agree, then the agreed label is used. However, if there is disagreement, then the worse-case label is selected (e.g., {minor inaccurate, major inaccurate} is mapped to major inaccurate).”).
Regarding claim 12, Manakul et al. in combination with Mondlock et al. teaches the limitations as in claim 1, above.
Manakul et al. further teaches:
12. The method of claim 10, wherein the second prompt is a prompt that has been learned through prompt tuning on a training data set that includes a plurality of second training examples (see ¶ 5.3. SelfCheckGPT with n-gram and and ¶ 5.5 SelfCheckGPT with Prompt citations as in claim 11, above.),
each second training example comprising: (i) a training query, (ii) a plurality of candidate responses to the training query, (iii) a training response to the training query, and (iv) a ground truth label indicating whether the training response contains an error of the second type (see ¶ 5.3. SelfCheckGPT with n-gram and further ¶ 6 Data and Annotation as in claim 11 above).
Regarding claim 13, Manakul et al. in combination with Mondlock et al. teaches the limitations as in claim 1, above.
Manakul et al. further teaches:
13. The method of claim 1, wherein the chat bot software provides responses generated by one or more large language models (LLMs) in response to user queries (see Figure 1 (LLM e.g., GPT-3) and ¶ 1 of section 7. Experiments: “…For SelfCheckGPT-Prompt, we consider both GPT-3 (which is the same LLM that is used to generate passages) as well as the newly released ChatGPT (gpt-3.5-turbo).” and ¶ 4 under section A. Models and Implementation: “LLM for Prompting: We consider two LLMs, GPT-3 (text-davinci-003) and ChatGPT (gpt-3.5-turbo) We note that during the data creation and annotation, GPT-3 (text-davinci-003) was the stateof-the-art LLM available; hence, GPT-3 was used as the main LLM generating WikiBio passages.”).
Regarding claim 14, Manakul et al. in combination with Mondlock et al. teaches the limitations as in claim 1, above.
Manakul et al. further teaches:
14. The method of claim 1, wherein the language model input further comprises (v) text referencing the first error type (see ¶ 5.5 SelfCheckGPT with Prompt: “LLMs have recently been shown to be effective in assessing information consistency between a document and its summary in zero-shot settings (Luo et al., 2023). Thus, we query an LLM to assess whether the i-th sentence is supported by sample Sn (as the context) using the following prompt. Context: {} Sentence: {} Is the sentence supported by the context above? Answer Yes or No:”).
Claims 3-5 and 18-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Manakul et al. ("Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models." Proceedings of the 2023 conference on empirical methods in natural language processing. 2023. https://arxiv.org/pdf/2303.08896) and further in view of Mondlock et al. (US 20250131247 A1) as applied to claims 1 and 16 above, and further in view of Somech et al. (US 12670327 B2).
Regarding claims 3 and 18, Manakul et al. in combination with Mondlock et al. teaches the limitations as in claims 1 and 16, above.
However, Manakul et al. in combination with Mondlock et al. do not explicitly teach, but Somech et. al. does teach:
3 and 18. The method/system of claims 1 and 16, (the operations – claim 17) further comprising:
determining whether to deploy the chat bot software for responding to user queries based at least in part on the classification output (see ¶ Col. 6, line 39 – Col. 7, line 3: “In yet another example of a technical solution, particular embodiments train or fine-tune a language model based on generating hallucination scores indicating the likelihood of hallucination. For example, generating such hallucination scores indicating a likelihood of hallucination can be a part of prompt-based learning or answer learning. Prompt-based learning (also known as “prompting”) is a training method, where, in some embodiments, users directly specify the task they want completed in natural language for a pre-trained language model to interpret and complete. This is in contrast with traditional Transformer training methods where models are first pre-trained using unlabeled data and then fine-tuned, using labelled data. In some embodiments, a prompt is essentially an instruction written in natural language for the model to execute or complete. Depending on the complexity of the task being trained for, several prompts may be used. In some embodiments, “prompt engineering” refers to a process of designing or using structured input to the model (referred to as a prompt or prompts) to cause a desired response to be generated by the model. In some embodiments, prompt engineering includes creating the best or optimal prompt, or series of prompts, for the desired user task or output. Accordingly, given a first prompt (which may include target content), if the model produces a first output with a high likelihood of hallucination, particular embodiments learn (e.g., adjust neural network weights) such that a second output (indicative of low likelihood of hallucination) is always produced when such first prompt is provided as input. In this way, at model deployment time, no output is ever produced with a high likelihood of hallucination if the first prompt (or variation thereof) is provided, thereby increasing the accuracy of the model's generative outputs.”).
Manakul et al., Mondlock et al. and Somech et al. are considered to be analogous to the claimed invention because they are in the same field of endeavor in natural language processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Manakul et al. in combination with Mondlock et al. to incorporate the teachings of Somech et al. of determining whether to deploy the chat bot software for responding to user queries based at least in part on the classification output which provides the benefit of improving accuracy relative to existing models by implementing the technical solution of determining a hallucination score indicating a likelihood of hallucination (Col. 2, lines 42-56 of Somech et al.)
Regarding claims 4 and 19, Manakul et al. in combination with Mondlock et al. teaches the limitations as in claims 1 and 16, above.
However, Manakul et al. in combination with Mondlock et al. do not explicitly teach, but Somech et. al. does teach:
4 and 19. The method/system of claims 1 and 16, wherein the classification output is a confidence score that represents a likelihood that the response has an error of the first error type (see ¶ Col. 6, line 39 – Col. 7, line 3 citation as in claim 3 and 18, above, more specifically: “In yet another example of a technical solution, particular embodiments train or fine-tune a language model based on generating hallucination scores indicating the likelihood of hallucination. For example, generating such hallucination scores indicating a likelihood of hallucination can be a part of prompt-based learning or answer learning…”).
Manakul et al., Mondlock et al. and Somech et al. are considered to be analogous to the claimed invention because they are in the same field of endeavor in natural language processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Manakul et al. in combination with Mondlock et al. to incorporate the teachings of Somech et al. of wherein the classification output is a confidence score that represents a likelihood that the response has an error of the first error type which provides the benefit of improving accuracy relative to existing models by implementing the technical solution of determining a hallucination score indicating a likelihood of hallucination (Col. 2, lines 42-56 of Somech et al.)
Regarding claims 5 and 20, Manakul et al., Mondlock et al. and Somech et al. teach the limitations as in claims 4 and 19, above.
Manakul et al. further teaches:
5 and 20. The method/system of claims 4 and 19, wherein processing an input that comprises (i) the input query, (ii) the plurality of candidate responses, and (ii) the response generated by the chat bot software using a language model neural network to generate a classification output that characterizes whether the response generated by the chat bot has an error (see Figure 1 and ¶ 1-2 of section 5. SelfCheckGPT citations as in claims 1 and 15 above and further see ¶ 2 of section 5. SelfCheckGPT: “…We design SelfCheckGPT to predict the hallucination score of the i-th sentence, S(i), such that S(i) ∈ [0.0, 1.0], where S(i) → 0.0 if the i-th sentence is grounded in valid information and S(i) → 1.0 if the i-th sentence is hallucinated.”) comprises:
processing an input that comprises (i) the input query, (ii) the plurality of candidate responses, and (ii) the response generated by the chat bot software using the language model neural network to generate a first score for a first natural language label that indicates that the response contains an error of the first error type and a second score for a second natural language label that indicates that the response does not contain an error of the first error type (see Figure 1 and ¶ 1-2 of section 5. SelfCheckGPT citations as in claims 1 and 15 above and further see ¶ 2 of section 5. SelfCheckGPT: “…We design SelfCheckGPT to predict the hallucination score of the i-th sentence, S(i), such that S(i) ∈ [0.0, 1.0], where S(i) → 0.0 if the i-th sentence is grounded in valid information and S(i) → 1.0 if the i-th sentence is hallucinated.” and ¶ 2 of subsection 5.5. SelfCheckGPT with Prompt: “…The output from prompting when comparing the i-th sentence against sample Sn is converted to score xni through the mapping {Yes: 0.0, No: 1.0, N/A: 0.5}. The final inconsistency score is then calculated as: Eq. (11) SelfCheckGPT-Prompt is illustrated in Figure 1….”); and
generating the confidence score from at least the first score and the second score (see Figure 1 and ¶ 1-2 of section 5. SelfCheckGPT citations as in claims 1 and 15 above and further ¶ 2 of section 5. SelfCheckGPT and ¶ 2 of subsection 5.5. SelfCheckGPT with Prompt citations as in limitations above and further ¶ 5 of section 6. Data and Annotation: “…Furthermore, passage-level scores are obtained by averaging the sentence-level labels in each passage. The distribution of passage-level scores is shown in Figure 4, where we observe a large peak at +1.0…”).
Claim 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Manakul et al. ("Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models." Proceedings of the 2023 conference on empirical methods in natural language processing. 2023. https://arxiv.org/pdf/2303.08896) and further in view of Mondlock et al. (US 20250131247 A1) and as applied to claims 5 above, and further in view of Ramesh et al. (US 20250164978 A1).
Regarding claim 6, Manakul et al., Mondlock et al. and Somech et al. teach the limitations as in claim 5, above.
However, Manakul et al., Mondlock et al. and Somech et al. do not explicitly teach, but Ramesh et al. does teach:
6. The method of claim 5, wherein the confidence score is a probability and wherein generating the confidence score comprises applying a softmax function to a set of scores that includes the first score and the second score (see ¶ [0022 and 0036]: “[0022] The use of a contextually enhanced LLM prevents the LLM from “hallucinating” or generating irrelevant predictions. By incorporating the data from the specific domain, the LLM's predictions become more grounded and relevant to the specific scenario, leading to more accurate and context-aware recommendations. The model architecture can ingest both structured and unstructured data, including action strategies, product descriptions, and system attributes. This capability allows the model to generate recommendations based on the synthesis of diverse data sources. In particular, the capability to process unstructured text data provides unique generalization, as the model can handle novel scenarios, strategies, and products without pre-defined labels. [0036] FIG. 5 is a diagram showing an example 500 of contextual grounding and scoring in a computing system for providing recommendations using an LLM according to certain embodiments. The score for a candidate answer can be obtained using the product of softmaxed logits. The softmax function is a normalized exponential function that converts a vector of K real numbers into a probability distribution of K possible outcomes. This process ensures that the LLM considers the context of the question when generating answers and avoids generating answers that are inconsistent or unrelated. Prompt 502 is from the labeled data generated in FIG. 4 from the verbalized data. If the system involved in the simulation is an account and the action is a sales play, the prompt may be, as an example, a combination of a description of a characteristic of the account, such as one beginning with, “The account has,” and a description of a sales play, such as one beginning with, “The proposed sales play is.” This technique provides contextually enriched input for the LLM, ensuring that it considers the proposed sales play context before generating predictions. Various actions for the simulation include action 504, action D pertaining to product A, action 506, action B pertaining to product E, and action 508, action F pertaining to product C. These prompts are provided to fine tuned LLM 510.”).
Manakul et al., Mondlock et al., Somech et al., and Ramesh et al. are considered to be analogous to the claimed invention because they are in the same field of endeavor in natural language processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Manakul et al. in combination with Mondlock et al. and Somech et al. to incorporate the teachings of Ramesh et al. of wherein the confidence score is a probability and wherein generating the confidence score comprises applying a softmax function to a set of scores that includes the first score and the second score which provides the benefit of ensuring that the LLM considers the context of the question when generating answers and avoids generating answers that are inconsistent or unrelated ([0036] of Ramesh et al.)
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Regarding factuality and/or error detection in generative AI / chat bots (pertinent to claims 1-20):
Chern et al. ("FacTool: Factuality Detection in Generative AI--A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios." arXiv preprint arXiv:2307.13528 (2023); https://arxiv.org/abs/2307.13528)
Tian et al. (Fine-tuning Language Models for Factuality; https://arxiv.org/abs/2311.08401)
Chen et al. (Unveiling the Siren’s Song: Towards Reliable Fact-Conflicting Hallucination Detection; https://arxiv.org/abs/2310.12086v1)
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Keisha Y Castillo-Torres whose telephone number is (571)272-3975. The examiner can normally be reached Monday - Friday, 9:00 am - 4:00 pm (EST).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pierre-Louis Desir can be reached at (571)272-7799. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Keisha Y. Castillo-Torres
Examiner
Art Unit 2659
/Keisha Y. Castillo-Torres/Examiner, Art Unit 2659
Prosecution Insights