DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The amendment filed 02/18/2025 has been entered. Applicant has amended claims 4-9 and 12-13 to correct minor informalities. No claims were added or cancelled.
Specification
The lengthy specification has not been checked to the extent necessary to determine the presence of all possible minor errors. Applicant’s cooperation is requested in correcting any errors of which applicant may become aware in the specification.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-15 is/are rejected under 35 U.S.C. 102(a)(1) as being anticiapted by Dziri et al (Neural Path Hunter: Reducing Hallucination in Dialogue Systems via Path Grounding)
Dziri discloses:
Claim 1 (Original): A computer-implemented method for improving accuracy and reliability of a language model system, the method comprising: encoding a query using a language model that has been pre-trained to generate responses to text queries and that has been further trained to adhere to retrieved facts for queries relating to a domain text, the retrieved facts being a subset of text chunks of the domain text (pg 2 - the open problem of hallucination of factually invalid statements in knowledge grounded dialogue systems where the source of knowledge is a KG. We first identify prominent modes of hallucination by conducting a systematic human study on gener ated responses which reveals one major source of hallucination as the (mis)-use of wrong entities to describe factual content (Kryscinski et al., 2020), a problem that persists when naively applying lan guage models in dialogue systems; retrieving one or more relevant facts to the encoded query based on the text chunks from the domain text (pg 2 We propose NEURAL PATH HUNTER, which leverages facts supplied by a KG to reduce hallu cination in any machine-generated response.); and generating a response to the encoded query using the retrieved relevant facts. (Table 1: Aselected response based onaGPT2-KGtestresponsebeforeandafterapplying NEURAL PATH HUNTER. The span of texts highlighted in red indicate the hallucinated entity mentions whereas the ones highlighted in green indicate the retrieved correct entity mentions.)
Claim 2 (Original): The computer-implemented method according to claim 1, wherein the relevant facts are retrieved by a fact retriever trained to extract one or more of the text chunks that are known to be valid, and/or that have a smaller distance to the encoded query and/or a higher attention relative to other ones of the text chunks. (pg 4 -3 Neural Path Hunter We seek to design a dialogue refinement system capable of fixing generated utterances such that they are semantically relevant given the conversa tion history and supported within a provided KG. To do so, we introduce NEURAL PATH HUNTER (NPH) a refinement strategy that can be easily ap plied to any generated response without retraining the model. NPH is composed of two modules: A token-level hallucination critic and an entity men tion retriever. The first module flags and masks out hallucinated entities in an existing response and can be trained offline. The second module accepts masked representations identified by the critic and builds contextual representation of these proble)
Claim 3 (Original): The computer-implemented method according to claim 2, wherein the fact retriever is a neural network trained to extract the one or more text chunks having the higher attention using a Gumbel-Softmax layer. (pg 5 - KG-Entity Memory. Viewed another way, each qi can be interpreted as a relation embedding for the masked position in ¯xn+1. To effectively query Gkc, we must also represent all nodes in the same embedding space as qi and in doing so effectively build a representation of Gkc which we call KG Entity Memory. We explore two approaches to wards this goal. The first uses the final hidden layer of a pre-trained GPT2 to obtain initial em beddings for each node in Gkc 2. Our second ap proach uses CompGCN (Vashishth et al., 2020), which is a Graph Convolutional Network (Kipf and Welling, 2017) purposely built for multi-relational data.)
Claim 4 (Currently Amended): The computer-implemented method according to claim 2 wherein the fact retriever is trained using an objective that brings the retrieved facts closer to corresponding ones of the queries. (Table 1: Aselected response based onaGPT2-KGtestresponsebeforeandafterapplying NEURAL PATH HUNTER. The span of texts highlighted in red indicate the hallucinated entity mentions whereas the ones highlighted in green indicate the retrieved correct entity mentions.)
Claim 5 (Currently Amended): The computer-implemented method according to claim 1, further comprising generating an initial response to the encoded query using the language model, and using the initial response to retrieve the relevant facts. (Table 1: Aselected response based onaGPT2-KGtestresponsebeforeandafterapplying NEURAL PATH HUNTER. The span of texts highlighted in red indicate the hallucinated entity mentions whereas the ones highlighted in green indicate the retrieved correct entity mentions.)
Claim 6 (Currently Amended): The computer-implemented method according to claim 1, further comprising verifying that the generated response does not contradict one of the retrieved relevant facts. (pg – 6 Negative Candidates. We consider two different negative sampling strategies in order to compute LNCE: SANS (Ahrabian et al., 2020) and In-batch negatives. SANS selects hard negatives by lever aging the graph structure and selecting negative samples from a context entity’s k-hop subgraph (e.g. G1c). Meanwhile, In-batch-negatives consid ers the ground truth triple of each sample within a batch as a negative candidate for the other samples in the same batch. Using this approach, the number of candidates is equal to the batch size.)
Claim 7 (Currently Amended): The computer-implemented method according to claim 1, further comprising displaying to a user which of the text chunks of the domain text were used as the retrieved relevant facts for generating the response. (Table 1: Aselected response based onaGPT2-KGtestresponsebeforeandafterapplying NEURAL PATH HUNTER. The span of texts highlighted in red indicate the hallucinated entity mentions whereas the ones highlighted in green indicate the retrieved correct entity mentions.)
Claim 8 (Currently Amended): The computer-implemented method according to claim 1, further comprising concatenating the generated response and the encoded query for generating additional text or a follow-up query. (Table 1: Aselected response based onaGPT2-KGtestresponsebeforeandafterapplying NEURAL PATH HUNTER. The span of texts highlighted in red indicate the hallucinated entity mentions whereas the ones highlighted in green indicate the retrieved correct entity mentions.)
Claim 9 (Currently Amended): The computer-implemented method according to claim1, wherein the language model is trained to adhere to the retrieved facts using an objective that increases a likelihood of the generated response with a constraint and/or an external reward. (Table 1: Aselected response based onaGPT2-KGtestresponsebeforeandafterapplying NEURAL PATH HUNTER. The span of texts highlighted in red indicate the hallucinated entity mentions whereas the ones highlighted in green indicate the retrieved correct entity mentions.)
Claim 10 (Original): The computer-implemented method according to claim 9, wherein the constraint is directly incorporated into the language model to increase mutual information between the generated response and the retrieved relevant facts. (pg 7 -Q1: Identifying Hallucinations Analogous to the study conducted in §2.1, we ask humans to identify the span of text that is hallucinated w.r.t. to the given triples in 500 re sponses generated greedily from GPT2-KG. We report the average Krippendorf’s alpha coefficient to be 0.73 on the annotator responses. Table 5 outlines our results. To explore the robustness of our corruption strategies as discussed in §3.2, we fine-tune a large RoBERTa model (Liu et al., 2019a) on three different synthetic datasets: (i) RoBERTa-Extrin corresponds to the negative examples crafted using an extrinsic hallucinations, where entity mentions are first extracted using the SpaCy NER tagger (Honnibal and Montani, 2017). (ii) RoBERTa-Intrin consists of neg ative examples that contain intrinsic hallucina tions. (iii) Finally, RoBERTa-Intrin-Extrin)
Claim 11 (Original): The computer-implemented method according to claim 9, wherein the external reward is used in a reinforcement learning-based training of the language model in which a penalty is based on a distance between the generated response and the retrieved relevant facts. (pg 7 Q3: Query Generation We now investigate NPH’s ability to retrieve the correct entity using the crafted query. We present the results in Table 4 along with different ablation studies. We find that key metrics such as Hits@3 and Hits@10 are nearly saturated when using the complete NPH module with GPT2 embeddings for the KG-Entity memory. Furthermore, we notice that all retrieval metrics drop dramatically (e.g.↓ 70 Hits@1 ) when LNCE is omitted. Finally, we ob serve that SANS negatives lead to lower perplexit)
Claim 12 (Currently Amended): The computer-implemented method according to claim1, wherein the language model is trained to adhere to the retrieved facts by: encoding representations of each of the text chunks of the domain text, the queries relating to the domain text and responses from the language model; compressing the representations; retrieving the retrieved facts from the text chunks of the domain text for each of the queries using the compressed representations; and updating the language model using an objective that configures the language model to adhere to the retrieved facts. (pg 7-8 Throughout our experiments, we rely on three representative baselines for response generation: GPT2-KG, AdapterBot (Lin et al., 2020), and GPT2-KE (Madotto et al., 2020). GPT2-KG is a small pre-trained GPT2 model (Radford et al., 2019) fine-tuned on the dialogue corpus. AdapterBot uses a fixed backbone conversational model such as DialGPT (Zhang et al., 2020) and encodes multiple dialogue skills via different adapters (Houlsby et al., 2019). Both GPT2-KG and AdapterBot process inputs by concatenating D, Kn and the generated response. GPT2-KE on the other hand uses a GPT2 model trained on a knowledge-augmented training set)
Claim 13 (Currently Amended): The computer-implemented method according to claim 1 wherein the objective has the form of: argmin EstesEsjes\{si},qreQ\{qild(hi,qj) - A1d(h,qi) - Azdthi,q,) where e is the parametrization of the language model, S is the set of text chunks, si is a relevant text chunk, hi is the encoded representation of the relevant text chunk, qi is the corresponding query, s is a random chunk from the set of text chunks S, h is the encoded representation of the random chunk, q,. is a random query q,., d is the distance measure in an embedding space, E is expectation to be minimized, and A1 and A2 are hyper-parameters to be fine-tuned. (see formulas in sections 3.1-3.4)
Claims 14 and 15 are rejected using similar reasoning seen in the rejection of claim 1 due to reciting similar limitation but directed towards different statutory categories.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SAMUEL SHARPLESS whose telephone number is (571)272-1521. The examiner can normally be reached M-F 7:30 AM- 3:30 PM (ET).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ALEKSANDR KERZHNER can be reached at 571-270-1760. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/S.C.S./Examiner, Art Unit 2165
/ALEKSANDR KERZHNER/Supervisory Patent Examiner, Art Unit 2165
Prosecution Insights