DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
The following title is suggested: Training a Text Question and Answer (Q & A) Model by Determining Uncertainty Degree of Predicted Answer
The abstract of the disclosure is objected to because it is not in narrative form. MPEP §608.01(b) I. C. states that an abstract should be in narrative form and should avoid a legal phraseology of patent claims. Mainly, Applicants’ abstract has a legal phraseology of a patent claim. Applicants can overcome this objection by submitting a new abstract in narrative form on a separate sheet as required by 37 CFR 1.72(b).
A corrected abstract of the disclosure is required and must be presented on a separate sheet, apart from any other text. See MPEP §608.01(b).
The disclosure is objected to because of the following informalities:
In ¶[0033], “Assuming that” should be “Assume that” to provide a complete grammatical sentence.
In ¶[0038], “loss value by may for example be” is not grammatical but could be “loss value may be for example”.
In ¶[0044], “has large difficult to predict” is not grammatical but could be “has a great difficulty in predicting”.
In ¶[0106], “configured to, determine” should be “configured to determine” because the comma is unnecessary.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1 to 2, 6, 12 to 14, 18, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Devarakonda et al. (U.S. Patent Publication 2018/0137433) in view of Pentyala et al. (U.S. Patent Publication 2024/0411991).
Concerning independent claims 1, 13, and 20, Devarakonda et al. discloses a method, apparatus, and computer program product for training a question answering system, comprising:
“at least one processor; and a memory connected in communication with the at least one processor, wherein the at least one processor is configured to:” – a computer program product may include a computer readable storage medium having computer readable program instructions thereon for causing a processor to carry out aspects of the present invention. (¶[0033]); data processing system 400 employs processing unit 406 and main memory 408 (¶[0107]: Figure 4);
“determine a sample question text set and a sample answer text corresponding to a sample question text in the sample question text set” – self-training engine trains a model based on a labeled data set comprising a finite number of question and answer pair data structures (Abstract); a method comprises training, by the self-training engine, a model based on a labeled data set comprising a finite number of question and answer pair data structures (¶[0007]); in order to ensure that all aspects of a patient's electronic medical records (EMRs) are covered by the QA system, a large set of training questions and ground truth answers (typically tens of thousands of question-answer (QA) pairs) must be generated and used to train the QA system (¶[0021]); Labeled Set (or Labeled Training Set) includes Questions that have been associated with correct answers; a <question, answer> pair in the labeled training set is <What medications does the patient take for diabetes?, a regular dose of insulin>; a label corresponding to this instance would be 1, thereby indicating that the answer correctly answers the question (¶[0023]);
“input the sample question text into a text Q&A model to be trained, and obtaining a predicted answer text output by the text Q&A model [and at least one prediction probability of at least one reference character on each character position in the predicted answer text]” – self-training engine pairs an unanswered natural language question with portions of an electronic document to generate an unlabeled data set; self-training engine applies the model to the unlabeled data set to identify an answer from the portions of the electronic document to the unanswered natural language question (Abstract); Unlabeled Set includes Questions that have not yet been associated with correct answers and thus, do not have an associated label indicating that an answer is correct or not correct for the question (¶[0023]); trained model 140 is applied to the unlabeled set U 120: the model 140 is applied to the unlabeled set U 120 of unanswered questions that are paired with all clinical notes of a patient EMR to generate confidence scores associated with each of these pairings (¶[0042]: Figure 1); here, an answer to an unanswered question generated by a trained model is “a predicted answer text output by the text Q&A model”;
“determine an uncertainty degree of the predicted answer text based on [the at least one prediction probability of at least one reference character on each character position in] the predicted answer text” – self-training process trains a model on the limited labeled data, applies the semi-trained model on the unlabeled set, and selects one or more question-answer (QA) pairs for which corresponding confidence scores are sufficiently high, e.g., equal to or greater than a predetermined threshold (¶[0025]); model 140 is applied to the unlabeled set U 120 of unanswered questions that are paired with all clinical notes of a patient EMR to generate confidence scores associated with each of these pairings, where the confidence scores indicate a confidence of the model 140 that the answer of a QA pair is a correct answer for the unanswered question of the QA pair (¶[0026]: Figure 1); a threshold value determines the confidence of the model 140 for a QA pair in the unlabeled set U 120 being classified as positive, i.e. that the answer is a correct answer for the question (¶[0042]: Figure 1); confidence scores indicate a confidence of the model 140 that the answer of a QA pair is a correct answer for the unanswered question of the QA pair; this generates, for each unanswered question, a set of answers and confidence scores associated with each of the answers (¶[0042]: Figure 1); here, a high confidence score of a correct answer to an unanswered question corresponds to “an uncertainty degree of the predicted answer”; that is, a higher confidence score implies a lower uncertainty degree;
“obtain a trained text Q&A model by performing parameter adjustment on the text Q&A model based on the sample answer text, the predicted answer text and the uncertainty degree of the predicted answer text” – a set of training questions and the corresponding answers are used to train the QA based cognitive system by modifying operational parameters of the QA based cognitive system so that it generates the correct given answer for the training question (“performing parameter adjustment on the text Q&A model”) (¶[0019]); a machine learning model 140 is trained using the labeled training set L 110; QA pairs are selected for each question based on the confidence score meeting or exceeding a threshold confidence score and thus, indicating a high confidence (¶[0042]: Figure 1); based on an identified dynamic threshold or default threshold, and the confidence scores associated with the QA pairs in the unlabeled set U 120 for the question, QA pairs that have confidence scores meeting or exceeding the threshold confidence score value are selected (¶[0043]: Figure 1); operations may be repeated until all the unanswered questions in the unlabeled set U 120 are answered and corresponding QA pairs are added to the labeled training set L 110, or the training machine learning model 140 converges, i.e. the changes to parameters of operation of the model 140 do not change above a predetermined threshold value (“performing parameter adjustment on the text Q & A model”) (¶[0044]: Figure 1); process is repeated until either all of the unanswered questions from the unlabeled set (U) 326 have been added to the labeled training set (L) 322, i.e. at least one QA pair for each unanswered question has been added to the labeled training set (L) 322, or the machine learning model 350 has reached a convergence state, e.g., a change in operational parameters of the machine learning model 350 does not meet or exceed a threshold change level, e.g., no further appreciable change in the operational parameters of the model 350 is performed (¶0101]: Figure 3); a model is trained, then, by adjusting parameters based on QA pairs that meet a confidence score threshold and answers to a set of unanswered questions (“based on the sample answer text, the predicted answer text and the uncertainty degree of the predicted answer text”).
Concerning independent claims 1, 13, and 20, Devarakonda et al. discloses all of the limitations with the exception of obtaining “at least one prediction probability of at least one reference character on each character position in the predicted answer text” and determining an uncertainty degree “based on the at least one prediction probability of at least one reference character on each character position in the predicted answer text”. Mainly, Devarakonda et al. does not consider prediction of individual characters at positions in a predicted answer based on a prediction probability. However, question answering is implicitly probabilistic in nature and these limitations appear to describe a generative question answering model. Devarakonda et al. discloses that QA pipeline generates answers for input questions by evaluating content to identify the most probable answers, i.e. candidate answers, for the input question. (¶[0083])
Concerning independent claims 1, 13, and 20, Pentyala et al. teaches training a neural network model by a causal language modeling loss computed based on a predicted next token and the ground-truth token from the training dialogue. (Abstract) The predicted next token distribution may then be compared with the ground-truth token to compute a cross-entropy loss as the causal language modeling loss 140. The causal language modeling loss 140 may then be used to update the NLP model 130 via backpropagation. (¶[0024]: Figure 1) Next token prediction may be performed progressively. Each next token prediction may be generated based on an input concatenating the document embeddings 123a-n and token embeddings of previous tokens in the token level input 112. Document embeddings and the currently decoded tokens are used to predict a next token distribution for the next position. Progressively causal language modeling loss may train the NLP model 130 to sequentially predict tokens that form a response to an input user utterance. (¶[0025]: Figure 1) Each token embedding 113 may include a combination of the token embedding 113 and a position embedding 114, indicating the position of the respective token in the training input. (¶[0032]: Figure 3B) Here, a predicted next token distribution corresponds to “at least one prediction probability of at least one reference character on each character position”. That is, a token is a ‘character’ at a position has a probability based on prior tokens. An objective is to provide a training framework for generative NLP models to generate responses with improved accuracy and factuality. (¶[0020] - ¶[0021]) It would have been obvious to one having ordinary skill in the art to perform training of a question answering model in Devarakonda et al. based on a prediction probability of at least one reference character at character positions as taught by Pentyala et al. for a purpose of training a generative model to generate responses with improved accuracy and factuality.
Concerning claims 2 and 14, Devarakonda et al. discloses:
“input the sample question text in the sample question text set into the text Q&A model to be trained” – a model is applied to the unlabeled data set to identify an answer from the portions of the electronic document to the unanswered natural language question (Abstract); based on the identified confidence scores associated with the QA pairs in the unlabeled set U 120 for the question, QA pairs that have confidence scores meeting or exceeding the threshold confidence score value are selected via process 150 (¶[0043]: Figure 1);
“obtain a plurality of candidate answer texts output by the text Q&A model” – a model is applied to the unlabeled data set to identify an answer from the portions of the electronic document to the unanswered natural language question (Abstract); a question profile may comprise a cardinality that indicates the maximum number of passages or clinical notes that likely provide a portion of the answer to the question (“a plurality of candidate answer texts”) (¶[0050]);
“determine a complexity degree of the sample question text based on the plurality of candidate answer texts” – for each question in the unlabeled set U 120, a dynamic threshold for selecting QA pairs based on their confidence scores is determined; QA pairs are selected for each question based on the confidence score meeting or exceeding a threshold confidence score and thus, indicating a high confidence (¶[0042]); a question profile may comprise a cardinality that indicates the maximum number of passages or clinical notes that likely provide a portion of the answer to the question; cardinality may be used to determine the number of QA pairs that are added to the labeled training set (¶[0050]); logic may be provided that selects a number of QA pairs corresponding to the cardinality of the question profile associated with the unanswered question in the unlabeled set U 120 that have confidence scores equal to or above the dynamically generated confidence score threshold value (¶[0051]); here, “a complexity degree of the sample question” is determined by a cardinality of a number of passages or clinical notes that answer the question, so that a question is more ‘complex’ if it has a greater number of source documents to generate an answer;
“perform a filtration processing on each sample question text in the sample question text set based on the complexity degree of the sample question text in the sample question text set” – cardinality may be used to determine the number of QA pairs that are added to the labeled training set; logic may be provided that selects a number of QA pairs corresponding to the cardinality of the question profile associated with the unanswered question in the unlabeled set U 120 that have confidence scores equal to or above the dynamically generated confidence score threshold value; logic may be provided that selects a number of QA pairs corresponding to the cardinality of the question profile associated with the unanswered question in the unlabeled set U 120 that have confidence scores equal to or above the dynamically generated confidence score threshold value; only those that are equal to or above the threshold are available for selection and thus, less than the cardinality number of QA pairs may be selected (¶[0051]); here, only selecting a limited number of QA pairs based on a cardinality of answers is “perform a filtration processing on each sample question text in the sample question text set based on the complexity degree of the sample question text in the sample question text”; that is, if there are too many answers to a question, then a question is too complex, and a question answer pair is ‘filtered out’ so that it is not used in training even if it has a sufficiently high confidence score.
Concerning claims 6 and 18, Devarakonda et al. discloses at least one of the two alternatives required by an “or” limitation:
“wherein performing the filtration processing on each sample question text in the sample question text set based on the complexity degree of the sample question text in the sample question text set comprises one of: . . . or obtaining a ranking result by ranking each sample question text in descending order based on the complexity degree of each sample question text; obtaining a preset number of first sample question texts that rank in the top of the ranking result; and filtering from the sample question text set, other sample question texts than the first sample question texts” – if there are not enough QA pairs that have a sufficiently high enough confidence score to equal or exceed the threshold, then the corresponding top highest scoring confidence score QA pairs may be selected; alternatively, in other illustrative embodiments, only those that are equal to or above the threshold are available for selection and thus, less than the cardinality number of QA pairs may be selected (¶[0051]: Figure 1); here, determining top highest scoring confidence score QA pairs for selection implies “obtaining a ranking result by ranking each sample question text in descending order”, and a top highest scoring QA pairs is “a preset number of first sample question text that rank in the top of the ranking result”; selecting only a top highest scoring confidence score QA pairs implies “filtering from the sample question text set, other sample question texts than the first sample question texts.”
Concerning claim 12, Devarakonda et al. discloses an embodiment for medical personnel asking natural language questions about a particular patient, and the healthcare cognitive system will provide answers to those questions based on its analysis of the patient's EMRs using a trained model which is trained using a self-training process in accordance with the illustrative embodiments (¶[0055]); resulting trained models 350 may then be output or otherwise provided to the cognitive system 300 which implements these models 350 in the operation of the request processing pipeline 308; models 350 may be used to analyze the input questions/requests (“obtaining a question text to be processed”), generate candidate answers/results for the input questions/requests, score the candidate answers/results, and/or select one or more final answers/results to be returned to an originator of the input question/request (“determining the answer text as an answer text corresponding to the question text”), e.g., a user of client computing device 310 or 312 (¶[0104]: Figure 3); question profile self-training engine 540 provides trained models 560 to the healthcare cognitive system 500 for implementation during runtime processing of requests 508, based on supplied patient attributes 518, to provide treatment recommendations 528; trained models 560 may be implemented by the healthcare cognitive system 500 to evaluate the various data from corpora 522, 524, and 526 to generate such treatment recommendations 528 (¶[0128]: Figure 5). Consequently, a question is input into a trained model (“inputting the question text into a text Q & A model” to generate an answer (“and obtaining an answer text output by the text Q & A model, wherein the text Q & A model is determined based on the method for training a text Q & A model”).
Claims 7 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Devarakonda et al. (U.S. Patent Publication 2018/0137433) in view of Pentyala et al. (U.S. Patent Publication 2024/0411991) as applied to claims 1 and 13 above, and further in view of Vasseur et al. (U.S. Patent Publication 2025/0086205).
Devarakonda et al. discloses determining sample answer text corresponding to sample question text but does not do this using “a teacher model” in the limitation of “for the sample question text set, obtaining answer text output by a teacher text Q & A model by inputting the sample question text into the teacher text Q & A model” and “taking the answer text output by the teacher text Q & A model as the sample text corresponding to the sample question text.” Specifically, Devarakonda et al. discloses self-training process trains a model on the limited labeled data, applies the semi-trained model on the unlabeled set, and selects one or more question-answer (QA) pairs for which corresponding confidence scores are sufficiently high, e.g., equal to or greater than a predetermined threshold, and a training process is iteratively repeated with incrementally expanding the labeled set of QA pairs until convergence. (¶[0025]) However, Vasseur et al. teaches generating a response to user input using a first language model, determining that the response is an erroneous response, generating a resolution to the erroneous response using a teacher language model, and updating the first language model using the resolution from the teacher language model. (Abstract) A ‘smaller’ language model learns from a larger language model (e.g., an LLM), whereby the larger model is only used on a subset of the queries to the smaller model that were unsuccessful/erroneous. Selective knowledge distillation allows the larger model to teach the smaller model to correct its past mistakes, thereby improving its performance over time. (¶[0061]) Vasseur et al., then, teaches a teacher language model that updates a smaller language model on only a subset of queries that were erroneous. An objective is to fine-tune a language model by enabling a larger teacher model to correct mistakes of a smaller model. (¶[0061]) It would have been obvious to one having ordinary skill in the art to provide a teacher model as taught by Vasseur et al. to train a Q & A model of Devarakonda et al. for a purpose of fine-tuning a language model by enabling a larger teacher model to correct mistakes of a smaller model.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Devarakonda et al. (U.S. Patent Publication 2018/0137433) in view of Pentyala et al. (U.S. Patent Publication 2024/0411991) as applied to claim 1 above, and further in view of Lyu et al. (U.S. Patent Publication 2023/0061398).
Devarakonda et al. discloses updating parameters to train a question answering model based on a confidence score (“uncertainty degree”), but does not disclose “determining a first loss value based on the sample answer text, the predicted answer text and a loss function of the text Q&A model”, “obtaining a second loss value by adjusting the first loss value based on the uncertainty degree of the predicted answer text”, and “obtaining the trained text Q&A model by adjusting the parameter of the text Q&A model based on the second loss value.” That is, Devarakonda et al. does not disclose training a model with first and second loss functions. However, Lyu et al. teaches training a model based on the real answer and the predicted answer that includes generating a first loss value based on the real answer and the predicted answer, generating a second loss value based on the similarity predicted value and a similarity real value between the question sample and the long rich-text document sample, and training the reading comprehension model based on the first loss value and the second loss value. (¶[0069]) A first loss value is generated based on a preset first loss function based on the similarity real value between the real answer and the predicted answer, the second loss value is generated by bringing into a preset second loss function based on the similarity predicted value and the similarity real value between the question sample and the long rich-text document sample, and related parameters of the reading model are adjusted based on the first loss function and the second loss function, to reduce the loss value, and train the reading comprehension model. (¶[0070]) An objective is to improve the accuracy of answer information generated by the reading comprehension model. (¶[0071]) It would have been obvious to one having ordinary skill in the art to train a model based on first and second loss functions as taught by Lyu et al. with a second loss function based on a confidence score (“uncertainty degree”) in Devarakonda et al. for a purpose of improving an accuracy of answer information generated by a model.
Allowable Subject Matter
Claims 3 to 5, 8 to 9, 11, and 15 to 17 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to Applicants’ disclosure.
Alkov et al., Carvalho et al., Byron et al., Bruno et al., Canada et al., Sikka et al., He et al., Zhang et al., Imanigooghari et al., and Stremmel et al. disclose related prior art.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MARTIN LERNER whose telephone number is (571) 272-7608. The examiner can normally be reached Monday-Thursday 8:30 AM-6:00 PM.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Richemond Dorvil can be reached at (571) 272-7602. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MARTIN LERNER/Primary Examiner
Art Unit 2658 July 27, 2026
Prosecution Insights