Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Regarding independent claims, Applicant’s arguments, see pages 8-9, filed 3/19/2026, with respect to the rejection(s) of claim(s) 1-7,9-17,19-20 under 35 USC 102 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Durg et al (US Publication No.: 20250173330) in view of Lei et al (US Publication No.: 20250061311).
Regarding claim 7, Applicant's arguments filed 3/19/2026 have been fully considered but they are not persuasive. The applicant contends Durg et al fails to disclose the limitation due to the feedback is not used to “adjust[ing] a language model based agent based on a quantification of how critical the failure is”, specifically, Durg et al does not disclose training or adjusting a language model or updating a language model by unlearning knowledge it used to perform a task.
The examiner disagrees. The recited limitation states “using … the feedback metric to update the language model-based agent by unlearning knowledge it used to perform the first task …”. Such recited limitation does not specify how the language model-based agent is updated or adjusted nor specify training of the language model. The limitation also does not specify what occurs for the language model to unlearn knowledge it used to perform a task. Due to the breath of the recited limitation, the claimed language is interpreted as an update of the language model is performed using the feedback metric by unlearning knowledge it used to perform the first task. Durg et al discloses updating the prompt to the LLM, wherein the information in the prompt is used to perform a task by the LLM. By changing or adjusting or updating the information in the prompt, this will cause the LLM to unlearn knowledge, such as previously included information in the previous prompt when performing the first task. (paragraph 24 discloses updating the prompt to the LLM with feedback.) The changes or updates or adjustments to the information in the prompt is determined based on feedback on how critical the failure is as per paragraph 34.
Regarding claims 8,18, the applicant contends such claims are allowable at least for the reasons described regarding the independent claims. The examiner disagrees. Please see the office action below and rebuttal for independent claims above.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-7,9-17,19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Durg et al (US Publication No.: 20250173330) in view of Lei et al (US Publication No.: 20250061311).
Claim 1, Durg et al discloses
obtaining, by a device (Fig. 1), an indication of a failure (Fig. 1, label feedback processor. Paragraph 24 discloses “the feedback from the user for the answer 194 can be received by the feedback processor 140.” The user feedback can indicate negative feedback indicating failure to perform a task such as answer a user query (Fig. 1, label 110,194).) by a language model-based agent for a computer network (Fig. 1, label 100 as the language model based agent for a computer network such as chatbot interaction.) to perform a first task requested by a first prompt (Fig. 1, label 110,194, where the user query includes a first task requested by a prompt. Paragraph 23 discloses prompting an LLM to perform a task such as answer a user query.);
determining, by the device (Fig. 1) and using a classifier (Paragraph 24 discloses feedback from the user is received by a processor 140, where negative feedback is counted (classification: positive, negative, also disclosed in paragraph 34). Paragraph 34 discloses classification of the feedback: positive and negative.), a feedback metric that quantifies how critical the failure is (Paragraph 34 discloses “If the feedback from a predetermined number of users is negative indicating that the answer 194 is not responsive to the user query 110, the retrieved information/answer with the high negative feedback can be added to the prompt to increase the likelihood that the non-responsive LLM is not selected for similar user inputs that may be received in the future.” Quantifying how critical the failure is determined via the if statement “if the feedback from a predetermined number of users is negative …”.) based on one or more of (i) a type, category, or class of the first task, (ii) entities involved in the first task, or (iii) whether the first task leads to a change in the computer network (Paragraph 34 discloses determination of how critical the failure is performed based on entities involved in the first task such as feedback for a number of users receiving the feedback 194.);
identifying, by the device (Fig. 1), a subsequent prompt for the language model-based agent to perform a new task of a similar type as the first task (Fig. 1, label 110 is a user query, wherein such user query can be a subsequent request for a new task. Paragraph 24,34 discloses a prompt to the LLM (Fig. 1, label 100,192) is generated based on knowledge bases including answers with negative feedbacks in response to a future or subsequent user query.); and
adjusting, by the device (Fig. 1, label) and based on the feedback metric (paragraph 34), the subsequent prompt to avoid the language model-based agent failing the new task (Paragraph 24,34 discloses modifying or adjusting or generating a prompt to the LLM in response to a subsequent or future user query based on feedback metric (paragraph 34 as indicated above).).
fails to disclose the classifier or processor receiving feedback where feedback is classified as negative and positive (paragraphs 24,34), but fails to disclose the classifier is trained on past failures.
Lei et al discloses a classifier classifying user feedback into positive and negative (Paragraph 56 discloses a classifier-based user feedback identification model to distinguish whether the first feedback is negative feedback. The model is trained using a sample feedback data set annotated with real attribute labels such as positive and negative feedback. The sample feedback dataset indicates past or previous or feedback provided prior to user feedback under evaluation. Fig. 3, label 301 shows the user feedback model.). It would be obvious to one skilled in the art before the effective filing date of the application to modify Durg et al’s classification by incorporating a trained classifier or trained classification model, trained using past failures as disclosed by Lei et al so to improve the accuracy of determining whether feedback is positive or negative, hence improving the system’s accuracy in response to a query.
Claim 2, Durg et al discloses wherein the indication of the failure is based on user feedback regarding the first task (Paragraph 34 discloses “if the feedback from a predetermined number of users is negative indicating that the answer 194 is not responsive to the user query 110, the retrieved information/answer with the high negative feedback can be added to the prompt to increase the likelihood that the non-responsive LLM is not selected for similar user inputs that may be received in the future.” The highlighted portions emphasize the disclosure indicates a failure based on the user feedback regarding the first task.).
Claim 3, Durg et al discloses wherein the device adjusts the subsequent prompt by indicating in the subsequent prompt that a particular set of chain-of-thought steps taken by the language model-based agent was not able to successfully perform the first task (Paragraph 34 discloses if the feedback metric indicates negative feedback, the negative feedback is added to the prompt to increase likelihood that the non-responsive LLM of label 100 of Fig. 1, where the LLM includes a particular set of chain of thought steps, is not selected for future similar queries.).
Claim 4, Durg et al discloses further comprising: repeating the particular set of chain-of-thought steps in the subsequent prompt based on the feedback metric (Paragraph 34 discloses LLM with associated particular set of chain of thought steps is not likely to be selected when the prompt includes negative feedback when feedback from a predetermined number of users is negative. This indicates when a predetermined number of users providing negative feedback is not reached, the LLM with associated particular set of chain of thought steps can be selected. In this scenario, the LLM can be selected and repetition of the particular chain of thought steps associated with the LLM is performed.).
Claim 5, Durg et al discloses wherein the first task and the new task comprise troubleshooting a particular type of issue in the computer network (Fig. 1, label user query and answer is a first task and new task, wherein depending on the user query, the query can be troubleshooting a particular type of issue in the computer network such as providing a responsive answer to similar question to a previous query.).
Claim 6, Durg et al discloses wherein the indication of the failure is generated by an evaluation framework for the language model-based agent (Fig. 1, label feedback processor, paragraph 24 discloses receiving feedback from users on the language model-based agent’s answer to the user’s query.) that evaluates performance of the first task by the language model-based agent in a testing environment (Paragraph 24,34 discloses evaluation framework such as receiving user feedback and determining adjustment to an LLM prompt for future similar queries. When evaluation framework is performed, such is considered a testing environment.).
Claim 7, Durg et al discloses using, by the device, the feedback metric to update the language model-based agent by unlearning knowledge it used to perform the first task (Paragraph 34 discloses the feedback metric is used to adjust the prompt to prevent selection of the unresponsive LLM. Based on the definition of unlearning (discard: to undo the effect of: discard the habit of (https://www.merriam-webster.com/dictionary/unlearn), by preventing selection of the unresponsive LLM, the language model based agent unlearns or discards or undo the effect of using the unresponsive LLM.).
Claim 9, Durg et al discloses further comprising: determining, by the device, whether the failure is eligible to be used to adjust the subsequent prompt according to a defined policy (Paragraph 34 discloses a policy of determining feedback metric and based on the feedback metric, adjusting the subsequent prompt to include negative feedback and answer.).
Claim 10, Durg et al discloses wherein the defined policy is conditioned on an application or set of users associated with the first task (Paragraph 34 discloses the policy is determining when a predetermined number of users (set of users associated with the first task) provide negative feedback regarding a user query and response. When this occurs, the policy is to adjust the prompt to include the negative feedback and answer.).
Claim 11 recites similar limitations as found in claim 1 and is rejected on the same grounds as claim 1. In addition, Durg et al discloses
One or more network interfaces (Fig. 9, label 904);
A processor coupled to the one or more network interfaces (Fig. 9, label 902,904); and
A memory configured to store a process that is executable by the processor (Fig. 9, label 906,902, paragraph 52), the process when executed configured to perform limitations similarly recited in claim 1 (see claim 1).
Claim 12 recites similar limitations as found in claim 2 and is rejected on the same grounds as claim 2.
Claim 13 recites similar limitations as found in claim 3 and is rejected on the same grounds as claim 3.
Claim 14 recites similar limitations as found in claim 4 and is rejected on the same grounds as claim 4.
Claim 15 recites similar limitations as found in claim 5 and is rejected on the same grounds as claim 5.
Claim 16 recites similar limitations as found in claim 6 and is rejected on the same grounds as claim 6.
Claim 17 recites similar limitations as found in claim 7 and is rejected on the same grounds as claim 7.
Claim 19 recites similar limitations as found in claim 9 and is rejected on the same grounds as claim 9.
Claim 20 recites similar limitations as found in claim 1 and is rejected on the same grounds as claim 1.
Claim(s) 8,18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Durg et al (US Publication No.: 20250173330) in view of Lei et al (US Publication No.: 20250061311), further in view of Mahindru et al (US Publication No.: 20230140553).
Claim 8, Durg et al discloses the feedback metric (paragraph 34), but fails to disclose the feedback metric is based in part on how frequently the language model-based agent fails tasks of a similar type as the first task.
Mahindru et al discloses tracked metrics of user feedback (feedback metric) includes failure of chatbot (language model-based agent) to provide correct response to a same or similar customer question (task). (paragraph 34) By tracking metrics including chatbot’s failure to provide correct response to a same or similar customer question, this indicates the feedback metric based in part on frequency of failure to provide an answer to similar or same customer questions. It would be obvious to one skilled in the art before the effective filing date of the application to modify Durg et al’s feedback metric to include frequency of failure of the chatbot to correctly respond to similar or same customer questions as disclosed by Mahindru et al so to improve the model’s or chatbot’s ability to provide correct response or answer, hence improving the user’s experience with the chatbot.
Claim 18 recites similar limitations as found in claim 8 and is rejected on the same grounds as claim 8.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LINDA WONG whose telephone number is (571)272-6044. The examiner can normally be reached 9-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew C Flanders can be reached at 571-272-7516. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/LINDA WONG/Primary Examiner, Art Unit 2655
Prosecution Insights