DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA and is in response to communications filed on 4/14/2026 in which claims 1-20 are presented for examination.
Priority
Acknowledgment is made of Provisional Application 63/712,795.
Considerations under - 35 USC § 101
Claims are NOT directed to an abstract idea because at least the following is true:
Independent claims 1, 11 and 20 contain claims which amount to an inventive concept as found in pages 1 and 2 of the specification and represented in the claim limitations, “first prompt classifier and a second prompt classifier in parallel at inference time”, and “causing, by the device and responsive to a determination that the feedback is required, a solicitation for the feedback to be provided to the user”.
Dependent claims also recite significantly more than a judicial exception due at least to their dependency on their respective independent claims.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 6, 11, 16, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Kawasaki et al. US 20250337775 A1 (hereinafter referred to as “Kawasaki”) in view of Sun et al. US 12488184 B1 (hereinafter referred to as “Sun”).
As per claim 1, Kawasaki teaches:
A method, comprising:
classifying, by a device, a prompt from a user to a language model by a first prompt classifier and a second prompt classifier in parallel at inference time by the language model (Kawasaki, [0054] – The ensemble of prompt injection classifiers can be arranged such that two or more of the classifiers are executing in parallel);
evaluating, by the device and based on a classification of the prompt by the first prompt classifier and a classification of the prompt by the second prompt classifier, an accuracy of the classification of the prompt by the first prompt classifier (Kawasaki, [0010] – The analysis engine uses the prompt injection classifier and the intermediate result to determine a category for the prompt which is indicative of whether the prompt comprises or elicits malicious content. [0058] – The model prediction for a binary classifier can predict the confidence of the prompt injection classifier. The output of the model can take varying forms including, for example, a score closer to 1 indicating that the prompt is malicious and a score closer to 0 is indicating that the prompt is benign, wherein the prediction of confidence is interpreted as an accuracy);
Kawasaki doesn’t explicitly teach a determination of accuracy to determine if a solicitation of feedback is required, however, Sun teaches:
determining, by the device and based on the accuracy, whether feedback from the user on the classification of the prompt by the first prompt classifier is required (Sun, column 9, lines 32-41 – The system 100 may employ one or more feedback determining techniques to determine whether the user was satisfied with the system response); and
causing, by the device and responsive to a determination that the feedback is required, a solicitation for the feedback to be provided to the user (Sun, column 9, lines 50-60 – After outputting the system response, the system may cause a device to output the synthesized speech “did I answer your question,” “did I respond correctly,” or a similar natural language output requesting feedback from the user).
It would have been obvious for one of ordinary skill in the art prior to the effective filing date of the claimed invention to modify Kawasaki’s invention in view of Sun in order to include user feedback; this is advantageous because the system may determine whether an interaction was satisfactory or unsatisfactory based on explicit and/or implicit feedback corresponding to the system response (Sun, column 9, lines 50-60).
As per claim 6, Kawasaki in view of Sun teaches:
The method of claim 1, wherein the first prompt classifier includes a model trained with a loss function that penalizes when a predicted class from the first prompt classifier is different from a ground truth class (Sun, column 14, lines 50-67 – The outputs of the cross attention component 530 rep′.sub.p (where 1≤p≤m(k+1)) is then inputted to the classifier 540 to predict the importance score logits g.sub.p. Cross-entropy may be used to calculate the loss between each entity's predicted logits g.sub.p and its true label l.sub.p, and the final loss may be defined as the average of these losses. Column 28, lines 16-25 – Training a machine learning model requires establishing a “ground truth” for the training examples).
Claims 11 and 16 are directed to an apparatus performing steps recited in claims 1 and 6 with substantially the same limitations. Therefore, the rejections made to claims 1 and 6 are applied to claims 11 and 16.
Claim 20 is directed to a non-transitory, computer-readable medium performing steps recited in claim 1 with substantially the same limitations. Therefore, the rejection made to claim 1 is applied to claim 20.
Claims 2-4, 9, 12-14, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Kawasaki in view of Sun and further in view of Podogorny et al. US 11093951 B1 (hereinafter referred to as “Podogorny”).
As per claim 2, Kawasaki as modified doesn’t go into detail about quality of user feedback, however, Podgorny teaches:
The method of claim 1, further comprising:
performing quality control checks on feedback received from the user in response to the solicitation (Podgorny, column 15, lines 54-61 – Receives structured and/or unstructured feedback from one or more content generating/trusted users, wherein the feedback identifies categories, and/or topics, and/or classifications, that are relevant to the self-help content that is responsive to the query).
It would have been obvious for one of ordinary skill in the art prior to the effective filing date of the claimed invention to modify Kawasaki’s invention as modified in view of Podgorny in order to include a quality check on user feedback; this is advantageous because the system can then find relevant helpful content as feedback (Sun, column 9, lines 50-60).
As per claim 3, Kawasaki as modified with Podgorny teaches:
The method of claim 1, further comprising:
adapting the solicitation for the feedback based on attributes of prior feedback received from the user (Podgorny, column 4, lines 32-40 – The customer self-help system then provides the relevant self-help content from the identified relevant customer self-help systems to the user. Next paragraph: structured and/or unstructured feedback is then solicited and received from users of the customer self-help systems. In one embodiment, the feedback is used to verify or augment/improve topics, and/or classifications, associated with self-help content and customer self-help systems).
As per claim 4, Kawasaki as modified with Podgorny teaches:
The method of claim 1, further comprising:
providing incentives to the user to provide the feedback based on a reliability of prior feedback received from the user (Podgorny, column 4, lines 41-47 – The feedback is then used to generate one or more additional self-help relationship models to replace or update the initial self-help relationship model, and/or to update/improve labeled training data sets, wherein improvement of the model is interpreted as providing an incentive).
As per claim 9, Kawasaki as modified with Podgorny teaches:
The method of claim 1, further comprising:
intercepting feedback received from the user in response to the solicitation without intervention by the language model (Podgorny, column 4, lines 32-40 –Structured and/or unstructured feedback is then solicited and received from users of the customer self-help systems. In one embodiment, the feedback is used to verify or augment/improve topics, and/or classifications, associated with self-help content and customer self-help systems, wherein no intervention from a language model is mentioned).
Claims 12-14 and 19 are directed to an apparatus performing steps recited in claims 2-4 and 9 with substantially the same limitations. Therefore, the rejections made to claims 2-4 and 9 are applied to claims 12-14 and 19.
Claims 5, 8, 10, 15, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Kawasaki in view of Sun and further in view of Tonnetti et al. US 20200043485 A1 (hereinafter referred to as “Tonnetti”).
As per claim 5, Kawasaki as modified doesn’t teach feedback volume determination, however, Tonnetti teaches:
The method of claim 1, further comprising:
determining whether feedback from the user on the classification of the prompt by the first prompt classifier is required additionally based on a volume of previous feedback requests to the user (Tonnetti, [0014] – Improves the quality of responses from an automatic dialogue system by dynamically adjusting response thresholds. More particularly, the automatic dialogue system may dynamically determine response threshold values in response to user feedback).
It would have been obvious for one of ordinary skill in the art prior to the effective filing date of the claimed invention to modify Kawasaki’s invention as modified in view of Tonnetti in order to dynamically adjust response thresholds; this is advantageous because the system can adjust the response threshold values to provide a better user experience as the amount of user-interaction with the system increases (Tonnetti, [0014]).
As per claim 8, Kawasaki as modified with Tonnetti teaches:
The method of claim 1, further comprising:
determining the accuracy of the classification of the prompt by the first prompt classifier based on whether there is agreement between the classification of the prompt by the first prompt classifier and the classification of the prompt by the second prompt classifier (Tonnetti, [0020] – The system may subsequently receive implicit or explicit user feedback that the response indicates a misclassification of the input).
As per claim 10, Kawasaki as modified with Tonnetti teaches:
The method of claim 1, wherein the classification of the prompt by the first prompt classifier identifies features of the prompt indicative of one or more of a task requested in the prompt, a constraint applicable to completing the task, or data needed to complete the task (Tonnetti, [0045] – The automatic classification system 108 of FIG. 1 may return a user intent 110 of “GET_WEATHER_REGION” with a confidence rating 114 of 0.23 based on determined intent 110, wherein the classification determining the intent of the prompt is interpreted as determining a constraint of the prompt).
Claims 15 and 18 are directed to an apparatus performing steps recited in claims 5 and 8 with substantially the same limitations. Therefore, the rejections made to claims 5 and 8 are applied to claims 15 and 18.
Claims 7 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Kawasaki in view of Sun and further in view of Tonnetti et al. US 20200043485 A1 (hereinafter referred to as “Tonnetti”).
As per claim 7, Kawasaki as modified with Sun doesn’t explicitly teach different loss function, however, Gafni teaches:
The method of claim 6, wherein the second prompt classifier includes a model trained with a different loss function that promotes disagreement between a predicted class from the second prompt classifier and a predicted class from the first prompt classifier when it is different from the ground truth class (Gafni, [0031] – Which may predict image tokens. During the training, the transformer 250 may compare the predicted image tokens with the ground-truth image tokens 245 based on loss functions).
It would have been obvious for one of ordinary skill in the art prior to the effective filing date of the claimed invention to modify Kawasaki’s invention as modified in view of Gafni in order to include a different loss function; this is advantageous because the transformer may update itself based on the comparison thus saving time (Gafni, [0031]).
Claim 17 is directed to an apparatus performing steps recited in claim 7 with substantially the same limitations. Therefore, the rejection made to claim 7 is applied to claim 17.
Response to Arguments
Applicant’s arguments filed 4/14/2026 have been fully considered but they are not persuasive. Applicant’s arguments begin on page 8 of Remarks where there is 1 argument which is addressed below.
Argument: Applicant argues in Remarks on page 9 that the cited references don’t teach or suggest “evaluating, by the device and based on a classification of the prompt by the first prompt classifier and a classification of the prompt by the second prompt classifier, an accuracy of the classification of the prompt by the first prompt classifier” as recited in claim 1. Applicant further argues that Kawasaki is a binary prompt injection classifier when it teaches including a maliciousness score which may be closer to either 1 or 0 which is different from the claimed accuracy of the classification of the prompt.
In Response: Reference Kawasaki teaches in [0058] – The model prediction for a binary classifier can predict the confidence of the prompt injection classifier. The output of the model can take varying forms including, for example, a score closer to 1 indicating that the prompt is malicious and a score closer to 0 is indicating that the prompt is benign, wherein the prediction of confidence is interpreted as an accuracy and a malicious prompt is a type of classification of the prompt. Further search and consideration of Kawasaki also shows that a category for the prompt can be determined. [0011] teaches that categories can take varying forms. As one example, the category can specify a threat severity for the prompt (malicious, suspicious, unknown, or benign, etc.). In other variations, the category can specify a type of prompt injection attack. Pages 13, 16, of the specification describe accuracy and confidence prediction as closely associated with each other. Based on a reasonable interpretation in view of the specification, the prior art of record teaches the claimed limitation.
The reference, Sun isn’t used to teach the accuracy portion of the claims. Instead, it’s used to teach whether feedback is necessary. Therefore, the arguments against Sun don’t apply.
Applicant could clarify around the references by clarifying the difference between “accuracy” and “confidence” if there are supported differentiations of these terms in the specification.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Bai et al. 12 Apr 2022, https://arxiv.org/pdf/2204.05862, “Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback”, pgs. 4-14
Foltz et al. US 20190258903 A1 automated custom training of a scoring model which includes receiving a plurality of responses received from a plurality of students in response to providing of a prompt; identifying an evaluation model relevant to the provided prompt, which evaluation model can be a machine learning model trained to output a score relevant to at least portions of a response; generating a training indicator that provides a graphical depiction of the degree to which the identified evaluation model is trained; determining a training status of the model; receiving at least one evaluation input when the model is identified as insufficiently trained; updating training of the evaluation model based on the at least one received evaluation input; and controlling the training indicator to reflect the degree to which the evaluation model is trained subsequent to the updating of the training of the evaluation model (Abstract).
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Matthew J. Ellis whose telephone number is (571)270-3443. The examiner can normally be reached on Monday-Friday 8AM-5PM.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kavita Stanley can be reached at (571) 272-8352. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
June 1, 2026
/MATTHEW J ELLIS/Primary Examiner, Art Unit 2153
Prosecution Insights