DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on February 28, 2024 has been considered by the Examiner.
Claim Objections
Claims 8, 9, 18 and 19 are objected to because of the following informalities. Appropriate correction is required.
In each of claims 8 and 18, there appears to be a typographical error in the phrase “more than the predetermined number of incorrect response.”
Claim 9 depends from claim 8 and thereby includes all of the limitations of claim 8, and is therefore objected to for the same reasons. Similarly, claim 19 depends from claim 18 and this therefore objected to for the same reasons as claim 18.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 10, 11 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over the article entitled, “Learning to Repair: Repairing model output errors after deployment using a dynamic memory of feedback” by Tandon et al. (“Tandon”), and also over the article entitled “proScript: Partially Ordered Scripts Generation” by Sakaguchi et al. (“Sakaguchi”).
Regarding claims 1, 11 and 20, Tandon generally describes an approach for enabling a large language model (LM) to continue to improve after deployment, without retraining, using feedback from a user (see e.g. the Abstract). Like claimed, Tandon particularly teaches:
deploying a foundational generative AI model, wherein the foundational generative AI model is configured to generate responses to input prompts, and wherein the foundational generative AI model is a closed loop model (Tandon discloses that the approach “pairs an LM with (i) a growing memory of cases where the user identified an output error and provided general feedback on how to correct it [and] (ii) a corrector model, trained to translate this general feedback into specific edits to repair the model output.” Abstract. Tandon further teaches that the LM is an existing fixed model such as “proScript,” which takes as input a goal to achieve and outputs a partially ordered sequence of steps required to achieve that goal:
We consider the class of problems where the model’s output is repairable, namely a structured output that is (typically) nearly correct, and fixable through a small number of edit operations. Our system is general and admits a general graph based input, so in principle it applies to a large number of tasks. In this paper, we apply our approach to the task of script generation that provides a natural setting for users to critique, and has applications in smart assistants (Zhang et al., 2021). We use an existing, fixed model: proScript (Sakaguchi et al., 2021) that satisfies the constraint of the model’s output to be repairable. proScript takes as input a goal to achieve (expressed in natural language), and outputs a partially ordered sequence of steps - a script - required to achieve that goal. Our interest here is not in proScript itself, but in what to do when proScript’s output contains an error.
(Section 1 “Introduction.” Emphasis added.).
An existing large language model like proScript is considered a deployed foundational generative AI model that is configured to generate responses, e.g. a partially ordered sequence of steps, to input prompts, e.g. a goal. As noted in the above excerpt, proScript is “an existing fixed model” and is therefore further considered a closed loop model.);
receiving, after deploying of the foundational generative AI model is complete, updated information, wherein the updated information is relevant to the generation of the responses (Tandon discloses that the approach entails training and using a system, “FBNet,” that comprises a corrector module and a memory, inter alia; training the system requires collecting examples of bad outputs from the LM, general feedback, and specific edits to the bad outputs that the feedback should translate to:
This instantiation of our approach is illustrated in Figure 1. Here, proScript has generated a script x to achieve the goal “see an alligator”, but the script contains an error: it states that the steps of “driving to the zoo” and “get in car” can be applied in any order. To repair this, the user provides the general feedback “Get in a car before driving”. The corrector model G then takes that feedback and the erroneous script, translates it into appropriate edit operations on the script, and applies those edits to generate a corrected script (y in Figure 1). The feedback is stored in memory M so it can also be retrieved in the future. Our system, FBNet, comprises the corrector module G, the memory M, and searching and writing operations. To train our system, we collect examples of bad outputs, general feedback, and specific edits that the feedback should translate to (Section 4.2). This allows G to learn how to translate general feedback into specific edits to apply. Pairing G with the memory M allows FBNet to repair new, unseen scripts containing similar errors to the one the user corrected.
(Section 1 “Introduction.” Emphasis added.).
To train the corrector G, as well as evaluate our approach, we collected a set of (x, fb, y) tuples using crowdworkers, where x is a possibly erroneous script generated by PROSCRIPTgen, fb is general feedback about the error (if any), and y is the corrected script. In practice, crowdworkers specified the edits to x to create y (using simple graph operations we can generate y from ye – see Table 7 for an example). We collected 1542 tuples of data, randomly splitting it into 843 train, 154 validation, and 545 test points. Examples of the resulting dataset are shown in Table 1.
(Section 4.2 “Feedback Data Collection.” Emphasis added.).
The collected examples of bad outputs from the LM, general feedback, and/or specific edits that the feedback should translate to are considered “updated information” like claimed, which is received after deploying the foundational generative AI model, i.e. the LM, and wherein the updated information is relevant to the generation of the responses.);
training, based on the updated information, an outer layer model, wherein the outer layer model comprises a model that is dynamically updatable, and wherein training the outer layer model configures the outer layer model to evaluate the responses from the foundational generative AI model and to modify incorrect responses (as noted above, Tandon describes a system, “FBNet,” that comprises a corrector module and a memory, inter alia, and whereby training the system requires collecting examples of bad outputs from the LM, general feedback, and specific edits to the bad outputs that the feedback should translate to. This system, FBNet, is considered an “outer layer model” like claimed, which is trained based on the updated information, i.e. based on the collected examples of bad outputs from the LM, general feedback, and specific edits to the bad outputs. Tandon particularly discloses that the training enables the corrector module to receive an output x from the LM (B) and feedback fb indicating an error in the LM output, and to generate an edit ye to correct the LM output x based on the feedback fb:
Fig. 2 gives an overview of FBNet. The input is a potentially noisy graph x generated by a base model B and the output y is a corrected graph. At inference time, i.e., after deployment, a user can critique y by providing natural language feedback fb on an error e. As output, the model generates the corrected graph y that accounts for fb.
The corrector model G is responsible for improving the potentially noisy output from B. G achieves it using user feedback stored in a continuously updated memory M.
(Section 3.1 “Overview of the Architecture.” Emphasis added.).
The graph corrector model G generates an improved output y given a noisy graph x and fb. This is done in a two-step process, (i) learning to predict a graph edit operation ye given x and fb [and] (ii) using simple graph operations to apply ye to x to produce y. Our approach of generating an edit instead of directly generating the corrected graph is beneficial for two reasons. First, generating edits is simpler for the model than generating entire graphs. Second, it simplifies evaluation metrics as it is much simpler to compare two smaller generated edits. Note that we can deterministically fix a script given an edit. Thus, the two-step process helps us achieve the same end goal (corrected scripts from noisy scripts and feedback).
(Section 3.4 “Corrector model G.” Emphasis added.).
As mentioned, the graph corrector G first generates an edit ye, which is applied to the incorrect graph x to generate the correct graph y. We need a corpus of (x, fb, y) to train this system. Specifically, we extract an edit from each such tuple, where edit ye is the difference between the output y and the input x. x and y can be expressed in a string representation using a graph description language such as DOT. We then train a language model to estimate Pθ(ye | x, fb), which allows us to generate an edit for a given (x, fb) using greedy sampling, where θ denotes the parameters of the language model.
(Section 3.5 “Training and Inference.” Emphasis added.).
Tandon further discloses that the memory M stores a history of erroneous LM outputs and corresponding feedback; in response to receiving a new output x from the LM, the memory is checked via a lookup function to see if there is a similar LM output already stored in the memory M. If so – the new LM output is deemed to comprise an error like the identified LM output already stored in memory – the new LM output x and feedback fb accessed from the memory is input to the corrector module G to generate a corrected output y:
The Memory M is a growing lookup table of key-value pairs: key (xi) - value (fbi), where xi is a particular incorrect graph, and fbi is the corresponding feedback. This memory supports lookup (read) and write operations. Given a new query x, FBNet uses feedback fb from similar, prior queries in the memory to enrich x. This feedback fb is retrieved using the lookup function Ω(x, M). The corrector then combines fb with x, and generates y. The write operation is used whenever a user gives new feedback.
(Section 3.1 “Overview of the Architecture.”).
As mentioned, the feedback is stored in a memory of key (x), value (fb) pairs. Ω is a retrieval function that matches a query key (xj) to a similar xi in memory implicitly on the similarity of the errors ei and ej.
(Section 3.3 “Memory M and Ω.” Emphasis added.).
It is apparent that, if there is not a similar output already stored in memory, meaning FBNet does not identify an error in the LM output, the LM output is provided to the user as is. Tandon further discloses that if the user identifies an error in the LM output, the user provides feedback indicating the error; the user-provided feedback and the LM output are then input to the corrector module to correct the LM output based on the feedback, and the LM output and feedback are stored in the memory so as to be applied to similar LM outputs in the future:
This instantiation of our approach is illustrated in Figure 1. Here, proScript has generated a script x to achieve the goal “see an alligator”, but the script contains an error: it states that the steps of “driving to the zoo” and “get in car” can be applied in any order. To repair this, the user provides the general feedback “Get in a car before driving”. The corrector model G then takes that feedback and the erroneous script, translates it into appropriate edit operations on the script, and applies those edits to generate a corrected script (y in Figure 1). The feedback is stored in memory M so it can also be retrieved in the future. Our system, FBNet, comprises the corrector module G, the memory M, and searching and writing operations. To train our system, we collect examples of bad outputs, general feedback, and specific edits that the feedback should translate to (Section 4.2). This allows G to learn how to translate general feedback into specific edits to apply. Pairing G with the memory M allows FBNet to repair new, unseen scripts containing similar errors to the one the user corrected.
(Section 1 “Introduction.” Emphasis added.).
Fig. 2 gives an overview of FBNet. The input is a potentially noisy graph x generated by a base model B and the output y is a corrected graph. At inference time, i.e., after deployment, a user can critique y by providing natural language feedback fb on an error e. As output, the model generates the corrected graph y that accounts for fb.
The corrector model G is responsible for improving the potentially noisy output from B. G achieves it using user feedback stored in a continuously updated memory M.
The Memory M is a growing lookup table of key-value pairs: key (xi) - value (fbi), where xi is a particular incorrect graph, and fbi is the corresponding feedback. This memory supports lookup (read) and write operations. Given a new query x, FBNet uses feedback fb from similar, prior queries in the memory to enrich x. This feedback fb is retrieved using the lookup function Ω(x, M). The corrector then combines fb with x, and generates y. The write operation is used whenever a user gives new feedback.
(Section 3.1 “Overview of the Architecture.” Emphasis added.).
Tandon thus teaches training, based on the updated information (i.e. based on the collected examples of bad outputs from the LM, general feedback, and specific edits to the bad outputs), an outer layer model (i.e. FBNet), wherein the outer layer model comprises a model that is dynamically updatable (i.e. FBNet is dynamically updated by storing LM output and new user feedback in the memory M), and wherein training the outer layer model configures the outer layer model to evaluate the responses from the foundational generative AI model (i.e. from the LM) and to modify (i.e. via the corrector module) incorrect responses.);
receiving, from a user device, a first input prompt, and inputting the first input prompt into the foundational generative AI model to produce an initial response (Tandon teaches that a prompt can be input to the foundational generative AI model, i.e. into the LM, whereby the foundational generative AI model produces an initial response x:
This instantiation of our approach is illustrated in Figure 1. Here, proScript has generated a script x to achieve the goal “see an alligator”, but the script contains an error: it states that the steps of “driving to the zoo” and “get in car” can be applied in any order. To repair this, the user provides the general feedback “Get in a car before driving”. The corrector model G then takes that feedback and the erroneous script, translates it into appropriate edit operations on the script, and applies those edits to generate a corrected script (y in Figure 1). The feedback is stored in memory M so it can also be retrieved in the future. Our system, FBNet, comprises the corrector module G, the memory M, and searching and writing operations. To train our system, we collect examples of bad outputs, general feedback, and specific edits that the feedback should translate to (Section 4.2). This allows G to learn how to translate general feedback into specific edits to apply. Pairing G with the memory M allows FBNet to repair new, unseen scripts containing similar errors to the one the user corrected.
(Section 1 “Introduction.” Emphasis added.)
PNG
media_image1.png
466
366
media_image1.png
Greyscale
The first input prompt, e.g. “goal: see an alligator” in Figure 1 above, is understandably received from a user device for input into the LM.);
inputting the first response into the outer layer model, and based on identifying, using the outer layer model, that the initial response is incorrect: (i) modifying the initial response to produce a modified response; and (ii) sending the modified response to the user device for display (As noted above, Tandon describes “FBNet,” which is considered an “outer layer model” like claimed, and which comprises a corrector module G and a memory M, inter alia. As further described above, Tandon teaches that the memory M stores a history of erroneous LM outputs and corresponding feedback; in response to receiving a new output x from the LM, the memory M is checked via a lookup function to see if there is a similar LM output already stored in the memory M. If so – the new LM output is deemed to comprise an error like the identified LM output already stored in memory – the new LM output x and feedback fb accessed from the memory is input to the corrector module G to generate a corrected output y. Tandon demonstrates that the corrected output y is provided to the user so that the user can provide any new feedback:
PNG
media_image2.png
437
364
media_image2.png
Greyscale
Accordingly, Tandon further teaches inputting the initial response from the LM into the outer layer model (i.e. into FBNet), and based on identifying, using the outer layer model, that the initial response is incorrect (i.e. that the initial response x is similar to an erroneous response already stored in the memory M), modifying the initial response to produce a modified response (i.e. inputting the initial response x and feedback fb from memory M into the corrector module G to produce corrected output y), and sending the modified response to the user device for display.).
The above-described tasks are understandably implemented via computer-readable instructions stored within the memory of a computing system that further comprises at least one processor for executing the instructions, and a communication interface communicatively coupled to the at least one processor. Such a computing system implementing the above-described tasks taught by Tandon is considered a computing platform similar to that of claim 1. The computing system/platform is considered to implement a method similar to that of claim 11. The memory of such a computing system/platform implementing the above-described tasks taught by Tandon is considered one or more non-transitory computer-readable media similar to that of claim 20. Tandon, however, does not explicitly disclose that the foundational generative AI model is trained using historical information so as to configure the foundational generative AI model to generate the responses to the input prompts, as is required in claims 1, 11 and 20.
Training machine learning models using historical information is nevertheless well-known in the art. Sakaguchi in particular teaches training, using historical information (e.g. using collected partially ordered scripts), a foundational generative AI model (i.e. a T5 large language model/proscript model), wherein training the foundational generative AI model configures the foundational generative AI model to generate responses (e.g. scripts) to input prompts (e.g. input scenarios) (see e.g. the Abstract, section 1 “Introduction,” section 4 “Datasets,” section 6.1 “Models” and section 6.3 “Experiments”).
It would have been obvious to one of ordinary skill in the art, having the teachings of Tandon and Sakaguchi before the effective filing date of the claimed invention, to modify the computing platform, method and one or more non-transitory computer readable media taught by Tandon so as to train the foundational generative AI model using historical information like taught by Sakaguchi, wherein training the foundational generative AI model configures the foundational generative AI model to generate the responses to the input prompts. It would have been advantageous to one of ordinary skill to utilize such a combination because it would enable the foundational generative AI model to learn more particular tasks, as is evident from Sakaguchi (see e.g. section 4 “Datasets,” section 6.1 “Models” and section 6.3 “Experiments”). Accordingly, Tandon and Sakaguchi are considered to teach, to one of ordinary skill in the art, a computing platform like that of claim 1, a method like that of claim 11, and one or more non-transitory computer-readable media like that of claim 20.
As per claim 10, Tandon further teaches that the outer layer model extends a functional life of the foundational generative AI model (see e.g. section 1 “Introduction”: Tandon generally teaches that the outer layer model, FBNet, is able to repair outputs of a foundational generative AI model, i.e. a large language model, without requiring retraining of the foundational generative AI model. This reduction in errors without requiring retraining would understandably extend the functional life of the foundational generative AI model.). Accordingly, the above-described combination of Tandon and Sakaguchi is further considered to teach a computing platform like that of claim 10.
Claims 2, 5-7, 12 and 15-17 are rejected under 35 U.S.C. 103 as being unpatentable over the above-described combination of Tandon and Sakaguchi, and also over the article entitled “Model assertions for monitoring and improving ML models” by Kang et al. (“Kang”).
Regarding claims 2 and 12, Tandon and Sakaguchi teach a computing platform like that of claim 1 and a method like that of claim 11, as is described above, wherein a computing platform trains an outer layer model to evaluate responses from a foundational generative AI model and to modify incorrect responses. Tandon and Sakaguchi, however, do not explicitly teach, based on identifying, using the outer layer model, that the initial response is correct, routing the initial response to one or more adaptation models, as is required by claims 2 and 12.
Kang nevertheless generally teaches applying “model assertions” to monitor machine learning models (see e.g. the Abstract). A model assertion is an arbitrary function over a model’s input and output that indicates whether an error may be occurring (see e.g. the Abstract and section 1 “Introduction”).
It would have been obvious to one of ordinary skill in the art, having the teachings of Tandon, Sakaguchi and Kang before the effective filing date of the claimed invention, to modify the computing platform and method taught by Tandon and Sakaguchi so as to also apply model assertions like taught by Kang to the foundational generative AI model, i.e. in sequence with the outer layer model. That is, when the outer layer model (FBNet) taught by Tandon and Sakaguchi identifies that the initial response of the foundational generative AI model is correct, it would have been obvious to further route the initial response to one or more model assertions like taught by Kang. Such model assertions are considered “adaptation models” like claimed. It would have been advantageous to one of ordinary skill to utilize such model assertions because they can identify systematic errors produced by the ML model (i.e. by the foundational generative AI model), as is taught Kang (see e.g. section 1 “Introduction”). Accordingly, Tandon, Sakaguchi and Kang are considered to teach, to one of ordinary skill in the art, a computing platform like that of claim 2 and a method like that of claim 12.
As per claims 5 and 15, it would have been obvious, as is described above, to modify the computing platform and method taught by Tandon and Sakaguchi so as to also apply model assertions (i.e. adaptation models) like taught by Kang to the foundational generative AI model. That is, when the outer layer model (FBNet) taught by Tandon and Sakaguchi identifies that the initial response of the foundational generative AI model is correct, it would have been obvious to further route the initial response to one or more model assertions like taught by Kang. Like noted above, Kang discloses that a model assertion is an arbitrary function over a model’s input and output that indicates whether an error may be occurring (see e.g. the Abstract and section 1 “Introduction”). Kang further teaches that the model assertions can provide runtime monitoring, and that errors identified by the model assertions can be logged (see e.g. section 1 “Introduction” which recites, “[f]irst, we show that model assertions can be used for runtime monitoring: they can be used to log unexpected behavior or automatically trigger corrective actions, e.g., shutting down an autopilot.” Section 2.3 “Using Model Assertions for QA” recites, “[t]hus, concretely, model assertions can be used to validate human labels (data collection) or historical data (validation), and to monitor deployments (e.g. to populate dashboards).”). It is thus apparent that, if a model assertion (i.e. adaptation model) identifies that the initial response is correct, the initial response would be provided to an end user: the initial response would be sent to the user device for display along with commands directing the user device to display the initial response, wherein sending the one or more commands directing the user device to display the initial response causes the user device to display the initial response. On the other hand, if the model assertion identifies that the initial response is incorrect, it is apparent that the error would be logged, i.e. negative feedback indicating the incorrect response would be sent to an administrator device. Accordingly, the above-described combination of Tandon, Sakaguchi and Kang is further considered to teach a computing platform like that of claim 5 and a method like that of claim 15.
Regarding claims 6 and 16, Tandon and Sakaguchi teach a computing platform like that of claim 1 and a method like that of claim 11, as is described above, wherein a computing platform trains an outer layer model to evaluate responses from a foundational generative AI model and to modify incorrect responses, and wherein the modified responses are sent to a user device for display. Tandon and Sakaguchi however do not explicitly teach that sending the modified response to the user device for display comprises sending, after validating the modified response at one or more adaptation models, the modified response, as is required by claims 6 and 16.
As described above, Kang generally teaches applying “model assertions” to monitor machine learning models (see e.g. the Abstract). A model assertion is an arbitrary function over a model’s input and output that indicates whether an error may be occurring (see e.g. the Abstract and section 1 “Introduction”).
It would have been obvious to one of ordinary skill in the art, having the teachings of Tandon, Sakaguchi and Kang before the effective filing date of the claimed invention, to modify the computing platform and method taught by Tandon and Sakaguchi so as to also apply model assertions like taught by Kang to the foundational generative AI model, i.e. in sequence with the outer layer model. That is, when the outer layer model (FBNet) taught by Tandon and Sakaguchi modifies the initial response of the foundational generative AI model, it would have been obvious to further route the modified response to one or more model assertions like taught by Kang, which indicate whether an error may be occurring (i.e. validate the modified response). Such model assertions are considered “adaptation models” like claimed. Sending the modified response to the user device for display would thus comprise sending the modified response after validating the modified response at one or more adaptation models. It would have been advantageous to one of ordinary skill to utilize such model assertions because they can identify systematic errors produced by the ML model (i.e. by the foundational generative AI model), as is taught Kang (see e.g. section 1 “Introduction”). Accordingly, Tandon, Sakaguchi and Kang are considered to teach, to one of ordinary skill in the art, a computing platform like that of claim 6 and a method like that of claim 16.
Regarding claims 7 and 17, Tandon and Sakaguchi teach a computing platform like that of claim 1 and a method like that of claim 11, as is described above, wherein a computing platform trains an outer layer model to evaluate responses from a foundational generative AI model and to modify incorrect responses. Tandon further teaches that feedback can be used to dynamically refine the outer layer model (see e.g. section 3.1 “Overview of the Architecture” and section 5.2 “RQ2: How well can FBNet learn from prior mistakes?”: Tandon discloses that feedback used to correct incorrect model outputs is stored in the memory M of FBNet so that it can be used on new outputs. The outer layer model, i.e. FBNet, is thus dynamically refined using the feedback.). Tandon and Sakaguchi, however, do not explicitly teach that feedback from one or more adaptation models is used to dynamically refine the outer layer model, as is required by claims 7 and 17.
As described above, Kang generally teaches applying “model assertions” to monitor machine learning models (see e.g. the Abstract). A model assertion is an arbitrary function over a model’s input and output that indicates whether an error may be occurring (see e.g. the Abstract and section 1 “Introduction”). Kang further teaches that the model assertions can provide runtime monitoring, and that errors identified by the model assertions can be logged (see e.g. section 1 “Introduction” which recites, “[f]irst, we show that model assertions can be used for runtime monitoring: they can be used to log unexpected behavior or automatically trigger corrective actions, e.g., shutting down an autopilot.” Section 2.3 “Using Model Assertions for QA” recites, “[t]hus, concretely, model assertions can be used to validate human labels (data collection) or historical data (validation), and to monitor deployments (e.g. to populate dashboards).”).
It would have been obvious to one of ordinary skill in the art, having the teachings of Tandon, Sakaguchi and Kang before the effective filing date of the claimed invention, to modify the computing platform and method taught by Tandon and Sakaguchi so as to also apply model assertions like taught by Kang to the foundational generative AI model, i.e. in sequence with the outer layer model, wherein the model assertions also provide feedback (i.e. logged errors). Such model assertions are considered “adaptation models” like claimed. It would have been obvious to dynamically refine the outer layer model using such feedback (i.e. to store the feedback in the memory of FBNet and enable foundational generative AI model outputs to be modified using such stored feedback). It would have been advantageous to one of ordinary skill to utilize such model assertions because they can identify systematic errors produced by the ML model (i.e. by the foundational generative AI model), as is taught Kang (see e.g. section 1 “Introduction”). Accordingly, Tandon, Sakaguchi and Kang are considered to teach, to one of ordinary skill in the art, a computing platform like that of claim 7 and a method like that of claim 17.
Claims 3, 4, 13 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over the above-described combination of Tandon, Sakaguchi, and Kang, and also over the article entitled “Towards Reliable and Fluent Large Language Models: Incorporating Feedback Learning Loops in QA Systems” by Lee et al. (“Lee”).
Regarding claims 3 and 13, Tandon, Sakaguchi and Kang teach a computing platform like that of claim 2 and a method like that of claim 12, as is described above, wherein a computing platform trains an outer layer model to evaluate initial responses from a foundational generative AI model; the computing platform routes an initial response to one or more adaptation models based on identifying that the initial response is correct. Tandon, Sakaguchi and Kang, however, do not explicitly teach that the one or more adaptation models are each trained to provide responses in a corresponding context, as is required by claims 3 and 13.
Similar to Kang’s model assertions, Lee describes a “critic model” that monitors a foundational generative AI model (i.e. a large language model) and indicates whether an error may be occurring (see e.g. the Abstract, section 3.2 “Training Critic Models to Provide Feedback on Heterogeneous Aspects” and section 3.3 “Iterative Feedback Learning for LLM Enhancement on Heterogeneous Aspects”). Lee particularly demonstrates that the critic model is trained to provide responses in a corresponding context (e.g. question answering) (see e.g. section 3.1 “Construction the Heterogeneous Aspect-Focused Dataset for Critic Model Training” and section 3.1 “Training Critic Models to Provide Feedback on Heterogeneous Aspects”).
It would have been obvious to one of ordinary skill in the art, having the teachings of Tandon, Sakaguchi, Kang and Lee before the effective filing date of the claimed invention, to modify the computing platform and method taught by Tandon, Sakaguchi and Kang so that the model assertions (i.e. adaptation models) additionally or alternatively include a critic model like taught by Lee, which is trained to provide responses in a corresponding context. It would have been advantageous to one of ordinary skill to utilize such a critic model (i.e. adaptation model) because it can identify particular issues (e.g. fluency) in the responses produced by the foundational generative AI model (i.e. large language model), as is taught by Lee (see e.g. the Abstract). Accordingly, Tandon, Sakaguchi, Kang and Lee are considered to teach, to one of ordinary skill in the art, a computing platform like that of claim 3 and a method like that of claim 13.
As per claims 4 and 14, it would have been obvious, as is described above, to modify the computing platform and method taught by Tandon, Sakaguchi and Kang so that the model assertions (i.e. adaptation models) additionally or alternatively include a critic model like taught by Lee, which is trained to provide responses in a corresponding context. Lee particularly teaches that the corresponding context comprises one of question answering (QA), sentiment analysis, information extraction, image captioning, object recognition, or instruction following (see e.g. the Abstract). Accordingly, the above-described combination of Tandon, Sakaguchi, Kang and Lee is further considered to teach a computing platform like that of claim 4 and a method like that of claim 14.
Claims 8, 9, 18 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over the above-described combination of Tandon and Sakaguchi, and also over U.S. Patent No. 12,724,659 to Paulraj et al. (“Paulraj”).
Regarding claims 8 and 18, Tandon and Sakaguchi teach a computing platform like that of claim 1 and a method like that of claim 11, as is described above, wherein a computing platform trains an outer layer model to evaluate responses from a foundational generative AI model and to modify incorrect responses. Tandon and Sakaguchi, however, do not explicitly teach: (i) identifying that more than a predetermined number of incorrect responses have been received from the foundational generative AI model during a predetermined time period; and (ii) based on identifying that more than the predetermined number of incorrect responses have been received, causing the foundational generative AI model to be decommissioned and rebuilt, as is required by claims 8 and 18.
Paulraj nevertheless generally teaches identifying that more than a predetermined number of incorrect responses (e.g. hallucinations) have been received from a generative AI model (i.e. a large language model) during a predetermined time period, and based on identifying that more than the predetermined number of incorrect responses have been received, causing the generative AI model to be decommissioned and rebuilt (i.e. retrained) (see e.g. column 29, lines 14-37).
It would have been obvious to one of ordinary skill in the art, having the teachings of Tandon, Sakaguchi and Paulraj before the effective filing date of the claimed invention, to modify the computing platform and method taught by Tandon and Sakaguchi so as to (i) identify that more than a predetermined number of incorrect responses have been received from the generative AI model (i.e. the foundational generative AI model) during a predetermined time period, and (ii) based on identifying that more than the predetermined number of incorrect responses have been received, cause the generative AI model to be decommissioned and rebuilt, as is taught by Paulraj. It would have been advantageous to one of ordinary skill to utilize such a combination, because it would reduce the amount of errors output by the generative AI model, as is evident from Paulraj (see e.g. column 29, lines 24-37). Accordingly, Tandon, Sakaguchi and Paulraj are considered to teach, to one of ordinary skill in the art, a computing platform like that of claim 8 and a method like that of claim 18.
As per claims 9 and 19, it would have been obvious, as is described above, to modify the computing platform and method taught by Tandon and Sakaguchi so as to (i) identify that more than a predetermined number of incorrect responses have been received from the generative AI model (i.e. the foundational generative AI model) during a predetermined time period, and (ii) based on identifying that more than the predetermined number of incorrect responses have been received, cause the generative AI model to be decommissioned and rebuilt, as is taught by Paulraj. Paulraj particularly teaches that the generative AI model is automatically rebuilt (i.e. retrained) based on feedback from one or more adaptation models (see e.g. column 29, lines 8-37: Paulraj discloses that a filtering process is used to identify and filter hallucinations output by a generative AI model, whereby the generative AI model is automatically retrained if a filtering threshold is reached. The filtering process is considered an “adaptation model” like claimed.). Accordingly, the above-described combination of Tandon, Sakaguchi and Paulraj is further considered to teach a computing platform like that of claim 9 and a method like that of claim 19.
Conclusion
The prior art made of record on form PTO-892 and not relied upon is considered pertinent to applicant’s disclosure. The applicant is required under 37 C.F.R. §1.111(C) to consider these references fully when responding to this action. In particular, the U.S. Patent Application Publication to Bycroft et al. cited therein describes a method for generating a supplemental machine learning model based on responses (i.e. feature space data) produced by an existing machine learning model. The U.S. Patent Application Publication to Ma et al. cited therein describes a method for verifying output data of a processing model, particularly by using a separate verification model. The U.S. Patent Application Publication to Fröhlich et al. cited therein describes a computer-implemented method for correcting at least one model output of a first trained machine learning model, including by determining at least one correction parameter using a second trained machine learning model. The article by Hartvigsen et al. cited therein (“Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors”) describes a lifelong editing method that implements spot-fixes on streaming errors of a deployed model.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BLAINE T BASOM whose telephone number is (571)272-4044. The examiner can normally be reached Monday-Friday, 9:00 am - 5:30 pm, EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matt Ell can be reached at (571)270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BTB/
9/16/2026
/MATTHEW ELL/Supervisory Patent Examiner, Art Unit 2141
Prosecution Insights