DETAILED ACTION
This Office Action is responsive to Request for Continued Examination filed on June 9th, 2026. Claims 1, 4 and 13 are amended, claims 1-20 are pending and have been examined.
Any previous objections/rejections not mentioned in this Office Action have been withdrawn by the Examiner.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on September 16th, 2025 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Response to Amendments and Arguments
Regarding rejections under 35 U.S.C. 102(a)(1), Applicant argues, "The method of Wang aims to replicate the response of the original model to the additional data for when the model is presented with the data to be forgotten. To do this, Wang uses knowledge gap alignment that compares the outputs of the different models to different inputs. By contrast, the method disclosed in the Specification uses smaller auxiliary models trained on larger datasets (e.g., the original training corpus D and the original training corpus with F removed D/F), see Specification at paragraphs [0087]-[0088]. The outputs of the smaller auxiliary models on the same input (e.g., a first input) may be used to adjust the parameters of the first language model to be closer to the output of the smaller auxiliary model trained on a corpus D/F. As such, Wang does not disclose 'training a first auxiliary LM using first training corpus D,' as recited in claim 4," (emphasis original, page 7 or Remarks).
Examiner respectfully disagrees. Wang teaches a method that uses a joint alignment technique to move a model's distribution space in line with other models. In order to achieve measurable outputs, Wang must use the same inputs for the learning pairs as disclosed in sections 4.1 Knowledge Gap Alignment, "Since An and Af are trained on Dn and Df, respectively, we expect that the knowledge gap when feeding Df to A* and Af should be similar to feeding Dn to AD and An according to Eq. 2. This is under the assumption that a similar knowledge deficit can be observed when the same architecture handles the seen (i.e., used for training) and unseen data with a similar distribution. And we believe that a successful unlearning method should make the target model A* handle Df as unseen data." Further details are provided below.
Applicant further argues, "while Wang discloses An(Af) can be trained with the combination of Dn(Df) and a small fraction of D/Df, this is due to the small nature of the Df and Dn, which may not be sufficient to train a model. The training of models with the data to be forgotten and a small portion of the original training data is not the same as training an auxiliary LM using a second training corpus D/F, wherein the second training corpus is the first training corpus with the data to be forgotten removed. As such, Wang does not disclose "generating, by removing the first data F from the first training corpus D, a second training corpus D/F," and "training a second auxiliary LM using the second training corpus D/F," as now recited in claim 4 with the amendments made herein," (emphasis original, page 8 of Remarks).
In this regard, Applicant's argument is moot, as new grounds of rejection are raised under 35 U.S.C. 103 in view of "Coded Machine Unlearning" by Aldaghri et al. which teaches a method for creating such a training dataset for unlearning. Further details are provided below.
Regarding rejections under 35 U.S.C. 103, Applicant argues, "Although the language of claim 1 is not identical to claim 4, the allowability of claim 1 will be apparent in light of the discussion above in reference to claim 4," (page 11 of Remarks).
Similarly to the above discussion of the application of Wang to the limitations of claim 4, Applicant's argument in moot in view of new grounds of rejection under "Coded Machine Unlearning" by Aldaghri et al.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3 are rejected under 35 U.S.C. 103 as being unpatentable over “KGA: A General Machine Unlearning Framework Based on Knowledge Gap Alignment” by Wang et al. (hereinafter, “Wang”) in view of "Coded Machine Unlearning" by Aldaghri et al. (hereinafter, "Aldaghri"), China Invention Application CN 116776160 to Shi (hereinafter, "Shi") and U.S. Patent Application Publication 2025/0265447 to Li et al. (hereinafter, "Li").
Regarding claim 1, Wang teaches A computer-implemented method comprising:determining a first language model (LM) trained on a first training corpus D (section 4.1 KGA Framework, "Apart from data, we have model A(D) as input, which is the original model trained with data D that needs unlearning (we abbreviate it as AD in the following parts of this paper).");
determining first data F, wherein the first data F is a subset of D, and wherein the first data F is identified as a set of data to be unlearned by the first LM (section 4.1 KGA Framework, "The input data consists of previous training data D, data to be forgotten Df, and a small set of extra data Dn to assist the unlearning, where Dn ∩ D = ∅.");
determining a first text input of the first data F (page 3, Knowledge Gap Alignment, "Since An and Af are trained on Dn and Df, respectively, we expect that the knowledge gap when feeding Df to A* and Af should be similar to feeding Dn to AD and An according to Eq. 2," and page 4, Objectives, "Objectives, "Pr(A)(z) is the output distribution given input z to model A, KL(a|b) measures the KL divergence between distribution a and b. y and z are from Dn and Df, respectively."); and
updating the first LM to generate an updated LM based at least in part by:
minimizing a first prediction difference between a first prediction of the first LM and a second prediction of the second n-gram language model for the first text input (page 4, Knowledge Gap Alignment, "For Goal 2, we maintain the ability of model A∗ when processing the remaining data, i.e., Dr. We treat the original model AD as a teacher and directly minimize the distance of output distributions when feeding samples in Dr to A∗ and AD."). Examiner interprets “prediction difference” as analogous to “distance of output distributions”. The instant claim compares the output from two different models and adjusts one model such that their outputs are similar. As taught by Wang, a teacher and student model’s output distribution is minimized to align their outputs on the same input data.
Wang teaches an arrangement of training data that is distinct from that of the claimed D and D/F in that Wang does not simply remove the subset F from the original training dataset D to create D/F. Aldaghri teaches a method for machine unlearning that includes training a first n-gram language model using the first training corpus D (section II. Problem setup and proposed protocol, B. Proposed protocol, 1) Learning, "The learning model M, also referred to as the master node, is launched to learn a regression model whose training dataset is assumed to have been preprocessed. The model starts by passing the training dataset through a encoder to produce a sharded coded training dataset that contains r coded shards. Then, each coded shard j is used to train a weak learner Lj to produce a model denoted as f*j. Once these weak learners are trained, the model M is ready for prediction.");
generating, by removing the first data F from the first training corpus D, a second training corpus D/F (section II. Problem setup and proposed protocol, B. Proposed protocol, 2) Unlearning, "Now that the model has been trained on the coded training dataset, we proceed to describe a protocol to unlearn samples from this model. Our goal is to remove such samples from the coded shards as well as to remove any trace of such samples from the affected weak learners where such samples appear."); and
training a second n-gram language model using the second training corpus D/F (section II. Problem setup and proposed protocol, B. Proposed protocol, 2) Unlearning, "The samples first need to be removed from the coded shard by subtracting them from the corresponding coded samples in all coded shards where they appear to eliminate their effect from the coded shards. Once all the coded shards are updated, they are then used to update their corresponding weak learners to unlearn these samples from the weak learner models followed by updating the final aggregate model using the updated weak learners’ estimates.").
Wang and Aldaghri are considered analogous because they are each concerned with language model training. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have combined Aldaghri’s training dataset arrangement with Wang’s parallel training goals for the purpose of achieving desired language model performance with particular dataset elements removed. Given that all the claimed elements were known in the prior art, one skilled in the art could have combined the elements by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
The combination of Wang and Aldaghri does not explicitly teach “maximizing a second prediction difference between the first prediction of the first LM and a third prediction of the first n-gram language model for the first text input,” and thus, Shi is introduced.
Shi teaches maximizing a second prediction difference between the first prediction of the first LM and a third prediction of the first n-gram language model for the first text input (page 2, "based on the first target training direction, training to obtain a click behaviour prediction model, the click behaviour prediction model is used for predicting the probability that the object clicks on the content to be recommended, the first target training direction is to minimize the similarity difference between the anchor point data and the positive sample data, and maximizing a similarity difference between the anchor point data and the first difficult negative sample data.").
Wang, Aldaghri and Shi are considered analogous because they are each concerned with language model training. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have combined Shi’s maximal-and-minimal training goals with Wang’s parallel training goals for the purpose of achieving desired language model performance with a push-and-pull architecture. Given that all the claimed elements were known in the prior art, one skilled in the art could have combined the elements by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
The combination of Wang, Aldaghri and Shi does not explicitly teach language models that are n-gram models, however, Li teaches at paragraph [0098], "For instance, applying the language models to generate the logits includes, for each language model (e.g., for model 241 and model 242), tokenizing the input (user input query) into discrete tokens (e.g., words or sub words) and then processing the tokens according to the specific architectures and parameters of each language model," and paragraph [0102], "At 380, the method includes generating an output token (or tokens in some cases) based on the determined (one or more) probabilities." Paragraph [0052] of the Specification discloses, "An N-gram LM indexes the statistics of n-grams (words and/or portions of words (e.g., lemmatized and/or stemmed tokens, etc.)) in training data and may use maximum likelihood estimation (MLE) to estimate probability of generated outputs."
Wang, Aldaghri, Shi and Li are considered analogous because they are each concerned with language model training. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have used the n-gram models of Li in the training methods taught by Wang, Aldaghri and Shi for the purpose of model training efficiency, in view of design incentives disclosed in paragraph [0052] of the Specification, “Due to the properties of N-gram LMs, unlearning is trivial as it can be done by simply subtracting out the corresponding statistics of F from the model.”
Regarding claim 2, Wang teaches The computer-implemented method of claim 1, further comprising:determining first loss comprising the first prediction difference (page 4, Knowledge Gap Alignment, "For Goal 2, we maintain the ability of model A∗ when processing the remaining data, i.e., Dr. We treat the original model AD as a teacher and directly minimize the distance of output distributions when feeding samples in Dr to A∗ and AD."). Examiner interprets “prediction difference” as analogous to “distance of output distributions”. The instant claim compares the output from two different models and adjusts one model such that their outputs are similar. As taught by Wang, a teacher and student model’s output distribution is minimized to align their outputs on the same input data.
Wang does not explicitly teach “determining second loss comprising the second prediction difference,” or “generating the updated LM based at least in part by updating parameters of the first LM to decrease the first loss and to increase the second loss,” however, Shi teaches determining second loss comprising the second prediction difference (page 10, “wherein the first expected loss function is used for identifying the second similarity between the anchor point data and the positive sample data, and the first similarity between the anchor point data and the first difficult negative sample data, the second expected loss function is used for the second similarity between the anchor point data and the positive sample data,”); andgenerating the updated LM based at least in part by updating parameters of the first LM to decrease the first loss and to increase the second loss (page 2, "based on the first target training direction, training to obtain a click behaviour prediction model, the click behaviour prediction model is used for predicting the probability that the object clicks on the content to be recommended, the first target training direction is to minimize the similarity difference between the anchor point data and the positive sample data, and maximizing a similarity difference between the anchor point data and the first difficult negative sample data.").
Wang and Shi are considered analogous because they are each concerned with language model training. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified Wang with the teachings of Shi for the purpose of achieving desired language model performance. Given that all the claimed elements were known in the prior art, one skilled in the art could have combined the elements by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Regarding claim 3, Wang teaches The computer-implemented method of claim 1, wherein:the first prediction difference comprises a Kullback-Leibler divergence between a first probability distribution of the first LM for the first text input and a second probability distribution of the second n-gram language model for the first text input (page 4, Knowledge Gap Alignment, "For Goal 2, we maintain the ability of model A∗ when processing the remaining data, i.e., Dr. We treat the original model AD as a teacher and directly minimize the distance of output distributions when feeding samples in Dr to A∗ and AD," and, "dis(D)(A1, A2) indicates the difference of the output distributions between model A1 and A2 on data D, which can be evaluated by KL divergence, Bregman divergence, or any other distributional distance measurements."); andthe second prediction difference comprises the Kullback-Leibler divergence between the first probability distribution of the first LM for the first text input and a third probability distribution of the first n-gram language model for the first text input (page 3, Knowledge Gap Alignment, "To achieve Goal 1, the output distribution of our target model A∗ on data Df (noted as A∗ (Df)) is expected to be similar to AD(Dn), where Dn should be an external set to D but with the similar distribution," and, "dis(D)(A1, A2) indicates the difference of the output distributions between model A1 and A2 on data D, which can be evaluated by KL divergence, Bregman divergence, or any other distributional distance measurements.").
Claims 4, 7, 10, 13, 16 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Wang in view of Aldaghri.
Regarding claims 4 and 13, Wang teaches a method and system for machine unlearning comprising: determining a first language model (LM) trained on a first training corpus D (section 3 Notations and Definition, "The training data set
D
∈
Z
*
is given as input. Given D, we train an ML model form a hypothesis space H… The trained model is denoted as A(D)," and section 4.1 KGA Framework, "Apart from data, we have model A(D) as input, which is the original model trained with data D that needs unlearning (we abbreviate it as AD in the following parts of this paper).");
determining first data F, wherein the first data F is a subset of D (section 3 Notations and Definition, "Then we denote the unlearning mechanism as a function U, which takes a training dataset
D
∈
Z
*
, a forget set
D
f
⊂
D
(containing data to be remoeved) and a model A(D) as input…" and section 4.1 KGA Framework, "The input data consists of previous training data D, data to be forgotten Df, and a small set of extra data Dn to assist the unlearning, where Dn ∩ D = ∅.");
determining a first text input (page 4, Objectives, "Pr(A)(z) is the output distribution given input z to model A, KL(a|b) measures the KL divergence between distribution a and b. y and z are from Dn and Df, respectively."); and
updating the first LM based at least in part on a first prediction difference between a first prediction of the first LM and a second prediction of the second auxiliary LM for the first text input and a second prediction difference between the first prediction of the first LM and a third prediction of the first auxiliary LM for the first text input (page 3, section 4.1 KGA Framework, "The output of KGA is a model A*, whose parameters are initialized with AD and are further updated with our KGA unlearning mechanism to remove Df… We reframe and summarize two goals to achieve the approximate unlearning defined in Sec. 3. They are Goal 1: Make our output model A*'s behavior on Df similar to its behavior on any unseen data (i.e. data not used for training); and Goal 2: Maintaining the performance of A* on Dr," and page 4, Objectives, "The two objectives are jointly optimized during unlearning to achieve Goal 1 and 2 simultaneously. Therefore, the final objective is defined as:L = La + α · Lr").
Wang does not explicitly teach “generating, by removing the first data F from the first training corpus D, a second training corpus D/F,” “training a first auxiliary LM using the first training corpus D,” or “training a second auxiliary LM using the second training corpus D/F,” however, Aldaghri teaches a method for machine unlearning that includes generating, by removing the first data F from the first training corpus D, a second training corpus D/F (section II. Problem setup and proposed protocol, B. Proposed protocol, 2) Unlearning, "Now that the model has been trained on the coded training dataset, we proceed to describe a protocol to unlearn samples from this model. Our goal is to remove such samples from the coded shards as well as to remove any trace of such samples from the affected weak learners where such samples appear.");
training a first auxiliary LM using the first training corpus D (section II. Problem setup and proposed protocol, B. Proposed protocol, 1) Learning, "The learning model M, also referred to as the master node, is launched to learn a regression model whose training dataset is assumed to have been preprocessed. The model starts by passing the training dataset through a encoder to produce a sharded coded training dataset that contains r coded shards. Then, each coded shard j is used to train a weak learner Lj to produce a model denoted as f*j. Once these weak learners are trained, the model M is ready for prediction."); and
training a second auxiliary LM using the second training corpus D/F (section II. Problem setup and proposed protocol, B. Proposed protocol, 2) Unlearning, "The samples first need to be removed from the coded shard by subtracting them from the corresponding coded samples in all coded shards where they appear to eliminate their effect from the coded shards. Once all the coded shards are updated, they are then used to update their corresponding weak learners to unlearn these samples from the weak learner models followed by updating the final aggregate model using the updated weak learners’ estimates.").
Wang and Aldaghri are considered analogous because they are each concerned with language model training. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the training taught by Wang with the datasets of Aldaghri for the purpose of achieving desired language model performance. Given that all the claimed elements were known in the prior art, one skilled in the art could have combined the elements by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Regarding claims 7 and 16, Wang further teaches determining the first prediction difference using a Kullback-Leibler divergence between a first probability distribution of the first LM for the first text input and a second probability distribution of the second auxiliary LM for the first text input (page 3, Knowledge Gap Alignment, "To achieve Goal 1, the output distribution of our target model A∗ on data Df (noted as A∗ (Df)) is expected to be similar to AD(Dn), where Dn should be an external set to D but with the similar distribution," and, "dis(D)(A1, A2) indicates the difference of the output distributions between model A1 and A2 on data D, which can be evaluated by KL divergence, Bregman divergence, or any other distributional distance measurements.").
Regarding claims 10 and 19, Wang further teaches partitioning the first training corpus D into n training partitions (page 2, Exact Unlearning, "As for more recent efforts in neural model unlearning, Bourtoule et al. (2021) propose a general method called SISA to train the model by partitioning the original dataset into several non-overlapping shards first and then designing effective mechanisms to aggregate models trained with shards.");
training a first plurality of auxiliary models using the n training partitions, wherein each auxiliary model of the first plurality of auxiliary models is trained using a respective one of the n training partitions (page 2, Exact Unlearning, "When handling data deletion, this method only has to retrain the models trained with the affected shards."); and
generating the second auxiliary LM by aggregating the first plurality of auxiliary models (page 2, Exact Unlearning, "As for more recent efforts in neural model unlearning, Bourtoule et al. (2021) propose a general method called SISA to train the model by partitioning the original dataset into several non-overlapping shards first and then designing effective mechanisms to aggregate models trained with shards.").
Claims 5 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Wang and Aldaghri as applied to claims 4 and 13 above, further in view of Shi.
Regarding claims 5 and 14, the combination of Wang and Aldaghri does not explicitly teach a method or system “further comprising updating the first LM based at least in part by updating parameters of the first LM to decrease the first prediction difference and increase the second prediction difference,” however, Shi teaches updating the first LM based at least in part by updating parameters of the first LM to decrease the first prediction difference and increase the second prediction difference (page 2, "based on the first target training direction, training to obtain a click behaviour prediction model, the click behaviour prediction model is used for predicting the probability that the object clicks on the content to be recommended, the first target training direction is to minimize the similarity difference between the anchor point data and the positive sample data, and maximizing a similarity difference between the anchor point data and the first difficult negative sample data.").
Wang, Aldaghri and Shi are considered analogous because they are each concerned with language model training. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the combination of Wang and Aldaghri with the teachings of Shi for the purpose of achieving desired language model performance. Given that all the claimed elements were known in the prior art, one skilled in the art could have combined the elements by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Claims 6, 8, 9, 15, 17 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Wang and Aldaghri as applied to claims 4 and 13 above, further in view of Li.
Regarding claims 6 and 15, the combination of Wang and Aldaghri does not explicitly teach a method or system “wherein a first number of parameters of the first LM is at least a magnitude greater than a second number of parameters of the first auxiliary LM,” however, Li teaches a first number of parameters of the first LM is at least a magnitude greater than a second number of parameters of the first auxiliary LM (paragraph [0146], "An analysis of mitigation performance is presented, with a focus on the results obtained using the Pythia 2.8B LLM and the Pythia 160M small language model (SLM). As demonstrated in Table 800 of FIG. 8, the ensemble strategy employing an SLM with only 160M parameters effectively reduces PII leakage while maintaining optimal model performance.").
Wang, Aldaghri and Li are considered analogous because they are each concerned with language model training. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the combination of Wang and Aldaghri with the teachings of Li for the purpose of improving model training efficiency. Given that all the claimed elements were known in the prior art, one skilled in the art could have combined the elements by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Regarding claims 8 and 17, the combination of Wang and Aldaghri does not explicitly teach a method or system “wherein the first auxiliary LM is a first n-gram LM and the second auxiliary LM is a second n-gram LM,” however, Li teaches the first auxiliary LM is a first n-gram LM and the second auxiliary LM is a second n-gram LM (paragraph [0098], "For instance, applying the language models to generate the logits includes, for each language model (e.g., for model 241 and model 242), tokenizing the input (user input query) into discrete tokens (e.g., words or sub words) and then processing the tokens according to the specific architectures and parameters of each language model," and paragraph [0102], "At 380, the method includes generating an output token (or tokens in some cases) based on the determined (one or more) probabilities."). Paragraph [0052] of the Specification discloses, "An N-gram LM indexes the statistics of n-grams (words and/or portions of words (e.g., lemmatized and/or stemmed tokens, etc.)) in training data and may use maximum likelihood estimation (MLE) to estimate probability of generated outputs."
Wang, Aldaghri and Li are considered analogous because they are each concerned with language model training. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the combination of Wang and Aldaghri with the teachings of Li for the purpose of model training efficiency, in view of design incentives disclosed in paragraph [0052] of the Specification, “Due to the properties of N-gram LMs, unlearning is trivial as it can be done by simply subtracting out the corresponding statistics of F from the model.”
Regarding claims 9 and 18, the combination of Wang and Aldaghri does not explicitly teach a method or system “wherein the second auxiliary LM is a language model trained on a dataset comprising public domain text data,” however, Li teaches the second auxiliary LM is a language model trained on a dataset comprising public domain text data (paragraph [0061], "For example, LLMs are trained on massive datasets that may contain copyrighted material, making it difficult to ensure compliance with intellectual property (IP) rights. This can result in legal, ethical, and financial consequences, especially for code models trained on open-source repositories without adhering to licensing terms. Lawsuits have already been filed against various organizations reproducing licensed code without following license terms. Various proposals have been put forth to mitigate this risk, including filtering licensed data and implementing techniques to 'unlearn' specific training data.").
Wang, Aldaghri and Li are considered analogous because they are each concerned with language model training. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the combination of Wang and Aldaghri with the teachings of Li for the purpose of legitimizing model training, in view of design incentives disclosed in paragraph [0012] of the Specification, “For example, a copyright owner of a work (e.g., a written work, an artwork, etc.) may want to have their work removed from the training corpus of an LM.”
Claims 11 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Wang and Aldaghri as applied to claims 4, 10, 13 and 19 above, further in view of "Machine Unlearning" by Bourtoule et al. (hereinafter, "Bourtoule").
Regarding claims 11 and 20, the combination of Wang and Aldaghri does not explicitly teach a method or system “wherein data of the n training partitions excludes the first data F,” and thus, Bourtoule is introduced. Bourtoule teaches data of the n training partitions excludes the first data F (page 2, Introduction, "In addition, rather than training each model on the entire shard directly, we can divide each shard’s data into slices and present slices incrementally during training. We save the state of model parameters before introducing each new slice, allowing us to start retraining the model from the last known parameter state that does not include the point to be unlearned—rather than a random initialization").
Wang, Aldaghri and Bourtoule are considered analogous because they are each concerned with language model training. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the combination of Wang and Aldaghri with the teachings of Bourtoule for the purpose of achieving desired language model performance. Given that all the claimed elements were known in the prior art, one skilled in the art could have combined the elements by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Wang and Aldaghri as applied to claim 4 above, further in view of "Reinforcement Unlearning" by Ye et al. (hereinafter, "Ye").
Regarding claim 12, the combination of Wang and Aldaghri does not explicitly teach “The method of claim 4, further comprising: determining a reinforcement learning policy with a reward term that rewards the first LM for generating outputs that are statistically similar to outputs of the second auxiliary LM and a penalty term that penalizes the first LM for generating outputs that are statistically similar to outputs of the first auxiliary LM for a given input, wherein statistical similarity is determined using a first statistical similarity metric,” and thus, Ye is introduced.
Ye teaches determining a reinforcement learning policy with a reward term that rewards the first LM for generating outputs that are statistically similar to outputs of the second auxiliary LM (page 1, Introduction, “With each action taken, the agent receives a reward and updates its state, creating an experience sample used to update its policy.”) anda penalty term that penalizes the first LM for generating outputs that are statistically similar to outputs of the first auxiliary LM for a given input (page 7, Direct Reward Inversion, “As both unlearning methods focus on reducing the agent’s received reward in the unlearning environment, a seemingly straightforward approach involves directly inverting the received reward: changing a real reward r to −r.”),wherein statistical similarity is determined using a first statistical similarity metric (page 7, Environmental Inference, "During the crossover process, pairs of transition functions are selected based on their fitness, determined by their similarity to the learned optimal policy.").
Wang, Aldaghri and Ye are considered analogous because they are each concerned with language model training. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the combination of Wang and Aldaghri with the teachings of Ye for the purpose of achieving desired language model performance. Given that all the claimed elements were known in the prior art, one skilled in the art could have combined the elements by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
China Invention Application CN 112529209 to Meng.
China Invention Application CN 113807455 to Kong et al.
U.S. Patent Application Publication 2022/0147864 to Chang et al.
“QUARK: Controllable Text Generation with Reinforced Unlearning” by Lu et al.
“Knowledge Unlearning for LLMs: Tasks, Methods, and Challenges” by Si et al.
“Learn to Unlearn: A Survey on Machine Unlearning” by Qu et al.
“Towards Unbounded Machine Unlearning” by Kurmanji et al.
“Towards Making Systems Forget with Machine Unlearning” by Cao et al.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SEAN T SMITH whose telephone number is (571)272-6643. The examiner can normally be reached Monday - Friday 8:00am - 5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, PIERRE-LOUIS DESIR can be reached at (571) 272-7799. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SEAN THOMAS SMITH/Examiner, Art Unit 2659
/PIERRE LOUIS DESIR/Supervisory Patent Examiner, Art Unit 2659
