DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This non-final office action is in response to the RCE filed 6 February 2026 and the amendment filed 2 January 2026.
Claims 1-6 and 13-18 are pending. Claims 1 and 13 are independent claims.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-3 and 13-15 are rejected under 35 U.S.C. 103 as being unpatentable over Cao et al. (Efficient Repair of Polluted Machine Learning Systems via Casual Unlearning, 2018, hereafter Cao) and further in view of Hackett (US 2020/0387834, filed 5 June 2019).
As per independent claim 1, Cao discloses a method for deleting training data used for a deep learning model, the method comprising:
calculating a result value for a label allocated to data to be deleted, wherein the data to be deleted is part of trained data used to train the deep learning model (pages 736: Here, a set of polluted data is identified via oracle. This oracle data and the resulting labels are compared via a divergence score (page 738). Based upon similarity, it is determined that these data samples are also polluted by training data that is incorrected)
reallocating the label of the data to be deleted based on the calculated result value (pages 737-738: Here, the misclassified data items are grouped into clusters (page 739). These clusters are initially identified as polluted data. The peak of the divergence score is identified and data items outside the peak are maintained within the dataset while those within the peak are identified as polluted and re-labeled and/or removed)
generating a neutral model by neutralizing the deep learning model with the data to be deleted and the reallocated label of the data to be deleted as inputs (page 738: Here, a neutral model is created by removing the subset of training data. Based upon the accuracy of the classification after removing this data, the removed subset is determined to be polluted)
training the neutral model using the trained data, excluding the data to be deleted (pages 735-736: Here, misclassified samples are removed from the training set to compute a new model and determine whether the new model correctly classifies previously misclassified samples. This unlearning is performed by removing the polluted data and retraining the model (pages 740-741))
Cao fails to specifically disclose wherein the deep learning model is neutralized until an accuracy of the learning model becomes a threshold value or lower.
However, Hackett, which is analogous to the claimed invention because it is directed toward training a machine learning model, discloses wherein the deep learning model is neutralized until an accuracy of the learning model becomes a threshold value or lower (paragraph 0065: Here, an ensemble model is trained by breaking the data into a plurality of subsets (Figure 1, item 104) and training the models in parallel (Figure 1, item 105). Based upon a determination that the error between a trained model and the ensemble model is below a threshold, the trained model is complete and deployed for use (no longer neutralized)).
It would have been obvious to one of ordinary skill in the art at the time of the applicant’s effective filing date to have combined Hackett with Cao, with a reasonable expectation of success, as it would have allowed for deploying a newly trained model only when the difference between the newly trained model and the accepted previous model (ensemble model) falls below a threshold (Hackett: paragraph 0065).
As per dependent claim 2, Cao and Hackett discloses the limitations similar to those in claim 1, and the same rejection is incorporated herein. Cao further discloses averaging result values calculated for each label associated with the data to be deleted (page 740-741: Here, polluted clusters are removed until one of two stopping conditions are met. The first stopping condition is that the accuracy of the learning model meets the expected accuracy value of the administrator. The second stopping condition is that no new clusters can be found to be unlearned. Both of these stopping conditions rely upon the average result values for data to be deleted to fall below a threshold. Specifically, the average result value of a cluster is no longer identified as being polluted or the average result value of the data set as a whole does not affect the accuracy of the model).
As per dependent claim 3, Cao and Hackett discloses the limitations similar to those in claim 1, and the same rejection is incorporated herein. Cao further discloses wherein the reallocating includes:
identifying an object label having a lowest result value, among calculated result values of the label associated with the data to be deleted (pages 738-739: Here, during a searching sub-phase, a divergence score is calculated and maintained. The divergence score represents the divergence between a labeled object and objects identified as being polluted. If the divergence between an object and a polluted object is less than the divergence minimum (object label having a lowest result value), then the object is identified as being polluted)
reallocating the identified object label as a label of the data to be deleted when the object label is not the same as a previously allocated label of the data to be deleted (pages 738-740: Here, objects that are identified as belonging to a cluster based upon their divergence score are analyzed. This includes processing the cluster to identify items that are polluted, and unlearn the associated object labels)
With respect to claims 13-15, the applicant discloses the limitations substantially similar to those in claims 1-3, respectively. Claims 13-15 are similarly rejected.
Claims 4 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Cao and Hackett, and further in view of Achan et al. (US 2022/0215453, filed 5 January 2021, hereafter Achan).
As per dependent claim 4, Cao and Hackett discloses the limitations similar to those in claim 1, and the same rejection is incorporated herein. Cao further discloses wherein the generating of a neutral model includes:
training the deep learning model using the data to be deleted and a label to which the data to be deleted is reallocated (pages 735-736: Here, a model is trained using training data including malicious polluted data. The polluted data is identified and removed from the training set to create a new neutral model, without the polluted training data)
calculating an accuracy for the data to be deleted (pages 738-740)
stopping the training of the deep learning model learning and generating a neutral model (page 737: Here, based upon reported misclassifications, polluted data is identified and removed. The model is then retrained)
Cao fails to specifically disclose when the accuracy is equal to or lower than a predetermined threshold value. However, Achan, which is analogous to the claimed invention because it is directed toward removing polluted data from a training set, discloses determining when the accuracy is equal to or lower than a predetermined threshold value (paragraph 0058: Here, when the accuracy of one or more values falls below a predetermined threshold, it is determined that that the session data includes polluted data). It would have been obvious to one of ordinary skill in the art at the time of the applicant’s effective filing date to have combined Achan with Cao-Hackett, with a reasonable expectation of success, as it would have allowed for triggering retraining based upon a polluted data threshold (Achan: paragraph 0058). This would have allowed for only retraining in the event that a threshold is reached, thereby avoiding training until it is determined to be an attack.
With respect to claim 16, the applicant discloses the limitations substantially similar to those in claim 4. Claim 16 is similarly rejected.
Claims 5 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Cao, Hackett, and Achan and further in view of Zwol et al. (US 2012/0158716, published 21 June 2012, hereafter Zwol).
As per dependent claim 5, Cao, Hackett, and Achan disclose the limitations similar to those in claim 4, and the same rejection is incorporated herein. Cao fails to specifically disclose wherein the threshold value is a reciprocal number of the number of labels allocated to the data to be deleted.
However, Zwol discloses wherein the threshold value is a reciprocal number of the number of items to be removed (paragraph 0072: Here, the threshold for removing images from consideration is selected based upon a reciprocal rank of the item). It would have been obvious to one of ordinary skill in the art at the time of the applicant’s effective filing date to have combined Zwol with Cao-Achan, with a reasonable expectation of success, as it would have allowed for removing items below a threshold (Zwol: paragraph 0072).
With respect to claim 17, the applicant discloses the limitations substantially similar to those in claim 5. Claim 17 is similarly rejected.
Claims 6 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Cao and Hackett and further in view of Fink et al. (US 2022/0109654, filed 7 October 2020, hereafter Fink).
As per dependent claim 6, Cao and Hackett discloses the limitations similar to those in claim 1, and the same rejection is incorporated herein. Cao fails to specifically disclose wherein the training includes training the neutralized model using a knowledge distillation technique in which the deep learning model serves as a teacher and the neutral model serves as a student. However, Fink, which is analogous to the claimed invention because it is directed toward knowledge distillation, discloses wherein the training includes training the neutralized model using a knowledge distillation technique in which the deep learning model serves as a teacher and the neutral model serves as a student (paragraph 0009). It would have been obvious to one of ordinary skill in the art at the time of the applicant’s effective filing date to have combined Fink with Cao, with a reasonable expectation of success, as it would have allowed for addressing size and processing requirements of training a model (Fink: paragraph 0009).
With respect to claim 18, the applicant discloses the limitations substantially similar to those in claim 6. Claim 18 is similarly rejected.
Response to Arguments
Applicant’s arguments have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Cao and Hackett.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Beandre et al. (US 2018/0322417): Discloses an ML trainer that is deployed when performance metrics are below a threshold level (paragraph 0153)
Beaver et al. (US 2020/0311346): Discloses deleting items from a training data set if the frequency falls below a predetermined threshold (paragraph 0007)
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KYLE R STORK whose telephone number is (571)272-4130. The examiner can normally be reached 8am - 2pm; 4pm - 6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Omar Fernandez Rivas can be reached at 571/272-2589. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KYLE R STORK/Primary Examiner, Art Unit 2128