Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 12 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph.
Regarding claim 12, this claim depends on itself rendering the claim indefinite. For examination purposes, this claim is being interpreted as depending on claim 11. Appropriate correction is required.
Examiner’s Note
In view of the specification, examiner interprets the phrase “intelligently identify” as identifying with use of some machine learning or artificial intelligence. The examiner notes that while the plain meaning of this term claim may be relative and subject to 112(b), the BRI in view of the specification provides definite scope to the term as discussed above and therefore no 112(b) rejection is warranted.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 3-6, 10-11, 13, 16, and 18 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Kotriwala et al (US 20230214724 A1), hererafter Kotriwala.
Regarding claims 1, 11, and 16, Kotriwala teaches deploying one or more Artificial Intelligence (AI)-based unlearning data identification agents within a plurality of virtual assistant services included in a virtual assistant platform to continuously crawl the plurality of virtual assistant services to intelligently identify first data sets that meet previously-learned unlearning data criteria (Machine learning system 100 processes data 104 to identify data that meet unlearning data criteria, P0058, figure 5. Identification is performed by operator 112, which may be an expert system such as a machine learning model, P0058, P0060); receiving, at one or more unlearning algorithms that include a plurality of unlearning rules, one or more first data sets from the AI-based unlearning data identification agents that meet the previously-learned unlearning data criteria (Machine learning system 100 receives data 104 that meets the unlearning data criteria, P0058, figure 5); determining one or more of the plurality of unlearning rules that are applicable to the received one or more first data sets (In the example shown, three data samples are determined to fit the criteria, P0058, figure 5); and retraining one or more first ML-models included in the virtual assistant platform and configured to process and respond to user-inputted queries, wherein retraining includes unlearning data related to or included within the identified one or more first data sets by applying the one or more determined unlearning rules (Data samples that meet the criteria are passed to machine unlearning unit 122 to be unlearnt based on the rule defined in explanation 110, P0058. The process of unlearning data includes applying rules such as determining whether a similarity is above or below a set threshold, P0046-P0048, P0050. End user 112 may provide feedback to the machine learning model after receiving an output in order for the machine learning model to then update the output in accordance with the feedback provided by end user 112, P0025, P0059).
Regarding claim 3, Kotriwala teaches the limitations of claim 1 as outlined above. Kotriwala further teaches wherein the one or more AI-based unlearning data identification agents include a principled data identification agent configured to intelligently identify first data sets that meet the previously-learned unlearning data criteria related to principled considerations impacting an entity controlling the virtual assistant (The decision to unlearn a data sample or not impacts end-user 112 which may or may not be a machine learning model providing feedback for unlearning data, P0058-P0060. Similarity between a sample and a previously rejected sample may be compared, and if the similarity is above a threshold, the sample may be a good candidate for unlearning, P0041, P0050).
Regarding claim 4, Kotriwala teaches the limitations of claim 1 as outlined above. Kotriwala further teaches wherein the one or more AI-based unlearning data identification agents include a stale data identification agent configured to intelligently identify first data sets that meet the previously-learned unlearning data criteria related to at least one of (i) outdated information and (ii) obsolete information (Machine learning models may recognize when changes may have been implemented over time, but the older rules for unlearning data may not reflect the newer reality, P0027).
Regarding claims 5, 13, and 18, Kotriwala teaches the limitations of claims 1, 11, and 16 as outlined above. Kotriwala further teaches wherein the one or more AI-based unlearning data identification agents are further configured to continuously crawl the one or more ML models to intelligently identify second data sets that meet previously-learned unlearning data criteria (Machine learning models may recognize when changes may have been implemented over time, but the older rules for unlearning data may not reflect the newer reality, P0027) and wherein the at least one unlearning algorithm is further configured to (i) receive, from the one or more ML models, one or more second data sets that meet the previously-learned unlearning data criteria (Machine learning models may repeat steps 401-404 for a number of samples, which includes obtaining data, P0031-P0039), (ii) determine one or more of the unlearning rules that are applicable to the received one or more second data sets (If the similarity of the sample compared to a separate unlearned data sample is above a threshold, the sample is a candidate for being unlearned, P0036-P0037), and (iii) retrain the one or more first ML-models to unlearn data related to or included within the identified one or more second data sets by applying the one or more determined unlearning rules (Machine unlearning unit 122 may retrain the machine learning model. Retraining my include applying unlearning rules, P0045-P0050, P0052).
Regarding claim 6, Kotriwala teaches the limitations of claim 1 as outlined above. Kotriwala further teaches wherein the at least one unlearning algorithm comprises one or more second Machine-Learning (ML) models trained to re-train the first ML models to unlearn the data related to or included within the identified first data sets by applying the one or more determined unlearning rules (Machine unlearning unit 122 may retrain the machine learning model. Retraining my include applying unlearning rules, P0045-P0050, P0052).
Regarding claim 10, Kotriwala teaches the limitations of claim 1 as outlined above. Kotriwala further teaches wherein the at least one unlearning algorithm is further configured to: receive, from a secondary entity, one or more second data sets that include data requiring unlearning (Machine learning models may repeat steps 401-404 for a number of samples, which includes obtaining data, P0031-P0039. Data may be obtained by an end user/operator 112, P0026), wherein the secondary entity comprises one chosen from the group consisting of (i) a data analyst and (ii) an unlearning data identification algorithm (The end user/operator 112 who evaluates data may be a domain expert, P0026, P0058, P0003, or be a system implemented as a machine learning model, P0060), determine one or more of the unlearning rules that are applicable to the identified one or more second data sets (If the similarity of the sample compared to a separate unlearned data sample is above a threshold, the sample is a candidate for being unlearned, P0036-P0037), and retrain the one or more first ML-models to unlearn data related to or included within the identified second data sets by applying the one or more determined unlearning rules (Machine unlearning unit 122 may retrain the machine learning model. Retraining my include applying unlearning rules, P0045-P0050, P0052).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 2, 12, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Kotriwala in view of Shi et al (Pub. No.: US 20250165863 A1), hereafter Shi.
Regarding claim 2, Kotriwala teaches the limitations of claim 1 as outlined above. Kotriwala does not appear to explicitly teach “wherein the one or more AI-based unlearning data identification agents include a bias data identification agent configured to intelligently identify first data sets that meet the previously-learned unlearning data criteria related to social identity bias”.
Shi teaches wherein the one or more AI-based unlearning data identification agents include a bias data identification agent configured to intelligently identify first data sets that meet the previously-learned unlearning data criteria related to social identity bias (During machine unlearning, sensitive information including information that includes biases may be unlearned in order to retrain models in an ethical manner, P0003, P0030, P0031).
Accordingly, it would have been obvious to a person having ordinary skill in the
art before the effective filing date of the claimed invention, having the teachings of
Kotriwala and Shi before them, to include Shi’s specific teaching of unlearning biased data in order to retrain models in Kotriwala’s system of Removing Undesirable Inferences From A Machine Learning Model. One would have been motivated to make such a combination of unlearning biased data in order to retrain models (see Shi P0003, P0030, P0031), and retraining models by applying unlearning rules to data (see Kotriwala P0045-P0050, P0052) to train more ethical models (see Shi P0003).
Regarding claims 12 and 17, Kotriwala teaches the limitations of claim 11 and 16 as outlined above. Kotriwala teaches deploying one or more AI-based unlearning data identification agents further comprises deploying the one or more AI-based unlearning data identification agents including at least one chosen from the group consisting of… (ii) an principled data identification agent configured to intelligently identify first data sets that meet the previously-learned unlearning data criteria related to principled considerations impacting an entity controlling the virtual assistant (The decision to unlearn a data sample or not impacts end-user 112 which may or may not be a machine learning model providing feedback for unlearning data, P0058-P0060. Similarity between a sample and a previously rejected sample may be compared, and if the similarity is above a threshold, the sample may be a good candidate for unlearning, P0041, P0050), and (iii) a stale data identification agent configured to intelligently identify first data sets that meet the previously-learned unlearning data criteria related to at least one of (a) outdated information and (b) obsolete information (Machine learning models may recognize when changes may have been implemented over time, but the older rules for unlearning data may not reflect the newer reality, P0027). Kotriwala does not appear to explicitly teach “(i) a bias data identification agent configured to intelligently identify first data sets that meet the previously-learned unlearning data criteria related to social identity bias”.
Shi teaches (i) a bias data identification agent configured to intelligently identify first data sets that meet the previously-learned unlearning data criteria related to social identity bias (During machine unlearning, sensitive information including information that includes biases may be unlearned in order to retrain models in an ethical manner, P0003, P0030, P0031).
Accordingly, it would have been obvious to a person having ordinary skill in the
art before the effective filing date of the claimed invention, having the teachings of
Kotriwala and Shi before them, to include Shi’s specific teaching of unlearning biased data in order to retrain models in Kotriwala’s system of Removing Undesirable Inferences From A Machine Learning Model. One would have been motivated to make such a combination of unlearning biased data in order to retrain models (see Shi P0003, P0030, P0031), and retraining models by applying unlearning rules to data (see Kotriwala P0045-P0050, P0052) to train more ethical models (see Shi P0003).
Claims 7-9, 14-15, 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Kotriwala in view of Miserendino, JR. et al (Pub. No.: US 20160260023 A1), hereafter Miserendino.
Regarding claims 7, 14, and 19, Kotriwala teaches the limitations of claims 1, 11, and 16 as outlined above. Kotriwala does not appear to explicitly teach “wherein the at least one unlearning algorithm is further configured to publish the data related to or included within the identified first data sets to one or more system of records (SORs), wherein the one or more systems of record include at least one of (i) governance SOR, (ii) data library SOR and (iii) context and intents SOR”.
Miserendino teaches wherein the at least one unlearning algorithm is further configured to publish the data related to or included within the identified first data sets to one or more system of records (SORs), wherein the one or more systems of record include at least one of (i) governance SOR, (ii) data library SOR and (iii) context and intents SOR (Digital object library management system may be used to store data for machine learning applications, P0023, P0033, P0040).
Accordingly, it would have been obvious to a person having ordinary skill in the
art before the effective filing date of the claimed invention, having the teachings of
Kotriwala and Miserendino before them, to include Miserendino’s specific teaching of including a digital object library management system in Miserendino’s system of Removing Undesirable Inferences From A Machine Learning Model. One would have been motivated to make such a combination of including a digital object library management system (see Miserendino P0023, P0033, P0040), and retraining models by applying unlearning rules to data (see Kotriwala P0045-P0050, P0052) to improve access of data during model training (see Miserendino P0023).
Regarding claims 8, 15, and 20, Kotriwala in view of Miserendino teaches the limitations of claims 7, 14, and 19 as outlined above. Kotriwala further teaches wherein the one or more AI-based unlearning data identification agents are further configured to continuously crawl the one or more SORs to intelligently identify second data sets that meet previously-learned unlearning data criteria (Machine learning models may recognize when changes may have been implemented over time, but the older rules for unlearning data may not reflect the newer reality, P0027) and wherein the at least one unlearning algorithm is further configured to (i) receive, from the one or more SORs, one or more second data sets that meet the previously-learned unlearning data criteria (Machine learning models may repeat steps 401-404 for a number of samples, which includes obtaining data, P0031-P0039), (ii) determine one or more of the unlearning rules that are applicable to the received one or more second data sets (If the similarity of the sample compared to a separate unlearned data sample is above a threshold, the sample is a candidate for being unlearned, P0036-P0037), and (iii) retrain the one or more first ML-models to unlearn data related to or included within the identified one or more second data sets by applying the one or more determined unlearning rules (Machine unlearning unit 122 may retrain the machine learning model. Retraining my include applying unlearning rules, P0045-P0050, P0052).
Regarding claim 9, Kotriwala in view of Miserendino teaches the limitations of claim 8 as outlined above. Miserendino further teaches wherein at least one of the one or more SOR are relied upon for training new first ML models included within the virtual assistant platform (Digital object library management system may be relied upon for training machine learning models, P0008, P0014-P0015).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20230102846 A1 (Hyland) teaches a compact surveillance system including a machine learning algorithm designed to cleanse data stored in sets of recorded data.
US 20240070525 A1 (Sun et al) teaches a system of unlearning recommendation models.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ISHAN MOUNDI whose telephone number is (703)756-1547. The examiner can normally be reached 8:30 A.M. - 5 P.M..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Ell can be reached at (571) 270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/I.M./Examiner, Art Unit 2141
/MATTHEW ELL/Supervisory Patent Examiner, Art Unit 2141
Prosecution Insights