DETAILED ACTION
Pending Claims 1-5, 7-12, 14-18, and 20 have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 U.S.C. § 101
35 U.S.C. § 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
The invention, as taught in Claims 1-5, 7-12, 14-18, and 20, is directed to “mental steps” and “mathematical steps” without significantly more.
The claims recite:
• identifying a sample data point (i.e., mental steps)
• provides a first prediction (i.e., determines) with a corresponding first explanation (i.e., mental steps) and … provides the first prediction with a corresponding second explanation (i.e., mental steps)
• the first ML model being a black box model paired with the second ML model being a local interpretable model (i.e., mental step of associating the two models together)
• the paired ML models having a common training data set (i.e., mental step of selecting a training data set)
• for each ML model, the first prediction and corresponding explanation are prediction/explanation pairs (i.e., mental steps)
• providing … , a set of prediction/explanation pairs responsive to the set of candidate samples (i.e., mental steps)
• the corresponding second explanation being a target explanation (i.e., mental steps)
• generating a set of candidate samples within a specified neighborhood of the sample data point (i.e., mental steps)
• selecting a subset of the candidate samples based on a degree of difference between candidate explanations of the set of prediction / explanation pairs provided by the first ML model and the second ML model for the first set of candidate samples (i.e., mental steps)
• adding the selected subset of samples to the common training data set to form a revised training data set (i.e., mental steps)
• provide (i.e., determine) the target explanation for the first prediction with the sample data point as input (i.e., mental steps)
Claim 1
Step 1 inquiry: Does this claim fall within a statutory category?
The preamble of the claim recites “1. A computer-implemented method comprising…” Therefore, it is a “method” (or “process”), which is a statutory category of invention. Therefore, the answer to the inquiry is: “YES.”
Step 2A (Prong One) inquiry:
Are there limitations in Claim 1 that recite abstract ideas?
YES. The following limitations in Claim 1 recite abstract ideas that fall within at least one of the groupings of abstract ideas enumerated in the 2019 PEG. Specifically, they are “mental steps” and “mathematical steps”:
• identifying a sample data point (i.e., mental steps)
• provides a first prediction (i.e., determines) with a corresponding first explanation (i.e., mental steps) and … provides the first prediction with a corresponding second explanation (i.e., mental steps)
• the first ML model being a black box model paired with the second ML model being a local interpretable model (i.e., mental step of associating the two models together)
• the paired ML models having a common training data set (i.e., mental step of selecting a training data set)
• for each ML model, the first prediction and corresponding explanation are prediction/explanation pairs (i.e., mental steps)
• providing … , a set of prediction/explanation pairs responsive to the set of candidate samples (i.e., mental steps)
• the corresponding second explanation being a target explanation (i.e., mental steps)
• generating a set of candidate samples within a specified neighborhood of the sample data point (i.e., mental steps)
• selecting a subset of the candidate samples based on a degree of difference between candidate explanations of the set of prediction / explanation pairs provided by the first ML model and the second ML model for the first set of candidate samples (i.e., mental steps)
• adding the selected subset of samples to the common training data set to form a revised training data set (i.e., mental steps)
• provide (i.e., determine) the target explanation for the first prediction with the sample data point as input (i.e., mental steps)
Step 2A (Prong Two) inquiry:
Are there additional elements or a combination of elements in the claim that apply, rely on, or use the judicial exception in a manner that imposes a meaningful limit on the judicial exception, such that it is more than a drafting effort designed to monopolize the exception?
Applicant’s claims contain the following “additional elements”:
(1) A “first machine learning (ML) model”/“second ML model”/“retraining the first ML model by using the revised training dataset”
(1) A “first machine learning (ML) model”/“second ML model”/“retraining the first ML model by using the revised training dataset” is a broad term which describes the mere idea of a model instead of any particular model or any improvement to that model. It is a description at a high level. M.P.E.P. § 2016.05(f) recites:
2106.05(f) Mere Instructions To Apply An Exception [R-10.2019]
Another consideration when determining whether a claim integrates a judicial exception into a practical application in Step 2A Prong Two or recites significantly more than a judicial exception in Step 2B is whether the additional elements amount to more than a recitation of the words “apply it” (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer. As explained by the Supreme Court, in order to make a claim directed to a judicial exception patent-eligible, the additional element or combination of elements must do “‘more than simply stat[e] the [judicial exception] while adding the words ‘apply it’”. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, for example, claims that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not render an abstract idea eligible. Alice Corp., 573 U.S. at 223, 110 USPQ2d at 1983. See also 573 U.S. at 224, 110 USPQ2d at 1984 (warning against a § 101 analysis that turns on “the draftsman’s art”).
The answer to the inquiry is “NO”, no additional elements integrate the claimed abstract idea into a practical application.
Step 2B inquiry:
Does the claim provide an inventive concept, i.e., does the claim recite additional element(s) or a combination of elements that amount to significantly more than the judicial exception in the claim?
Applicant’s claims contain the following “additional elements”:
(1) A “first machine learning (ML) model”/“second ML model”
(2) A “retraining” of “the first ML model by using the revised training dataset”
A “first machine learning (ML) model”/“second ML model” is a broad term which is described at a high level. Applicant’s Specification, paragraph [0002] recites:
[0002] A local interpretable model-agnostic explanation (LIME) is a local model interpretation technique that approximates any black box machine learning model with a local surrogate model to explain each individual prediction of the underlying black box model. Local surrogate models are interpretable models such as linear regression or decision tree models that are used to explain the individual predictions of a black box model. LIME trains the local surrogate model by generating a new dataset from the data point of interest with the data type being, for example, text, image, or tabular data.
Therefore, the claim as a whole does not amount to significantly more than the exception itself (i.e., there is no inventive concept in the claim). (See, M.P.E.P. § 2106.05(II)).
A “retraining the first ML model by using the revised training dataset” is a broad term which is described at a high level. Applicant’s Specification, paragraph [0033] recites:
[0033] Some embodiments of the present invention are directed to repairing a target machine learning model, M1, so that a decision, c1, made by the target model in the decision/explanation pair (c1,Exp1) is, instead, based on a favored explanation, exp2, provided by machine learning model, M2, making the same decision expressed as the decision/explanation pair (c2,Exp2). Upon repairing the target machine learning model, the decision/explanation pair for the model M1 would be (c1,Exp2). Several assumptions may apply to effective operation of some embodiments of the present invention. Assumptions may include: (i) the set of training data of the target model M1 is available; (ii) if the target model M1 is black-box model, it provides a retrain API with input data; (iii) if the target model M1 is a white box model, it has updateable weights; (iv) the machine learning model M2 provides a local model (i.e., provides the explanation of the outcome of sample s) defined by ExpM2,s around the sample s. The local model is interpretable, such as a lasso model, a logistic regression model, or a decision tree model.
In the broadest reasonable interpretation, the model is a generic black-box model. The retraining of a generic black-box model is also generic.
Therefore, the claim as a whole does not amount to significantly more than the exception itself (i.e., there is no inventive concept in the claim). (See, M.P.E.P. § 2106.05(II)).
Therefore, the answer to the inquiry is “NO”, no additional elements provide an inventive concept that is significantly more than the claimed abstract ideas the claimed abstract idea into a practical application.
Claim 1 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 2
Claim 2 recites:
2. The computer-implemented method of claim 1, wherein selecting the subset of the candidate samples is further based on: a closeness value comparing the sample data point to each candidate sample being above a threshold level of closeness.
Applicant’s Claim 2 merely teaches a calculated threshold parameter. It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 2 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 3
Claim 3 recites:
3. The computer-implemented method of claim 1, wherein:
the first ML model is a capable of being directly repaired; and further comprising:
constraining a gradient of the first ML model with the target explanation, causing a corresponding explanation of the first prediction to be the target explanation.
Applicant’s Claim 3 merely teaches a generic machine learning model that may be retrained and limiting the retraining result to the accuracy/gradient of the target explanation. The claim term “repaired” is interpreted to have “retrained” within its broadest reasonable interpretation. The claim term “gradient of the whitebox model,” in its broadest reasonable interpretation, means that the “whitebox model” is a “gradient descent” type model. The claim term “constraining a gradient,” in its broadest reasonable interpretation, means to match the accuracy/gradient of the model with the same explanation. It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 3 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 4
Claim 4 recites:
4. The computer-implemented method of claim 1, further comprising: selecting the second explanation as the target explanation based on user input.
Applicant’s Claim 4 merely teaches selection of an explanation, which is easily performed in the mind. It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 4 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 5
Claim 5 recites:
5. The computer-implemented method of claim 1, wherein each explanation is represented as a ranked list of input features of each model and weights respectively corresponding input features.
Applicant’s Claim 5 merely teaches a “ranked list” of input parameters (i.e., input features) and corresponding “weight” parameters (i.e., “weights”.) The broadest reasonable interpretation of these terms includes purely mathematical numbers. It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 5 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 7
Claim 7 recites:
7. The computer-implemented method of claim 1, wherein: the first model includes a retrain application programming interface (API) with input data; and the second model provides a local model around the sample data point.
Applicant’s Claim 7 merely teaches training an application programming interface (API) and generic machine learning models. This is well-understood, routine, and conventional as shown by: Qayyum, et al., page 13, right column, second full paragraph, where it recites:
5.2.8 Increasing Entropy and Reducing Precision
The training of attack using shadow training techniques against black box models in the cloud-based Google Prediction API and Amazon ML models are studied by Shokri et al. (2017). The attack does not require prior knowledge of training data distribution. The authors emphasize that in order to protect the privacy of medical related datasets or other public-related data, countermeasures should be designed. For instance, restriction of prediction vector to top k classes, which will prevent the leakage of important information or rounding down or up the classification probabilities in the prediction. They show that regularization can be effective to cope with overfitting and increasing the randomness of the prediction vector.
It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 7 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 8
Step 1 inquiry: Does this claim fall within a statutory category?
The preamble of the claim recites “8. A computer program product comprising...” Therefore, it is a “computer program product” (which is not limited to a “non-transitory computer readable medium”). The claimed “computer program product” is NOT a statutory category of invention. Therefore, the answer to the inquiry is: “NO.”
Step 2A (Prong One) inquiry:
Are there limitations in Claim 8 that recite abstract ideas?
YES. The following limitations in Claim 8 recite abstract ideas that fall within at least one of the groupings of abstract ideas enumerated in the 2019 PEG. Specifically, they are “mental steps” and “mathematical steps”:
• identifying a sample data point (i.e., mental steps)
• provides a first prediction (i.e., determines) with a corresponding first explanation (i.e., mental steps) and … provides the first prediction with a corresponding second explanation (i.e., mental steps)
• the first ML model being a black box model paired with the second ML model being a local interpretable model (i.e., mental step of associating the two models together)
• the paired ML models having a common training data set (i.e., mental step of selecting a training data set)
• for each ML model, the first prediction and corresponding explanation are prediction/explanation pairs (i.e., mental steps)
• providing … , a set of prediction/explanation pairs responsive to the set of candidate samples (i.e., mental steps)
• the corresponding second explanation being a target explanation (i.e., mental steps)
• generating a set of candidate samples within a specified neighborhood of the sample data point (i.e., mental steps)
• selecting a subset of the candidate samples based on a degree of difference between candidate explanations of the set of prediction / explanation pairs provided by the first ML model and the second ML model for the first set of candidate samples (i.e., mental steps)
• adding the selected subset of samples to the common training data set to form a revised training data set (i.e., mental steps)
• provide (i.e., determine) the target explanation for the first prediction with the sample data point as input (i.e., mental steps)
Step 2A (Prong Two) inquiry:
Are there additional elements or a combination of elements in the claim that apply, rely on, or use the judicial exception in a manner that imposes a meaningful limit on the judicial exception, such that it is more than a drafting effort designed to monopolize the exception?
Applicant’s claims contain the following “additional elements”:
(1) A “first machine learning (ML) model”/“second ML model”/ “retraining the first ML model by using the revised training dataset”
A “first machine learning (ML) model”/“second ML model”/ “retraining the first ML model by using the revised training dataset” is a broad term which describes the mere idea of a model instead of any particular model or any improvement to that model. It is a description at a high level. M.P.E.P. § 2016.05(f) recites:
2106.05(f) Mere Instructions To Apply An Exception [R-10.2019]
Another consideration when determining whether a claim integrates a judicial exception into a practical application in Step 2A Prong Two or recites significantly more than a judicial exception in Step 2B is whether the additional elements amount to more than a recitation of the words “apply it” (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer. As explained by the Supreme Court, in order to make a claim directed to a judicial exception patent-eligible, the additional element or combination of elements must do “‘more than simply stat[e] the [judicial exception] while adding the words ‘apply it’”. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, for example, claims that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not render an abstract idea eligible. Alice Corp., 573 U.S. at 223, 110 USPQ2d at 1983. See also 573 U.S. at 224, 110 USPQ2d at 1984 (warning against a § 101 analysis that turns on “the draftsman’s art”).
The answer to the inquiry is “NO”, no additional elements integrate the claimed abstract idea into a practical application.
Step 2B inquiry:
Does the claim provide an inventive concept, i.e., does the claim recite additional element(s) or a combination of elements that amount to significantly more than the judicial exception in the claim?
Applicant’s claims contain the following “additional elements”:
(1) A “first machine learning (ML) model”/“second ML model”
(2) A “retraining” of “the first ML model by using the revised training dataset”
A “first machine learning (ML) model”/“second ML model” is a broad term which is described at a high level. Applicant’s Specification, paragraph [0002] recites:
[0002] A local interpretable model-agnostic explanation (LIME) is a local model interpretation technique that approximates any black box machine learning model with a local surrogate model to explain each individual prediction of the underlying black box model. Local surrogate models are interpretable models such as linear regression or decision tree models that are used to explain the individual predictions of a black box model. LIME trains the local surrogate model by generating a new dataset from the data point of interest with the data type being, for example, text, image, or tabular data.
Therefore, the claim as a whole does not amount to significantly more than the exception itself (i.e., there is no inventive concept in the claim). (See, M.P.E.P. § 2106.05(II)).
A “retraining the first ML model by using the revised training dataset” is a broad term which is described at a high level. Applicant’s Specification, paragraph [0033] recites:
[0033] Some embodiments of the present invention are directed to repairing a target machine learning model, M1, so that a decision, c1, made by the target model in the decision/explanation pair (c1,Exp1) is, instead, based on a favored explanation, exp2, provided by machine learning model, M2, making the same decision expressed as the decision/explanation pair (c2,Exp2). Upon repairing the target machine learning model, the decision/explanation pair for the model M1 would be (c1,Exp2). Several assumptions may apply to effective operation of some embodiments of the present invention. Assumptions may include: (i) the set of training data of the target model M1 is available; (ii) if the target model M1 is black-box model, it provides a retrain API with input data; (iii) if the target model M1 is a white box model, it has updateable weights; (iv) the machine learning model M2 provides a local model (i.e., provides the explanation of the outcome of sample s) defined by ExpM2,s around the sample s. The local model is interpretable, such as a lasso model, a logistic regression model, or a decision tree model.
In the broadest reasonable interpretation, the model is a generic black-box model. The retraining of a generic black-box model is also generic.
Therefore, the claim as a whole does not amount to significantly more than the exception itself (i.e., there is no inventive concept in the claim). (See, M.P.E.P. § 2106.05(II)).
Therefore, the answer to the inquiry is “NO”, no additional elements provide an inventive concept that is significantly more than the claimed abstract ideas the claimed abstract idea into a practical application.
Claim 8 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 9
Claim 9 recites:
9. The computer program product of claim 8, wherein selecting the subset of the candidate samples is further based on: a closeness value comparing the sample data point to each candidate sample being above a threshold level of closeness.
Applicant’s Claim 9 merely teaches a calculated threshold parameter. It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 9 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 10
Claim 10 recites:
10. The computer program product of claim 8, wherein:
the first ML model is a capable of being directly repaired; and further comprising:
constraining a gradient of the first ML model with the target explanation, causing a corresponding explanation of the first prediction to be the target explanation.
Applicant’s Claim 10 merely teaches a generic machine learning model that may be retrained and limiting the retraining result to the accuracy/gradient of the target explanation. The claim term “repaired” is interpreted to have “retrained” within its broadest reasonable interpretation. The claim term “gradient of the whitebox model,” in its broadest reasonable interpretation, means that the “whitebox model” is a “gradient descent” type model. The claim term “constraining a gradient,” in its broadest reasonable interpretation, means to match the accuracy/gradient of the model with the same explanation. It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 10 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 11
Claim 11 recites:
11. The computer program product of claim 8, further causing the processor to perform a method comprising: selecting the second explanation as the target explanation based on user input.
Applicant’s Claim 11 merely teaches selection of an explanation, which is easily performed in the mind. It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 11 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 12
Claim 12 recites:
12. The computer program product of claim 8, wherein each explanation is represented as a ranked list of input features of each model and weights respectively corresponding input features.
Applicant’s Claim 12 merely teaches a “ranked list” of input parameters (i.e., input features) and corresponding “weight” parameters (i.e., “weights”.) The broadest reasonable interpretation of these terms includes purely mathematical numbers. It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 12 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 14
Step 1 inquiry: Does this claim fall within a statutory category?
The preamble of the claim recites “14. A computer system comprising…” Therefore, it is a “system” (or “apparatus”), which is a statutory category of invention. Therefore, the answer to the inquiry is: “YES.”
Step 2A (Prong One) inquiry:
Are there limitations in Claim 14 that recite abstract ideas?
YES. The following limitations in Claim 14 recite abstract ideas that fall within at least one of the groupings of abstract ideas enumerated in the 2019 PEG. Specifically, they are “mental steps” and “mathematical steps”:
• identifying a sample data point (i.e., mental steps)
• provides a first prediction (i.e., determines) with a corresponding first explanation (i.e., mental steps) and … provides the first prediction with a corresponding second explanation (i.e., mental steps)
• the first ML model being a black box model paired with the second ML model being a local interpretable model (i.e., mental step of associating the two models together)
• the paired ML models having a common training data set (i.e., mental step of selecting a training data set)
• for each ML model, the first prediction and corresponding explanation are prediction/explanation pairs (i.e., mental steps)
• providing … , a set of prediction/explanation pairs responsive to the set of candidate samples (i.e., mental steps)
• the corresponding second explanation being a target explanation (i.e., mental steps)
• generating a set of candidate samples within a specified neighborhood of the sample data point (i.e., mental steps)
• selecting a subset of the candidate samples based on a degree of difference between candidate explanations of the set of prediction / explanation pairs provided by the first ML model and the second ML model for the first set of candidate samples (i.e., mental steps)
• adding the selected subset of samples to the common training data set to form a revised training data set (i.e., mental steps)
• provide (i.e., determine) the target explanation for the first prediction with the sample data point as input (i.e., mental steps)
Step 2A (Prong Two) inquiry:
Are there additional elements or a combination of elements in the claim that apply, rely on, or use the judicial exception in a manner that imposes a meaningful limit on the judicial exception, such that it is more than a drafting effort designed to monopolize the exception?
Applicant’s claims contain the following “additional elements”:
(1) A “first machine learning (ML) model”/“second ML model”/“retraining the first model by using a revised training dataset including the subset of the candidate samples”
A “first machine learning (ML) model”/“second ML model”/ “retraining the first ML model by using the revised training dataset” is a broad term which describes the mere idea of a model instead of any particular model or any improvement to that model. It is a description at a high level. M.P.E.P. § 2016.05(f) recites:
2106.05(f) Mere Instructions To Apply An Exception [R-10.2019]
Another consideration when determining whether a claim integrates a judicial exception into a practical application in Step 2A Prong Two or recites significantly more than a judicial exception in Step 2B is whether the additional elements amount to more than a recitation of the words “apply it” (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer. As explained by the Supreme Court, in order to make a claim directed to a judicial exception patent-eligible, the additional element or combination of elements must do “‘more than simply stat[e] the [judicial exception] while adding the words ‘apply it’”. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, for example, claims that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not render an abstract idea eligible. Alice Corp., 573 U.S. at 223, 110 USPQ2d at 1983. See also 573 U.S. at 224, 110 USPQ2d at 1984 (warning against a § 101 analysis that turns on “the draftsman’s art”).
The answer to the inquiry is “NO”, no additional elements integrate the claimed abstract idea into a practical application.
Step 2B inquiry:
Does the claim provide an inventive concept, i.e., does the claim recite additional element(s) or a combination of elements that amount to significantly more than the judicial exception in the claim?
Applicant’s claims contain the following “additional elements”:
(1) A “first machine learning (ML) model”/“second ML model”
(2) A “retraining” of “the first ML model by using the revised training dataset”
A “first machine learning (ML) model”/“second ML model” is a broad term which is described at a high level. Applicant’s Specification, paragraph [0002] recites:
[0002] A local interpretable model-agnostic explanation (LIME) is a local model interpretation technique that approximates any black box machine learning model with a local surrogate model to explain each individual prediction of the underlying black box model. Local surrogate models are interpretable models such as linear regression or decision tree models that are used to explain the individual predictions of a black box model. LIME trains the local surrogate model by generating a new dataset from the data point of interest with the data type being, for example, text, image, or tabular data.
Therefore, the claim as a whole does not amount to significantly more than the exception itself (i.e., there is no inventive concept in the claim). (See, M.P.E.P. § 2106.05(II)).
A “retraining the first ML model by using the revised training dataset” is a broad term which is described at a high level. Applicant’s Specification, paragraph [0033] recites:
[0033] Some embodiments of the present invention are directed to repairing a target machine learning model, M1, so that a decision, c1, made by the target model in the decision/explanation pair (c1,Exp1) is, instead, based on a favored explanation, exp2, provided by machine learning model, M2, making the same decision expressed as the decision/explanation pair (c2,Exp2). Upon repairing the target machine learning model, the decision/explanation pair for the model M1 would be (c1,Exp2). Several assumptions may apply to effective operation of some embodiments of the present invention. Assumptions may include: (i) the set of training data of the target model M1 is available; (ii) if the target model M1 is black-box model, it provides a retrain API with input data; (iii) if the target model M1 is a white box model, it has updateable weights; (iv) the machine learning model M2 provides a local model (i.e., provides the explanation of the outcome of sample s) defined by ExpM2,s around the sample s. The local model is interpretable, such as a lasso model, a logistic regression model, or a decision tree model.
In the broadest reasonable interpretation, the model is a generic black-box model. The retraining of a generic black-box model is also generic.
Therefore, the claim as a whole does not amount to significantly more than the exception itself (i.e., there is no inventive concept in the claim). (See, M.P.E.P. § 2106.05(II)).
Therefore, the answer to the inquiry is “NO”, no additional elements provide an inventive concept that is significantly more than the claimed abstract ideas the claimed abstract idea into a practical application.
Claim 14 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 15
Claim 15 recites:
15. The computer system of claim 14, wherein selecting the subset of the candidate samples is further based on: a closeness value comparing the sample data point to each candidate sample being above a threshold level of closeness.
Applicant’s Claim 15 merely teaches a calculated threshold parameter. It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 15 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 16
Claim 16 recites:
16. The computer system of claim 14, wherein:
the first ML model is capable of being directly repaired; and further comprising:
constraining a gradient of the first ML model with the target explanation, causing a corresponding explanation of the first prediction to be the target explanation.
Applicant’s Claim 16 merely teaches a generic machine learning model that may be retrained and limiting the retraining result to the accuracy/gradient of the target explanation. The claim term “repaired” is interpreted to have “retrained” within its broadest reasonable interpretation. The claim term “gradient of the whitebox model,” in its broadest reasonable interpretation, means that the “whitebox model” is a “gradient descent” type model. The claim term “constraining a gradient,” in its broadest reasonable interpretation, means to match the accuracy/gradient of the model with the same explanation. It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 16 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 17
Claim 17 recites:
17. The computer system of claim 14, further causing the processor set to perform a method comprising: selecting the second explanation as the target explanation based on user input.
Applicant’s Claim 17 merely teaches selection of an explanation, which is easily performed in the mind. It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 17 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 18
Claim 18 recites:
18. The computer system of claim 14, wherein each explanation is represented as a ranked list of input features of each model and weights respectively corresponding input features.
Applicant’s Claim 18 merely teaches a “ranked list” of input parameters (i.e., input features) and corresponding “weight” parameters (i.e., “weights”.) The broadest reasonable interpretation of these terms includes purely mathematical numbers. It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 18 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 20
Claim 20 recites:
20. The computer system of claim 14, wherein: the first model includes a retrain application programming interface (API) with input data; and the second model provides a local model around the sample data point.
Applicant’s Claim 20 merely teaches the training of an application programming interface (API) and generic machine learning models. This is well-understood, routine, and conventional as shown by: Qayyum, et al., page 13, right column, second full paragraph, where it recites:
5.2.8 Increasing Entropy and Reducing Precision
The training of attack using shadow training techniques against black box models in the cloud-based Google Prediction API and Amazon ML models are studied by Shokri et al. (2017). The attack does not require prior knowledge of training data distribution. The authors emphasize that in order to protect the privacy of medical related datasets or other public-related data, countermeasures should be designed. For instance, restriction of prediction vector to top k classes, which will prevent the leakage of important information or rounding down or up the classification probabilities in the prediction. They show that regularization can be effective to cope with overfitting and increasing the randomness of the prediction vector.
It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 20 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claims 1-20 are not rejected under art because when reading the claims in light of the specification, as per MPEP § 2111.01, none of the references of record, whether taken alone or in combination, discloses or suggests the combination of limitations specified in independent Claim 1. Specifically:
Claim 1’s "...difference between candidate explanations..."
Claim 1’s "...candidate samples within a specified neighborhood of the sample data point..."
Further, none of the references of record, whether taken alone or in combination, discloses or suggests the combination of limitations specified in independent Claim 8. Specifically:
Claim 8’s "...difference between candidate explanations..."
Claim 8’s "...candidate samples within a specified neighborhood of the sample data point..."
Further, none of the references of record, whether taken alone or in combination, discloses or suggests the combination of limitations specified in independent Claim 14. Specifically:
Claim 14’s "...difference between candidate explanations..."
Claim 14’s "...candidate samples within a specified neighborhood of the sample data point..."
Response to Arguments
Applicant's arguments filed 24 MAR 2026 have been fully considered but they are not persuasive. Specifically, Applicant argues:
Argument 1
Further in this Amendment, clerical amendments are made to the claims, specifically claims 3, 7, 10, 16, and 20. These clerical amendments are made for clarity and continuity based on amendments to claims 1, 8, and 14. Applicant is not conceding that the subject matter encompassed by the claims prior to this Amendment is unpatentable over the art cited by the Examiner. Applicant respectfully reserves the right to pursue claims in one or more continuing applications, which may include claims capturing the subject matter encompassed by claims 3, 7, 10, 16, and 20 prior to this Amendment.
The clerical amendments are noted.
The rejections stand.
Argument 2
Allowable Subject Matter
Claims 1-5, 7-12, 14-18, and 20 are not rejected under art because when reading the claims in light of the specification, as per MPEP § 2111.01, none of the references of record, whether taken alone or in combination, discloses or suggests the combination of limitations specified in independent claims 1, 8, and 14.
The pending claims are rejected under 35 U.S.C § 101. Although they are not rejected under the art of record, they are not allowable because they are rejected under 35 U.S.C § 101.
Applicant’s argument is unpersuasive.
The rejections stand.
Argument 3
Meaningful Limitation of Judicial Exception
The claims, as amended, introduce a "combination of elements" that operate to cause a first ML model, trained with a common training dataset as a second ML model, to adopt a target explanation provided by the second ML model responsive to a sample data point as input. The first ML model is a black box model, which is paired with a second ML model, which is a local interpretable model, each trained on the common training dataset.
Applicant’s argument simply teaches two machine learning models that are trained on the same data set and are associated with the same explanation. Simply applying unspecified abstract data to a pair of generic machine learning models adds nothing to the original abstract ideas. And the common explanation is simply mental steps.
Applicant’s argument is unpersuasive.
The rejections stand.
Argument 4
For at least the reason that the claimed invention "compares explanations from two commonly trained, but different, models" and applies "both sample generation and subset selection strategy" to establish "a revised training dataset" that causes "the first model to provide the target explanation for the first prediction with the sample data point as input," the claimed invention is directed to a combination of elements that "apply, rely on, or use the judicial exception" of mental steps and mathematics to impose "a meaningful limit on the judicial exception."
The argued combination of elements is a combination of abstract ideas that are mental steps and mathematical steps. None of these elements is part of the training or design of the machine learning systems themselves. Therefore, the elements are not improvements on the machine learning systems. For instance, the comparison of explanations is a mental step. Sample generation may be either mental or mathematical. Subset selection may be mental or mathematical as well.
Applicant’s argument is unpersuasive.
The rejections stand.
Argument 5
Improvement to Computer Functionality
The Court of Appeals for the Federal Circuit held in Enfish that claims focused on a specific improvement to computer functionality are not directed to an abstract idea at step one of the Alice/Mayo inquiry. Consistent with Enfish, the present claims are directed to improving the operation of a computer-implemented explanation system including a first black box ML model and a second local interpretable model by repairing "feature attribution based explanations" of the first black box ML model and by "modif[ying] and/or repair[ing] explanations that a user can trust for specific business processes" (see paragraph 56 of the Specification). Under the Enfish analysis, the claims are not directed to an abstract idea.
The modification or the repair of explanations is a mental step. Human beings explain things every day. Oftentimes they modify those explanations. Implementing such an abstract idea on a generic computer does not add anything significantly more to the abstract idea.
Applicant’s arguments are unpersuasive.
The rejections stand.
Argument 6
It is respectfully submitted that the claims 1-5, 7-12, 14-18, and 20 have been amended to be clearly directed to patent-eligible subject matter, and that claims 1-5, 7-12, 14-18, and 20, as amended, are allowable under the standards of 35 USC 101.
Applicant’s arguments were unpersuasive.
The rejections stand.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiries concerning this communication or earlier communications from the examiner should be directed to Wilbert L. Starks, Jr., who may be reached Monday through Friday, between 8:00 a.m. and 5:00 p.m. EST. or via telephone at (571) 272-3691 or email: Wilbert.Starks@uspto.gov.
If you need to send an Official facsimile transmission, please send it to (571) 273-8300.
If attempts to reach the examiner are unsuccessful the Examiner’s Supervisor (SPE), Kakali Chaki, may be reached at (571) 272-3719.
Hand-delivered responses should be delivered to the Receptionist @ (Customer Service Window Randolph Building 401 Dulany Street, Alexandria, VA 22313), located on the first floor of the south side of the Randolph Building.
Finally, information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Moreover, status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have any questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) toll-free @ 1-866-217-9197.
/WILBERT L STARKS/
Primary Examiner, Art Unit 2122
WLS
14 JUL 2026
Prosecution Insights