Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
The present application is being examined under the claims filed 03/06/2026. The status of the claims are as follows:
Claims 1-20 are pending.
Claims 7 and 17 are canceled.
Claims 1, 9, 19 are amended.
Response to Amendment
The Office Action is in response to Applicant’s communication filed 03/06/2026 in response to office action mailed 12/10/2025. The Applicant’s remarks and any amendments to the claims or specification have been considered with the results that follow.
Response to Arguments
Regarding 35 U.S.C. § 101
The examiner has considered Applicant’s arguments (remarks, pg. 7-19) and finds them unpersuasive. Applicant argues that amended independent claims 1, 9, and 19 are patent-eligible because the claims now recite a weighted-average risk score in which results obtained using the target machine learning model are weighted higher than results obtained using the shadow model, and further recite deploying the target machine learning model, upon a determination that the risk score is below a threshold value, such that the target machine learning model processes runtime data. Applicant contends that these additional limitations integrate any alleged abstract idea into a practical application and improve machine learning model risk assessment and deployment. Applicant further relies on ¶¶ [0006], [0022], [0036], [0038], and [0085] of the specification and on Ex parte Desjardins.
These arguments are not persuasive. As amended, the claims continue to recite an abstract idea under Step 2A, Prong One. In particular, the claims recite generating a shadow model from data of a target machine learning model, performing predefined tests to obtain test results, and computing a risk score as a weighted average of those results, including weighting according to the model used for the test and weighting target-model results higher than shadow-model results. These limitations recite mathematical concepts and evaluation or judgment operations because they involve producing test results and mathematically combining those results according to a specified weighting rule. The amendment specifying that target-model results are weighted more heavily than shadow-model results merely further defines the mathematical scoring policy itself and does not remove the claims from the abstract-idea grouping.
Applicant’s arguments are also unpersuasive under Step 2A, Prong Two. The principal additional element beyond the abstract generation, testing, and scoring operations is the recitation of deploying, upon a determination that the risk score is below a threshold value, the target machine learning model so that it processes runtime data. However, this limitation merely uses the result of the abstract analysis to decide whether the target machine learning model will be deployed. The claims do not recite any specific technological mechanism for deployment, any specialized hardware arrangement, any particular runtime control architecture, or any specific improvement in how the machine learning model itself is trained, structed, or operated. Rather, the deployment step is recited at a high level of generality and amounts to using the abstract result in a generic go/no-go decision. Such post-analysis use of the abstract result does not integrate the judicial exception into a practical application.
Applicant’s reliance on Ex parte Desjardins is ineffectual. Unlike the claims at issue there, the present claims do not recite a specific improvement in how a machine learning model itself operates or is trained. Instead, the claims recite evaluating model risk using test results and a weighted score, and then using that score to decide whether to deploy the model. Nor do the claims recite the type of specific technical improvement found in cases such as Enfish, Research Corp., or BASCOM, as cited by Applicant. The claims remain at a high level of generality and do not specify a particular improved data structure, image-processing technique, network architecture, or machine-learning implementation.
For substantially the same reasons, Applicant’s arguments are unpersuasive under Step 2B. The deployment limitation is recited at a high level of generality and merely appends generic downstream use of the computed risk score. The claims therefore do not include additional elements amounting to significantly more than the judicial exception. Accordingly, the rejection of claims 1-6, 8-16, and 18-20 under 35 U.S.C. § 101 is maintained. See updated 101 rejection in this Office Action below.
Regarding 35 U.S.C. § 103
The Examiner has considered Applicant’s arguments (remarks, pages 19-25) and finds them persuasive. The prior rejection of claims 1-6, 8-16, and 18-20 under 35 U.S.C. § 103 over Sharma et al. in view of Liu et al., and further in view of EPSS where applicable, is withdrawn.
The prior Office Action relied on Sharma for generating and testing shadow models, Liu for combining results of multiple assessments into an overall attack-risk framework, and EPSS for weighted scoring concepts. However, as amended, independent claims 1, 9, and 19 recite specific limitations not taught or suggested by the prior art of record, namely:
computing a risk score comprising a weighted average in which a weight is set according to a model on which the predefined test was performed, wherein results obtained using the target machine learning model are weighted higher than results obtained using the shadow model; and
deploying, upon a determination that the risk score is below a threshold value, the target machine learning model, the deploying causing the target machine learning model to process runtime data.
The cited references do not disclose or suggest this specific weighting relationship between target-model results and shadow-model results in the claimed combined risk score. The cited references also do not disclose or suggest the claimed threshold-based deployment of the target machine learning model for runtime data processing based on that weighted risk score. While the prior art generally describes shadow-model testing, multiple model assessments, and weighted scoring concepts, the prior art of record does not teach or render obvious the claimed coordination of combining target-model and shadow-model test results into a weighted average risk score in which target-model results are weighted more heavily, and then using that score to determine whether to deploy the target machine learning model for runtime operation.
The amended claims therefore recite a specific combination of model-testing, weighting, and deployment constraints that is not taught or suggested by the prior art of record, either individually or in combination. Accordingly, the rejection of claims 1-6, 8-16, and 18-20 under 35 U.S.C. § 103 set forth in the prior Office Action is withdrawn.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-6, 8-16, and 18-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claim 1
Claim 1 – Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
Yes, the claim is to a process.
Claim 1 – Step 2A – Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
Yes, the claim recites an abstract idea.
generating, using data of a target machine learning model, a shadow model; - conceptually, this is creating a surrogate or auxiliary model from model-related data. At a high level, this is an act of modeling and evaluation of information, which falls within mental modeling/evaluation. See MPEP § 2106.04(a)(2)(III) (mental processes).
performing a predefined test on the shadow model, the performing resulting in a first test result; - this is an evaluation step that applies criteria to a model and produces a result. At a high level, this is observation, analysis, and evaluation, which falls within mental processes. See MPEP § 2106.04(a)(2)(III).
computing a risk score comprising the first test result and a second test result, the second test result obtained by performing a second predefined test using the data of the target machine learning model. – this recites deriving a score from multiple test results, i.e., a mathematical relationship or calculation. Computing a “risk score” from test results is a mathematical relationship or calculation (e.g., scoring function, aggregation of metrics). Such scoring, combining first and second test results according to some rule, constitutes a mathematical concept/mental process. (See MPEP § 2106.04(a)(2)(I); 2106.04(a)(2)(III)).
“wherein the risk score comprises a weighted average including the first test result and the second test result, wherein a weight in the weighted average is set according to a model on which the predefined test was performed, wherein results obtained using the target machine learning model are weighed higher than results obtained using the shadow model:” – this further defines the risk score as a weighted-average calculation with model-dependent weighting. A weighted average is a mathematical formula, and specifying that one category of results is weighted more heavily than another merely defines the mathematical scoring policy itself. This remains a mathematical concept and mental evaluation rule. See MPEP § 2106.04(a)(2)(I); 2106.04(a)(2)(III).
Claim 1 – Step 2A – Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
No. There are no additional elements that integrate the judicial exception into a practical application. The additional elements:
“and deploying, upon a determination that the risk score is below a threshold value, the target machine learning model, the deploying causing the machine learning model to process runtime data.” – this limitation merely uses the result of the abstract analysis to decide whether the target machine learning model will be deployed. That is, once the abstract idea produces a risk score, the claim applies a threshold comparison and authorizes subsequent use of the model if the score satisfies the threshold. The claim does not recite any specific technical mechanism for deployment, any specialized hardware arrangement, any particular runtime control architecture, or any specific improvement in how the machine learning model itself is trained, structured, or operated. Instead, the deployment step is recited at a high level of generality and amounts to using the abstract result in a generic go/no-go decision. Such point-analysis use of the abstract result is insignificant extra-solution activity and does not integrate the judicial exception into a practical application. See MPEP § 2106.05(g); 2106.05(a).
Claim 1 – Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
No. There are no additional elements that amount to significantly more than the judicial exception. The additional elements:
“and deploying, upon a determination that the risk score is below a threshold value, the target machine learning model, the deploying causing the machine learning model to process runtime data.” – this limitation merely appends generic downstream use of the computed score. Implementing a threshold comparison and permitting later use of a model based on the outcome is conventional control logic performed in a generic computing environment. The claim does not recite any unconventional deployment mechanism, any non-generic arrangement of components, or any other inventive concept beyond the abstract idea itself. See WURC, MPEP § 2106.05(d).
Regarding claim 2
Claim 2 – Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
Yes, the claim is to a process.
Claim 2 – Step 2A – Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
Yes, the claim recites an abstract idea.
Claim 2 – Step 2A – Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
No. There are no additional elements that integrate the judicial exception into a practical application. The additional elements:
wherein the shadow model is generated responsive to determining that the data of the target machine learning model is insufficient to perform the test on the target machine learning model. – this limitation simply specifies a conditional trigger for generating the shadow model, i.e., generate a surrogate model if the available data for the target model is insufficient to run the test directly. At a high level, this is just a rule for when to invoke one abstract evaluation path (using the shadow model) instead of another (using the target model), based on an informational sufficiency check. This limitation merely refines when the already abstract process of generating and using a shadow model is applied. The conditional “responsive to determining … insufficient data” clause is extra detail within the abstract idea itself (a choice of when to use the surrogate model) and does not integrate the judicial exception into a practical application. (See MPEP § 2106.04(d); 2106.05(a), 2106.05(f), 2106.05(g)).
Claim 2 – Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
No. There are no additional elements that amount to significantly more than the judicial exception. The additional elements are:
wherein the shadow model is generated responsive to determining that the data of the target machine learning model is insufficient to perform the test on the target machine learning model. – determining whether the data is “insufficient” to run a test and, based on that determination, generating a surrogate/shadow model is a routine pattern in data analysis and modeling (e.g., when direct evaluation is not possible or convenient, us a proxy model built from available data). Implementing such a conditional in a generic computing environment (e.g., using standard program logic to check data conditions and then call a model generation routine) is a well-understood, routine, and conventional (WURC) programming practice. (MPEP § 2106.05(d), 2106.05(f)).
Regarding claim 3
Claim 3 – Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
Yes, the claim is to a process.
Claim 3 – Step 2A – Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
Yes, the claim recites an abstract idea.
Claim 3 – Step 2A – Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
No. There are no additional elements that integrate the judicial exception into a practical application. The additional elements:
generating, using a plurality of frameworks, a corresponding plurality of shadow models; - the step describes generating multiple surrogate models, which is an informational / mathematical modeling operation, not a technical improvement to a computer or other technology. There is no specialized hardware, no constrained architecture, and no transformation of matter. The limitation merely changes how the abstract model evaluation idea is executed, not whether the computer is improved. It is an implementation detail within the abstract idea itself and is therefore an insignificant extra-solution/field-of-use choice, not an integration into a practical application. See MPEP § 2106.04(d); 2106.05(a), 2106.05(f), 2106.05(g).
and selecting, from the corresponding plurality of shadow models using a similarity metric, the shadow model, wherein the similarity metric measures a similarity between a plurality of outputs of the shadow model and a plurality of outputs of the target machine- learning model. – this limitation recites computing a similarity metric based on model outputs and selecting the shadow model using that metric. Performing such a measurement constitutes a mathematical comparison and decision rule, a classic abstract operation, not a technical improvement in computer functionality. It does not integrate the abstract idea into a practical application. See MPEP § 2106.05(a), 2106.05(g), 2106.04(d).
Claim 3 – Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
No. There are no additional elements that amount to significantly more than the judicial exception. The additional elements are:
generating, using a plurality of frameworks, a corresponding plurality of shadow models; - the specification teaches that shadow models can be generated using presently available techniques and frameworks, and that multiple shadow models (of same or different types) may be generated. Because the applicant’s own disclosure demonstrates that generating multiple shadow models using existing ML frameworks is standard practice, this limitation is well-understood, routine, and conventional (WURC), and therefore does not add significantly more than the abstract idea. (spec. ¶[0080]). See MPEP § 2106.05(d), 2106.05(f).
and selecting, from the corresponding plurality of shadow models using a similarity metric, the shadow model, wherein the similarity metric measures a similarity between a plurality of outputs of the shadow model and a plurality of outputs of the target machine- learning model. – the specification (spec. ¶[0026]) expressly identifies the similarity metric as a presently available model similarity metric such as
R
2
or classification error. Computing such metrics and selecting the model with best performance is standard, well-understood practice in model evaluation and does not introduce an unconventional technique or hardware. In a generic computer, this is ordinary numeric computation and comparison, routine programming logic, not an inventive concept. Individually and in combination, this limitation reflects well-understood, routine, and conventional (WURC) activities in the field of machine learning model evaluation and do not amount to “significantly more” than the underlying abstract idea. See MPEP § 2106.05(d), 2106.05(f).
Regarding claim 4
Claim 4 – Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
Yes, the claim is to a process.
Claim 4 – Step 2A – Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
Yes, the claim recites an abstract idea.
Claim 4 – Step 2A – Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
No. There are no additional elements that integrate the judicial exception into a practical application. The additional elements:
wherein the shadow model is generated using a framework specified by the data of the target machine learning model. – this limitation specifies that the framework used to generate the shadow model (e.g., a particular ML library/model framework) is chosen based on “data of the target model”. The specification describes this in terms of using model-related information (model type, framework, predictions, training data, etc.) to pick or infer a compatible framework and then generate a corresponding shadow model (see spec. ¶[0080]). This limitation reflects field-of-use/implementation detail and extra-solution activity (how the abstract modeling is carried out in software), and does not integrate the judicial exception into a practical application. See MPEP § 2106.04(d); 2106.05(a), 2106.05(f), 2106.05(g).
Claim 4 – Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
No. There are no additional elements that amount to significantly more than the judicial exception. The additional elements are:
wherein the shadow model is generated using a framework specified by the data of the target machine learning model. – the specification explains that module 230 uses “a specified framework” to generate a shadow model of the given type and further teaches that, where only certain information is available, a framework can be inferred and used (spec. ¶[0080]). Implementing this in software amounts to examining model-related information, mapping it to a known ML framework, and calling that framework to construct a corresponding model object. This is a well-understood, routine, and conventional (WURC) activity, as supported by the applicant’s own description, and does not provide an inventive concept beyond the abstract idea. (See MPEP § 2106.05(d), 2106.05(f)).
Regarding claim 5
Claim 5 – Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
Yes, the claim is to a process.
Claim 5 – Step 2A – Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
Yes, the claim recites an abstract idea.
Claim 5 – Step 2A – Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
No. There are no additional elements that integrate the judicial exception into a practical application. The additional elements:
wherein the shadow model is trained using training data of the target machine learning model. – this limitation specifies that the training data of the target model is used to train the shadow model. The specification repeatedly describes this as one of several straightforward options for generating and training shadow models (see spec. ¶[0028], FIG. 4, block 412). Functionally, this is still part of the abstract modeling pipeline, i.e., choose a dataset (here the training data) and use it to train a surrogate model. The limitation does not recite: a specific novel training algorithm, loss function, or optimization method, any non-conventional computer architecture, or any transformation of a physical article. Rather, it simply selects which data source (the known training data) is used when performing the otherwise abstract task of training a surrogate model. See MPEP § 2106.04(d); 2106.05(f), 2106.05(g).
Claim 5 – Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
No. There are no additional elements that amount to significantly more than the judicial exception. The additional elements are:
wherein the shadow model is trained using training data of the target machine learning model. – the specification presents training shadow models on the target model’s training data as normal, expected practice using existing ML techniques: “In particular, if training data for the target model is not available, an embodiment uses the training data to generate a shadow model corresponding to the target model.” (spec. ¶[0028]). Implementing this in software is just feeding the training data in a conventional framework, and training that model using the framework’s ordinary training routines. That is well-understood, routine, and conventional (WURC) ML practice and does not provide an inventive concept beyond the abstract idea. See MPEP § 2106.05(d), 2106.05(f).
Regarding claim 6
Claim 6 – Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
Yes, the claim is to a process.
Claim 6 – Step 2A – Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
Yes, the claim recites an abstract idea.
Claim 6 – Step 2A – Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
No. There are no additional elements that integrate the judicial exception into a practical application. The additional elements:
wherein the shadow model is trained using test data of the target machine learning model. – this limitation specifies that the test data of the target model is used to train the shadow model. The specification repeatedly describes using test data to generate/train a shadow model as one of several straightforward, alternative options, depending on what data is available. Functionally, this is still part of the abstract modeling pipeline, i.e., choose a dataset (here the testing data) and use it to train a surrogate model. The limitation does not recite: a specific novel training algorithm, loss function, or optimization method, any non-conventional computer architecture, or any transformation of a physical article. Rather, it simply selects which data source (the known training data) is used when performing the otherwise abstract task of training a surrogate model. See spec. ¶[0028]; MPEP § 2106.04(d); 2106.05(f), 2106.05(g).
Claim 6 – Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
No. There are no additional elements that amount to significantly more than the judicial exception. The additional elements are:
wherein the shadow model is trained using test data of the target machine learning model. – the specification presents training shadow models using test data as a normal and expected variation of known ML practice: “If training data for the target model is not available, but test data for the target model is available, an embodiment uses the test data to generate a shadow model corresponding to the target model.” (spec. ¶[0028]). The flowchart of FIG. 4 shows using test data when training data is absent as a straightforward fallback path in a generic application (spec. FIG. 4, block 414 [Wingdings font/0xE0] 416). In practice, implementing this limitation means taking the test data already available for the target model, feeding it into a shadow model instantiated in a conventional ML framework, and training that model using the framework’s standard training routines. This is well-understood, routine, and conventional (WURC) ML practice and does not provide an inventive concept beyond the underlying abstract idea. See MPEP § 2106.05(d), 2106.05(f).
Regarding claim 8
Claim 8 – Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
Yes, the claim is to a process.
Claim 8 – Step 2A – Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
Yes, the claim recites an abstract idea.
Claim 8 – Step 2A – Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
No. There are no additional elements that integrate the judicial exception into a practical application. The additional elements:
wherein the risk score comprises a weighted average including the first test result and the second test result – the specification explains that a risk score can be computed by normalizing test results and taking a weighted average: “One embodiment normalizes test results onto the same scale (e.g., 0 to 1), and uses a weighted average of the normalized results as the risk score.” (spec. ¶[0036]). A weighted average of numerical test results is a mathematical operation, falling in the “mathematical concepts” grouping of abstract ideas. The limitation does not recite any specialized hardware or unconventional system architecture; the computer is used in a straightforward way to perform numeric calculations. Accordingly, this additional element does not integrate the abstract idea into a practical application. See MPEP § 2106.04(d); 2106.05(a), 2106.05(g).
wherein a weight in the weighted average is set according to a type of data used to perform the predefined test. – the specification expressly discloses this exact weighting policy as one variant of the scoring scheme: “In another embodiment, weights used in the weighted average have predefined values, based on the type of data used to perform that test. For example, an embodiment might weight model results obtained using training data higher than model results obtained using testing data.” (spec. ¶[0036]). The rule “set the weight according to a type of data used to perform the predefined test” is a conditional on data provenance (training vs. testing, etc.) that determines numeric coefficients in the same weighted-average formula. This does not improve computer operation or any other technology; it is a design choice inside the mathematical aggregation (how much importance to assign to certain test results). This limitation is another abstract scoring rule that simply configures the math based on data type. It does not integrate the judicial exception into a practical application. Accordingly, this additional element does not integrate the abstract idea into a practical application. See MPEP § 2106.04(d); 2106.05(a), 2106.05(g).
Claim 8 – Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
No. There are no additional elements that amount to significantly more than the judicial exception. The additional elements are:
wherein the risk score comprises a weighted average including the first test result and the second test result – the applicant’s own specification shows that the weighted-average scoring are routine, designer-selected options: for the weights based on data type: “One embodiment normalizes test results onto the same scale (e.g., 0 to 1), and uses a weighted average of the normalized results as the risk score.” (spec. ¶[0036]). Implementing this limitation means taking numeric test results and optionally normalizing them. These are well-understood, routine, and conventional (WURC) mathematical post-processing steps that do not provide an inventive concept beyond the abstract idea. See MPEP § 2106.05(d), 2106.05(f), 2106.05(g).
wherein a weight in the weighted average is set according to a type of data used to perform the predefined test. – the applicant’s own specification shows that the choice of weights based on data type are routine, designer-selected options for the weighted-average risk score: “In another embodiment, weights used in the weighted average have predefined values, based on the type of data used to perform that test. For example, an embodiment might weight model results obtained using training data higher than model results obtained using testing data” (spec. ¶[0036]). Implementing this limitation means assigning weights based on the type of data used of each test (e.g., higher weight for training data results that for test data results), and computing an average. These are well-understood, routine, and conventional (WURC) mathematical post-processing steps that do not provide an inventive concept beyond the abstract idea. See MPEP § 2106.05(d), 2106.05(f), 2106.05(g).
Regarding claim 9
Claim 9 – Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
Yes, the claim is to a machine.
Claim 9 – Step 2A – Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
Yes, the claim recites an abstract idea. Claim 9 is the computer-program-product analog of claim 1. The functional language associated with the recited “program instructions executable by a processor to cause the processor to perform operations comprising” merely causes a generic processor to perform the same abstract operations discussed above with respect to claim 1 namely:
generating, using data of a target machine learning model, a shadow model;
performing a predefined test on the shadow model, the performing resulting in a first test result;
computing a risk score comprising the first test result and a second test result, the second test result obtained by performing a second predefined test using the data of the target machine learning model.
“wherein the risk score comprises a weighted average including the first test result and the second test result, wherein a weight in the weighted average is set according to a model on which the predefined test was performed, wherein results obtained using the target machine learning model are weighed higher than results obtained using the shadow model:”
These limitations recite mathematical concepts and mental processes for the same reasons discussed with respect to claim 1. See MPEP § 2106.04(a)(2)(I), 2106.04(a)(2)(III).
Claim 9 – Step 2A – Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
No. There are no additional elements that integrate the judicial exception into a practical application. The additional element beyond the abstract generation, testing, and scoring operations is:
“and deploying, upon a determination that the risk score is below a threshold value, the target machine learning model, the deploying causing the machine learning model to process runtime data.” – this limitation, as with claim 1, merely uses the result of the abstract analysis to decide whether the target machine learning model will be deployed. The system formulation does not recite any specific technological mechanism for deployment, any specialized hardware arrangement, any particular runtime control architecture, or any specific improvement in how the machine learning model itself is trained, structured, or operated. Rather, the deployment step is recited at a high level of generality and amounts to using the abstract result in a generic go/no-go decision. Such post-analysis use of the abstract result is insignificant extra-solution activity and does not integrate the judicial exception into a practical application. See MPEP § 2106.05(g); 2106.05(a).
Claim 9 – Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
No. There are no additional elements that amount to significantly more than the judicial exception. The additional element:
“and deploying, upon a determination that the risk score is below a threshold value, the target machine learning model, the deploying causing the machine learning model to process runtime data.” – deploying the target machine learning model upon a determination that the risk score is below a threshold value merely appends generic downstream use of the computed score. The recited processor and one or more computer readable storage media are generic computer components performing their ordinary functions. Claim 9 does not recite any unconventional deployment mechanism, any non-generic arrangement of components, or any other inventive concept beyond the abstract idea itself. See WURC, MPEP § 2106.05(d).
Regarding claims 10-16 and 18 (computer program product claims)
Each of claims 10-16 and 18 is the computer program product analog of an already analyzed method claim (12 [Wingdings font/0xDF][Wingdings font/0xE0] 2; 13 [Wingdings font/0xDF][Wingdings font/0xE0] 3; 14 [Wingdings font/0xDF][Wingdings font/0xE0] 4; 15 [Wingdings font/0xDF][Wingdings font/0xE0] 5; 16 [Wingdings font/0xDF][Wingdings font/0xE0] 6; 18 [Wingdings font/0xDF][Wingdings font/0xE0] 8; claims 10-11 add only generic storage/transfer and metering/invoicing details around the functionality of claim 9). For each of claims 10-16 and 18, the functional language associated with the “program instructions executable by a processor to cause the processor to …” merely cause a generic processor to perform the same abstract operations already identified in method claims 1-6 and 8, including generating a shadow model, performing predefined tests, computing a weighted risk score using model-based weighting, deploying the target machine learning model upon a threshold determination, and, where applicable, conditionally generating, selecting, or training shadow models and applying additional weighting policies. Under Step 2A (Prong 1/Prong 2), recasting the abstract idea as instructions on a computer-readable storage medium does not add a different judicial exception or integrate the exception into a practical application. Under Step 2B, the program instructions are implemented on generic computer-readable media and executed by generic processors in a conventional networked computing environment, which is well-understood, routine, and conventional (WURC); no inventive concept. Because each of claims 10-16 and 18 is the computer-program-product analog of a corresponding method claim and merely implements the same abstract mathematical/mental-process limitations on generic computer-readable media and processors without adding a practical application or inventive concept, claims 10-16 and 18 are rejected under 35 U.S.C. § 101 for the same reasons discussed above for claims 1-6 and 8, respectively.
Regarding claim 19
Claim 19 – Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
Yes, the claim is to a machine.
Claim 19 – Step 2A – Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
Yes, the claim recites an abstract idea. Claim 19 is the system analog of claim 1. The functional language associated with the recited processor, one or more computer readable storage media, and program instructions executable by the processor merely requires generic computing components configured to perform the same abstract operations discussed above with respect to claim 1, namely:
generating, using data of a target machine learning model, a shadow model;
performing a predefined test on the shadow model, the performing resulting in a first test result;
computing a risk score comprising the first test result and a second test result, the second test result obtained by performing a second predefined test using the data of the target machine learning model.
“wherein the risk score comprises a weighted average including the first test result and the second test result, wherein a weight in the weighted average is set according to a model on which the predefined test was performed, wherein results obtained using the target machine learning model are weighed higher than results obtained using the shadow model:”
These limitations recite mathematical concepts and mental processes for the same reasons discussed with respect to claim 1. See MPEP § 2106.04(a)(2)(I), 2106.04(a)(2)(III).
Claim 19 – Step 2A – Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
No. There are no additional elements that integrate the judicial exception into a practical application. The additional element beyond the abstract generation, testing, and scoring operations is:
“and deploying, upon a determination that the risk score is below a threshold value, the target machine learning model, the deploying causing the machine learning model to process runtime data.” – As with claim 1, this limitation merely uses the result of the abstract analysis to decide whether the target machine learning model will be deployed. The system formulation does not recite any specific technological mechanism for deployment, any specialized hardware arrangement, any particular runtime control architecture, or any specific improvement in how the machine learning model itself is trained, structured, or operated. Instead, the claimed processor and storage media are generic computing components used in a conventional manner to implement the abstract result in a generic go/no-go decision. Such post-analysis use of the abstract result is insignificant extra-solution activity and does not integrate the judicial exception into a practical application. See MPEP § 2106.05(g); 2106.05(a).
Claim 19 – Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
No. There are no additional elements that amount to significantly more than the judicial exception. The additional element:
“and deploying, upon a determination that the risk score is below a threshold value, the target machine learning model, the deploying causing the machine learning model to process runtime data.”
The additional element of deploying the target machine learning model upon a determination that the risk score is below a threshold value merely appends generic downstream use of the computed score. The recited processor and one or more computer readable storage media are generic computer components performing their ordinary functions. Claim 19 does not recite any unconventional deployment mechanism, any non-generic arrangement of components, or any other inventive concept beyond the abstract idea itself. See WURC, MPEP § 2106.05(d).
Regarding claim 20 (system claims)
Claim 20 is the system analog of already analyzed method claim 2 (20 [Wingdings font/0xDF][Wingdings font/0xE0] 2). Claim 20 depends from claim 19 and further recites that the shadow model is generated responsive to determining that the data of the target machine learning model is insufficient to perform the test on the target machine learning model. The functional language associated with the “processor” and “program instructions collectively stored on one or more computer readable storage media” merely requires generic computing components (processor, memory/storage, network module, etc.) configured to perform the same abstract operations already identified in method claim 19 and corresponding method claim 2, namely generating a shadow model, performing predefined tests, computing a risk score, deploying the target machine learning model upon a threshold determination, and conditionally generating the shadow model when target-model data is insufficient to perform the test directly. This system formulation does not materially change the eligibility analysis. Under Step 2A (Prong One/ Prong Two), the system formulation does not add a different judicial exception or integrate the abstract idea into a practical application; under Step 2B, the claimed system components are generic computing hardware operating in a conventional fashion and are well-understood, routine, and conventional (WURC), and thus do not provide an inventive concept. Because claim 20 is the system analog of corresponding method claim 2 and merely implements the same abstract mathematical and mental-process limitations on generic computing units without adding a practical application or significantly more, claim 20 is rejected under 35 U.S.C. § 101 for the same reasons discussed above with respect to claims 19 and 2.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner
should be directed to PAUL COLEMAN whose telephone number is (571)272-4687. The examiner can
normally be reached Mon-Fri.
Examiner interviews are available via telephone, in-person, and video conferencing using a
USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use
the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor,
DAVID YI can be reached at 571-270-7519. The fax phone number for the organization where this
application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from
Patent Center. Unpublished application information in Patent Center is available to registered users. To
file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit
https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and
https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional
questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like
assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or
571-272-1000.
/PAUL COLEMAN/
Examiner, Art Unit 2126
/DAVID YI/ Supervisory Patent Examiner, Art Unit 2126
Prosecution Insights