DETAILED ACTION
This action is in response to the Applicant Response filed 27 September 2024 for application 18/899,161 filed 27 September 2024.
Claim(s) 1-20 is/are pending.
Claim(s) 1-20 is/are rejected.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim(s) 6-7, 13 is/are objected to because of the following informalities:
Claim 6, line 1, wherein data anomalies detected by the ML model should read “wherein the representations of data anomalies detected by the ML model”
Claim 7, lines 3-4, to determine data anomalies for each time period should read “to determine a representation of data anomalies for each time period”
Claim 13, line 4, determine third output should read “determine a third output”
Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 101, because the claim(s) is/are directed to an abstract idea, and because the claim elements, whether considered individually or in combination, do not amount to significantly more than the abstract idea, see Alice Corporation Pty. Ltd. V. CLS Bank International et al., 573 US 208 (2014).
Regarding claim 1, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 1 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method.
The limitation of identifying a base data set to be used as input to a machine learning (ML) model, the base data set including one or more sets of structured data, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of generating a modified base data set by causing synthetic anomaly injection operations to be performed on data of the base data set, the synthetic anomaly injection operations configured to create a set of known data modifications that are represented in the modified base data set, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of determining, based on the first output, the second output, and the set of known data modifications, a set of performance metrics representing performance of the ML model at detecting data anomalies, the set of performance metrics including: a true positive detection rate metric; a true negative detection rate metric; a false positive detection rate metric; and a false negative detection rate metric, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of outputting a representation of the set of performance metrics, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites additional element(s) – computing system. The additional element(s) is/are recited at a high-level of generality (i.e., as generic computer components performing generic computer functions of executing instructions on the computers) such that it amounts to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b)).
The claim recites additional element(s) – machine learning (ML) model. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
The claim recites causing the ML model to run, using the base data set as input, to determine a first output of the ML model, wherein the first output includes a representation of data anomalies detected by the ML model in the base data set; causing the ML model to run, using the modified base data set as input, to determine a second output of the ML model, wherein the second output includes a representation of data anomalies detected by the ML model in the modified base data set which is simply applying a model recited at a high level of generality and amounts to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instructions to implement an abstract idea or other exception on a computer (MPEP 2106.05(f)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
computing system amount(s) to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b))
applying a model amount(s) to no more than mere instructions to apply the exception (MPEP 2106.05(f))
machine learning (ML) model amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 2, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 2 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method. The Step 2A Prong One Analysis for claim 1 is applicable here since claim 2 carries out the method of claim 1 but for the recitation of additional element(s) of wherein output of the ML model includes one or more of: a number of detected anomalies; one or more classifications of detected anomalies; one or more anomaly scores; one or more locations of detected anomalies in a respective data set used as input; and one or more values of data corresponding to detected anomalies.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the results and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the results do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 3, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 3 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method. The Step 2A Prong One Analysis for claim 2 is applicable here since claim 3 carries out the method of claim 2 but for the recitation of additional element(s) of wherein classifications of detected anomalies include one or more of: missing value; incorrect value; invalid value; incomplete row; duplicate row; temporal inconsistency; and schema change.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the results and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the results do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 4, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 4 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method. The Step 2A Prong One Analysis for claim 1 is applicable here since claim 4 carries out the method of claim 1 but for the recitation of additional element(s) of wherein the base data set and the modified base data set do not include labeled data that has been pre-identified as including anomalies.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the data and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the data do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 5, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 5 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method. The Step 2A Prong One Analysis for claim 1 is applicable here since claim 5 carries out the method of claim 1 but for the recitation of additional element(s) of wherein the one or more sets of structured data includes data in tabular form.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the data and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the data do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 6, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 6 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method. The Step 2A Prong One Analysis for claim 1 is applicable here since claim 6 carries out the method of claim 1 but for the recitation of additional element(s) of wherein data anomalies detected by the ML model represent an unexpected distribution change in one or more values in a respective data set used as input.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the results and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the results do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 7, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 7 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method. The Step 2A Prong One Analysis for claim 1 is applicable here since claim 7 carries out the method of claim 1 but for the recitation of additional element(s) of wherein causing the ML model to run, using a respective data set as input, to determine output includes causing the ML model to sequentially process time periods represented by the respective data set to determine data anomalies for each time period.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites wherein causing the ML model to run, using a respective data set as input, to determine output includes causing the ML model to sequentially process time periods represented by the respective data set to determine data anomalies for each time period which is simply applying the model recited at a high level of generality and amounts to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instructions to implement an abstract idea or other exception on a computer (MPEP 2106.05(f)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
applying the model amount(s) to no more than mere instructions to apply the exception (MPEP 2106.05(f))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 8, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 8 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method. The Step 2A Prong One Analysis for claim 1 is applicable here since claim 8 carries out the method of claim 1 but for the recitation of additional element(s) of wherein causing the ML model to run, using the modified base data set as input, to determine the second output includes: causing the ML model to process multiple sets of structured data that have been subject to the synthetic anomaly injection operations, wherein processing each respective set of structured data of the multiple sets of structured data includes processing the respective set using an instance of the ML model that has not processed another set of structured data of the multiple sets of structured data.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites … wherein processing each respective set of structured data of the multiple sets of structured data includes processing the respective set using an instance of the ML model that has not processed another set of structured data of the multiple sets of structured data which is simply additional information regarding the model, and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)).
The claim recites wherein causing the ML model to run, using the modified base data set as input, to determine the second output includes: causing the ML model to process multiple sets of structured data that have been subject to the synthetic anomaly injection operations, wherein processing each respective set of structured data of the multiple sets of structured data includes processing the respective set using an instance of the ML model that has not processed another set of structured data of the multiple sets of structured data which is simply applying the model recited at a high level of generality and amounts to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instructions to implement an abstract idea or other exception on a computer (MPEP 2106.05(f)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
applying the model amount(s) to no more than mere instructions to apply the exception (MPEP 2106.05(f))
additional information regarding the model do(es) not apply the exception in a meaningful way (MPEP 2106.05(e))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 9, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 9 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method. The Step 2A Prong One Analysis for claim 1 is applicable here since claim 9 carries out the method of claim 1 but for the recitation of additional element(s) of wherein the first output and the second output each include: a set of anomaly scores determined by the ML model; and a set of anomaly score thresholds set by the ML model.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the results and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the results do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 10, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 10 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method. The Step 2A Prong One Analysis for claim 9 is applicable here since claim 10 carries out the method of claim 9 but for the recitation of additional element(s) of wherein the set of anomaly scores includes an anomaly score for each data entry of the one or more sets of structured data.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the results and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the results do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 11, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 11 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method. The Step 2A Prong One Analysis for claim 9 is applicable here since claim 11 carries out the method of claim 9 but for the recitation of additional element(s) of wherein the set of anomaly score thresholds are dynamically set by the ML model over a set of time periods represented by the one or more sets of structured data.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the results and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the results do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 12, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 12 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method. The Step 2A Prong One Analysis for claim 9 is applicable here since claim 12 carries out the method of claim 9 but for the recitation of additional element(s) of wherein the ML model detects an anomaly for a data entry of a respective data set when an anomaly score determined for the data entry exceeds an applicable anomaly score threshold.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the model and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the model do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 13, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 13 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method.
The limitation of causing a change to a hyperparameter of the ML model, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of determining a performance metric representing comparison of the performance of the ML model with and without the change to the hyperparameter, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites causing the ML model with the change to the hyperparameter to run, using a data set as input, to determine third output of the ML model, wherein the third output includes a representation of data anomalies detected by the ML model in the data set which is simply applying the model recited at a high level of generality and amounts to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instructions to implement an abstract idea or other exception on a computer (MPEP 2106.05(f)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
applying the model amount(s) to no more than mere instructions to apply the exception (MPEP 2106.05(f))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 14, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 14 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method. The Step 2A Prong One Analysis for claim 1 is applicable here since claim 14 carries out the method of claim 1 but for the recitation of additional element(s) of wherein the set of performance metrics includes a metric representing an area under a curve, wherein the curve is a plot of the ratio of the true positive detection rate metric versus the false positive detection rate metric over a variable set of values of an anomaly score threshold of the ML model.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the performance metrics and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the performance metrics do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 15, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 15 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method. The Step 2A Prong One Analysis for claim 1 is applicable here since claim 15 carries out the method of claim 1 but for the recitation of additional element(s) of wherein the set of performance metrics includes an accuracy metric representing a percentage of time the ML model detected a known data modification of the set of known data modifications as a data anomaly.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the performance metrics and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the performance metrics do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 16, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 16 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method. The Step 2A Prong One Analysis for claim 1 is applicable here since claim 16 carries out the method of claim 1 but for the recitation of additional element(s) of wherein the second output includes descriptions of detected anomalies; and wherein the set of performance metrics includes one or more metrics representing whether the descriptions of the detected anomalies included in the second output match corresponding descriptions of the known data modifications that were detected as data anomalies by the ML model.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the results and the performance metrics and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the results and the performance metrics do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 17, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 17 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method. The Step 2A Prong One Analysis for claim 1 is applicable here since claim 17 carries out the method of claim 1 but for the recitation of additional element(s) of wherein the set of performance metrics includes one or more metrics representing one or more of: a length of time of running the ML model; a frequency of alerts generated to report data anomalies detected by the ML model; a precision of the ML model; a recall of the ML model; an F1 score of the ML model; and an average anomaly score output by ML model.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the performance metrics and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the performance metrics do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 18, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 18 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method. The Step 2A Prong One Analysis for claim 1 is applicable here since claim 18 carries out the method of claim 1 but for the recitation of additional element(s) of wherein the representation of the set of performance metrics includes one or more visualizations.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the performance metrics and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the performance metrics do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 19, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 19 is directed to a system with a processor, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) system.
The limitation of identifying a base data set to be used as input to a machine learning (ML) model, the base data set including one or more sets of structured data, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of generating a modified base data set by causing synthetic anomaly injection operations to be performed on data of the base data set, the synthetic anomaly injection operations configured to create a set of known data modifications that are represented in the modified base data set, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of determining, based on the first output, the second output, and the set of known data modifications, a set of performance metrics representing performance of the ML model at detecting data anomalies, the set of performance metrics including: a true positive detection rate metric; a true negative detection rate metric; a false positive detection rate metric; and a false negative detection rate metric, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of outputting a representation of the set of performance metrics, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites additional element(s) – system, one or more processors, computer-readable medium, instructions. The additional element(s) is/are recited at a high-level of generality (i.e., as generic computer components performing generic computer functions of executing instructions on the computers) such that it amounts to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b)).
The claim recites additional element(s) – machine learning (ML) model. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
The claim recites causing the ML model to run, using the base data set as input, to determine a first output of the ML model, wherein the first output includes a representation of data anomalies detected by the ML model in the base data set; causing the ML model to run, using the modified base data set as input, to determine a second output of the ML model, wherein the second output includes a representation of data anomalies detected by the ML model in the modified base data set which is simply applying a model recited at a high level of generality and amounts to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instructions to implement an abstract idea or other exception on a computer (MPEP 2106.05(f)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
system, one or more processors, computer-readable medium, instructions amount(s) to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b))
applying a model amount(s) to no more than mere instructions to apply the exception (MPEP 2106.05(f))
machine learning (ML) model amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 20, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 20 is directed to a computer-readable medium, which is directed to an article of manufacture, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) computer-readable medium.
The limitation of identifying a base data set to be used as input to a machine learning (ML) model, the base data set including one or more sets of structured data, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of generating a modified base data set by causing synthetic anomaly injection operations to be performed on data of the base data set, the synthetic anomaly injection operations configured to create a set of known data modifications that are represented in the modified base data set, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of determining, based on the first output, the second output, and the set of known data modifications, a set of performance metrics representing performance of the ML model at detecting data anomalies, the set of performance metrics including: a true positive detection rate metric; a true negative detection rate metric; a false positive detection rate metric; and a false negative detection rate metric, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of outputting a representation of the set of performance metrics, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites additional element(s) – computer-readable medium, instructions, data-processing apparatus. The additional element(s) is/are recited at a high-level of generality (i.e., as generic computer components performing generic computer functions of executing instructions on the computers) such that it amounts to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b)).
The claim recites additional element(s) – machine learning (ML) model. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
The claim recites causing the ML model to run, using the base data set as input, to determine a first output of the ML model, wherein the first output includes a representation of data anomalies detected by the ML model in the base data set; causing the ML model to run, using the modified base data set as input, to determine a second output of the ML model, wherein the second output includes a representation of data anomalies detected by the ML model in the modified base data set which is simply applying a model recited at a high level of generality and amounts to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instructions to implement an abstract idea or other exception on a computer (MPEP 2106.05(f)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
computer-readable medium, instructions, data-processing apparatus amount(s) to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b))
applying a model amount(s) to no more than mere instructions to apply the exception (MPEP 2106.05(f))
machine learning (ML) model amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-6, 15, 17-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Buggineni et al. (Enhancing Manufacturing Operations with Synthetic Data: A Systematic Framework for Data Generation, Accuracy, and Utility, hereinafter referred to as “Buggineni”).
Regarding claim 1, Buggineni teaches a method performed by a computing system (Buggineni, sections 2.2-2.2.2 – teaches various computer-oriented environments for synthetic data generation; Buggineni, section 5 - teaches assembly systems; Buggineni, section 6.1 - teaches a Kaggle dataset; Buggineni, section 7 - teaches manufacturing systems), the method comprising:
identifying a base data set to be used as input to a machine learning (ML) model (Buggineni, sections 5- 5.1 – teaches collecting baseline dataset and using both baseline and synthetic datasets in machine learning models), the base data set including one or more sets of structured data (Buggineni, section 6.1 – teaches case study with tabular manufacturing dataset; see also Buggineni, Figure 3);
generating a modified base data set by causing synthetic anomaly injection operations to be performed on data of the base data set, the synthetic anomaly injection operations configured to create a set of known data modifications that are represented in the modified base data set (Buggineni, sections 5.2-5.3 – teaches generating a synthetic [modified] dataset from the baseline dataset; Buggineni, section 6.3 – teaches example known modifications made to a manufacturing dataset; see also Buggineni, Table 2);
causing the ML model to run, using the base data set as input, to determine a first output of the ML model, wherein the first output includes a representation of data anomalies detected by the ML model in the base data set (Buggineni, section 5.4.1 - teaches determining fidelity using metrics based on comparison of performance a model trained on baseline data v. the model trained on synthetic data; Buggineni, section 6.1 - teaches an example of anomaly detection in manufacturing data);
causing the ML model to run, using the modified base data set as input, to determine a second output of the ML model, wherein the second output includes a representation of data anomalies detected by the ML model in the modified base data set (Buggineni, section 5.4.1 - teaches determining fidelity using metrics based on comparison of performance a model trained on baseline data v. the model trained on synthetic data; Buggineni, section 6.1 - teaches an example of anomaly detection in manufacturing data);
determining, based on the first output, the second output, and the set of known data modifications, a set of performance metrics representing performance of the ML model at detecting data anomalies (Buggineni, section 5.4.1 - teaches determining fidelity using metrics based on comparison of performance a model trained on baseline data v. the model trained on synthetic data; Buggineni, section 6.1 - teaches an example of anomaly detection in manufacturing data), the set of performance metrics including: a true positive detection rate metric; a true negative detection rate metric; a false positive detection rate metric; and a false negative detection rate metric (Buggineni, section 5.4.1 - teaches determining fidelity using metrics based on comparison of performance a model trained on baseline data v. the model trained on synthetic data including using positive and negative samples; Buggineni, section 6.5 - teaches performance metrics based on true/false positives and true/false negatives); and
outputting a representation of the set of performance metrics (Buggineni, Tables 3-4 – demonstrate a representation of performance metrics).
Regarding claim 2, Buggineni teaches all of the limitations of the method of claim 1 as noted above. Buggineni further teaches wherein output of the ML model includes one or more of: a number of detected anomalies; one or more classifications of detected anomalies; one or more anomaly scores; one or more locations of detected anomalies in a respective data set used as input; and one or more values of data corresponding to detected anomalies (Buggineni, section 6 – teaches a case study using a binary classifier to identify anomalies; see also Buggineni, section 6.5 – teaches a true/false positive/negative count for anomaly classification).
Regarding claim 3, Buggineni teaches all of the limitations of the method of claim 2 as noted above. Buggineni further teaches wherein classifications of detected anomalies include one or more of: missing value; incorrect value; invalid value; incomplete row; duplicate row; temporal inconsistency; and schema change (Buggineni, section 6.1 – teaches identify anomalies based on invalid dimensions).
Regarding claim 4, Buggineni teaches all of the limitations of the method of claim 1 as noted above. Buggineni further teaches wherein the base data set and the modified base data set do not include labeled data that has been pre-identified as including anomalies (Buggineni, section 6.1 – teaches a dataset with dimensions and operators without labels of anomalous or non-anomalous; see also Buggineni, Figure 3 – excerpt of dataset).
Regarding claim 5, Buggineni teaches all of the limitations of the method of claim 1 as noted above. Buggineni further teaches wherein the one or more sets of structured data includes data in tabular form (Buggineni, section 6.1 – teaches a dataset with dimensions and operators; see also Buggineni, Figure 3 – excerpt of dataset in tabular form).
Regarding claim 6, Buggineni teaches all of the limitations of the method of claim 1 as noted above. Buggineni further teaches wherein data anomalies detected by the ML model represent an unexpected distribution change in one or more values in a respective data set used as input (Buggineni, section 5.4.2 - teaches using distribution comparisons; Buggineni, section 6.2 – teaches the anomaly is based on being outside of a confidence interval; see also Buggineni, Figure 4).
Regarding claim 15, Buggineni teaches all of the limitations of the method of claim 1 as noted above. Buggineni further teaches wherein the set of performance metrics includes an accuracy metric representing a percentage of time the ML model detected a known data modification of the set of known data modifications as a data anomaly (Buggineni, section 6.5 – teaches accuracy and precision for anomaly detection).
Regarding claim 17, Buggineni teaches all of the limitations of the method of claim 1 as noted above. Buggineni further teaches wherein the set of performance metrics includes one or more metrics representing one or more of: a length of time of running the ML model; a frequency of alerts generated to report data anomalies detected by the ML model; a precision of the ML model; a recall of the ML model; an F1 score of the ML model; and an average anomaly score output by ML model (Buggineni, section 6.5 – teaches equations for accuracy, precision, recall, and F-1 score).
Regarding claim 18, Buggineni teaches all of the limitations of the method of claim 1 as noted above. Buggineni further teaches wherein the representation of the set of performance metrics includes one or more visualizations (Buggineni, Tables 3-4 –tables for visualization of performance metrics).
Regarding claim 19, it is the system embodiment of claim 1 with similar limitations to claim 1 and is rejected using the same reasoning found in claim 1. Buggineni further teaches a system comprising: one or more processors; and a computer-readable medium storing instructions that are operable when executed by the one or more processors to perform operations comprising (Buggineni, section 5 - teaches assembly systems; Buggineni, section 6.1 - teaches a Kaggle dataset; Buggineni, section 7 - teaches manufacturing systems) …
Regarding claim 20, it is the system embodiment of claim 1 with similar limitations to claim 1 and is rejected using the same reasoning found in claim 1. Buggineni further teaches a non-transitory computer-readable medium storing instructions that are operable when executed by a data-processing apparatus to perform operations comprising (Buggineni, section 5 - teaches assembly systems; Buggineni, section 6.1 - teaches a Kaggle dataset; Buggineni, section 7 - teaches manufacturing systems) …
Claim(s) 7, 9-12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Buggineni in view of Neznal, Martin (US 2020/0380355 A1 – Anomaly Detection in Business Intelligence Time Series, hereinafter referred to as “Neznal”).
Regarding claim 7, Buggineni teaches all of the limitations of the method of claim 1 as noted above. However, Buggineni does not explicitly teach wherein causing the ML model to run, using a respective data set as input, to determine output includes causing the ML model to sequentially process time periods represented by the respective data set to determine data anomalies for each time period.
Neznal teaches wherein causing the ML model to run, using a respective data set as input, to determine output includes causing the ML model to sequentially process time periods represented by the respective data set to determine data anomalies for each time period (Neznal, [0027] – teaches determining anomalies for sequential time periods).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to modify Buggineni with the teachings of Neznal in order to better detect anomalies in the field of anomaly detection (Neznal, [0005] – “A need therefore exists for analyzing business intelligence data in computerized systems to better detect various patterns or anomalies in data.”).
Regarding claim 9, Buggineni teaches all of the limitations of the method of claim 1 as noted above. Buggineni further teaches wherein the first output and the second output each include:
a set of anomaly scores determined by the ML model (Buggineni, section 6.2 – teaches a binary anomaly score for each input); and
a set of anomaly score thresholds set by the ML model (Buggineni, section 6.2 – teaches a confidence interval threshold for determining an anomaly).
While Buggineni teaches determining anomalies based on data points outside of a confidence interval, Buggineni does not explicitly teach a set of anomaly score thresholds set by the ML model.
Neznal teaches wherein the first output and the second output each include:
a set of anomaly scores determined by the ML model (Neznal, [0036] – teaches determining anomaly scores for each input); and
a set of anomaly score thresholds set by the ML model (Neznal, [0036] – teaches dynamic anomaly thresholds based on time periods).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to modify Buggineni with the teachings of Neznal in order to better detect anomalies in the field of anomaly detection (Neznal, [0005] – “A need therefore exists for analyzing business intelligence data in computerized systems to better detect various patterns or anomalies in data.”).
Regarding claim 10, Buggineni in view of Neznal teaches all of the limitations of the method of claim 9 as noted above. Buggineni further teaches wherein the set of anomaly scores includes an anomaly score for each data entry of the one or more sets of structured data (Buggineni, section 6.2 – teaches a binary anomaly score for each input).
It would have been obvious to one of ordinary skill in the art before the filing data of the claimed invention to combine the teachings of Buggineni and Neznal for the same reasons as disclosed in claim 9 above.
Regarding claim 11, Buggineni in view of Neznal teaches all of the limitations of the method of claim 9 as noted above. Neznal further teaches wherein the set of anomaly score thresholds are dynamically set by the ML model over a set of time periods represented by the one or more sets of structured data (Neznal, [0036] – teaches determining anomaly scores for each input).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to combine the teachings of Buggineni and Neznal in order to dynamically set thresholds to better detect anomalies (Neznal, [0005]).
Regarding claim 12, Buggineni in view of Neznal teaches all of the limitations of the method of claim 9 as noted above. Buggineni further teaches wherein the ML model detects an anomaly for a data entry of a respective data set when an anomaly score determined for the data entry exceeds an applicable anomaly score threshold (Buggineni, section 6.2 – teaches determining anomalies when the data exceeds the confidence interval).
It would have been obvious to one of ordinary skill in the art before the filing data of the claimed invention to combine the teachings of Buggineni and Neznal for the same reasons as disclosed in claim 9 above.
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Buggineni in view of Lakshmanan et al. (US 2022/0004935 A1 – Ensemble Learning for Deep Feature Defect Detection, hereinafter referred to as “Lakshmanan”).
Regarding claim 8, Buggineni teaches all of the limitations of the method of claim 1 as noted above. However, Buggineni does not explicitly teach wherein causing the ML model to run, using the modified base data set as input, to determine the second output includes: causing the ML model to process multiple sets of structured data that have been subject to the synthetic anomaly injection operations, wherein processing each respective set of structured data of the multiple sets of structured data includes processing the respective set using an instance of the ML model that has not processed another set of structured data of the multiple sets of structured data.
Lakshmanan teaches wherein causing the ML model to run, using the modified base data set as input, to determine the second output includes:
causing the ML model to process multiple sets of structured data that have been subject to the synthetic anomaly injection operations, wherein processing each respective set of structured data of the multiple sets of structured data includes processing the respective set using an instance of the ML model that has not processed another set of structured data of the multiple sets of structured data (Lakshmanan, [0102]-[0105] – teaches clustering data and performing anomaly detection for each cluster using a corresponding instance of a probabilistic model [While the reference does not discuss synthetic data, the data is irrelevant to the process and it would be obvious to apply the process to synthetic data with the combination of Buggineni.]).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to modify Buggineni with the teachings of Lakshmanan in order to accommodate demanding workloads and increased performance expectations in the field of anomaly detection (Lakshmanan, [0017] – “Implementations of the disclosure describe ensemble learning for deep feature defect detection. In computer engineering, computing architecture is a set of rules and methods that describe the functionality, organization, and implementation of computer systems. Today's computing systems are expected to deliver near zero-wait responsiveness and superb performance while taking on large workloads for execution. Therefore, computing architectures have continually changed (e.g., improved) to accommodate demanding workloads and increased performance expectations.”).
Claim(s) 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Buggineni in view of Dande et al. (US 2025/0315720 A1 – Integrated Multimodal Artificial Intelligence Framework for Automated Provisioning Systems, hereinafter referred to as “Dande”).
Regarding claim 13, Buggineni teaches all of the limitations of the method of claim 1 as noted above. However, Buggineni does not explicitly teach causing a change to a hyperparameter of the ML model; causing the ML model with the change to the hyperparameter to run, using a data set as input, to determine third output of the ML model, wherein the third output includes a representation of data anomalies detected by the ML model in the data set; and determining a performance metric representing comparison of the performance of the ML model with and without the change to the hyperparameter.
Dande teaches
causing a change to a hyperparameter of the ML model (Dande, [0085] – teaches dynamically varying hyperparameters);
causing the ML model with the change to the hyperparameter to run, using a data set as input, to determine third output of the ML model, wherein the third output includes a representation of data anomalies detected by the ML model in the data set (Dande,[0085] – teaches that for each iteration of hyperparameter variation, rerunning the data to generate output; see also Dande, [0104] – teaches anomaly detection); and
determining a performance metric representing comparison of the performance of the ML model with and without the change to the hyperparameter (Dande, [0085] – teaches for each iteration of hyperparameter variation, comparing the accuracy to previous hyperparameter variations).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to modify Buggineni with the teachings of Dande in order to significantly improve overall system performance in the field of anomaly detection (Dande, [0005] – “Systems, methods, and computer program products are provided for an integrated multimodal artificial intelligence framework for automated provisioning systems. With regard to large institutional entity operations, the proliferation of complex workflows and the frequent need for manual intervention and error management present significant challenges. To address these, the invention introduces a sophisticated multimodal artificial intelligence (AI) system designed to enhance the efficiency, accuracy, and agility of application workflows. This AI system leverages a combination of natural language processing (NLP), computer vision, and machine learning modalities to autonomously monitor transactions, resource flows, account actions, decisions, and errors across disparate systems. By learning from patterns over time, the system gains a deep understanding of both existing and newly introduced processes. This enables the system to autonomously manage decision points, either by prompting for human intervention where necessary, or by executing manual steps and notifying support teams. Furthermore, the system is capable of processing exceptions by undertaking any necessary reprocessing of actions to expedite the triage process, thereby significantly improving the overall system's performance.”).
Claim(s) 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Buggineni in view of Rieger et al. (US 2024/0411298 A1 – Detection of Behavior Anomalies in Control Systems and Related Systems, Methods, and Devices, hereinafter referred to as “Rieger”).
Regarding claim 14, Buggineni teaches all of the limitations of the method of claim 1 as noted above. However, Buggineni does not explicitly teach wherein the set of performance metrics includes a metric representing an area under a curve, wherein the curve is a plot of the ratio of the true positive detection rate metric versus the false positive detection rate metric over a variable set of values of an anomaly score threshold of the ML model.
Rieger teaches wherein the set of performance metrics includes a metric representing an area under a curve, wherein the curve is a plot of the ratio of the true positive detection rate metric versus the false positive detection rate metric over a variable set of values of an anomaly score threshold of the ML model (Rieger, [0052] - teaches anomaly detection with ML model; Rieger, [0080]-[0081] - teaches AUC for anomaly detection comprising true positive rate v. false positive rate over score thresholds).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to modify Buggineni with the teachings of Rieger in order to influence the efficiency of anomaly detection in the field of anomaly detection (Rieger, [0038] – “Various embodiments disclosed herein provide an ample evaluation of EM-based anomaly detection defenses in the presence of EM noise. Minute modifications in the control logic that are likely to go unnoticed are disclosed herein. The limits of EM-based anomaly detection are identified, and the impact of several factors influencing the efficiency of the corresponding defense mechanisms are quantified herein.”).
Claim(s) 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Buggineni in view of Bechara Prado et al. (US 2025/0315679 A1 – Generating Counterfactual Explanations from Human Preferences, hereinafter referred to as “Bechara”).
Regarding claim 16, Buggineni teaches all of the limitations of the method of claim 1 as noted above. However, Buggineni does not explicitly teach wherein the second output includes descriptions of detected anomalies; and wherein the set of performance metrics includes one or more metrics representing whether the descriptions of the detected anomalies included in the second output match corresponding descriptions of the known data modifications that were detected as data anomalies by the ML model.
Bechara teaches
wherein the second output includes descriptions of detected anomalies (Bechara, [0053] - teaches generating explanations for anomalies that match known SME anomaly explanations; Bechara, [0076] - teaches providing a SME explanation for an anomaly; see also Bechara, Fig. 3); and
wherein the set of performance metrics includes one or more metrics representing whether the descriptions of the detected anomalies included in the second output match corresponding descriptions of the known data modifications that were detected as data anomalies by the ML model (Bechara, [0053] - teaches generating explanations for anomalies that match known SME anomaly explanations; Bechara, [0076] - teaches providing a SME explanation for an anomaly; see also Bechara, Fig. 3).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to modify Buggineni with the teachings of Bechara in order to improve the operation and effectiveness of machine learning models in their evaluation and explanation of anomalous data in the field of anomaly detection (Bechara, [0014] – “In particular, one advantageous aspect of an embodiment that counterfactual explanations may be generated that can be readily understood by a human. An embodiment may improve the operation and effectiveness of ML (machine learning) models in their evaluation and explanation of anomalous time series data, and other problems relating to time series data and time series domains. Various other advantages of one or more example embodiments will be apparent from this disclosure.”).
Conclusion
Any inquiry concerning this communication or earlier communication from the examiner should be directed to MARSHALL WERNER whose telephone number is (469) 295-9143. The examiner can normally be reached on Monday – Thursday 7:30 AM – 4:30 PM ET.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kamran Afshar, can be reached at (571) 272-7796. The fax number for the organization where this application or proceeding is assigned is (571) 273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MARSHALL L WERNER/ Primary Examiner, Art Unit 2125