DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-22 are presented for examination.
Claim Objections
Claims 1 is objected to because of the following informalities:
Claim 1 [line 19]: the phrase “progressively reduces the impact of each data sample on the training of the AI model the less recent the data sample” is awkward and unclear. This could be rephrased as “progressively reduces the impact of each data sample on the training of the AI model as the data sample becomes less recent.”
Appropriate corrections are required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 10 and 11 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
With respect to claim 10, it is unclear what “the predicted classification that relates to financial accounting classifications.” [line 1] refers to. Claim 10 is depended on claim 8, and claim 8 is depended on claim 1. However, both claims 1 and 8 never recite “a predicted classification.” For the purposes of examination, examiner will interpret the limitation as “the classification data is associated with a predicted classification that relates to financial accounting classifications.”
With respect to claim 11, it is rejected based on its virtual dependency of claim 10.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-22 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Independent claims
Step 1
Claim 1 is drawn to a method and claim 12 is drawn to a system that comprises a covariate drift detection unit and a model retraining module to execute the method of claim 1. Therefore, each of these claim groups falls under one of four categories of statutory subject matter (process/method, machines/product/apparatus, manufactures, and composition of matter).
Step 2A – Prong 1
Claims 1 and 12 are directed to a judicially recognized exception of an abstract idea without significantly more.
Claims 1 and 12 recite a method of applying a covariate shift quantification process to said input data, wherein said covariate shift quantification process comprises computing a statistical value to quantify the drift in said input data relative to said first training dataset that under its broadest reasonable interpretation enumerates a mathematical concept. A human can perform the calculation using words or using mathematical symbols to compute a value to quantify a drift. Therefore, the step of applying a covariate shift that comprises computing a statistical value is nothing more than a mathematical concept (MPEP 2106.04(a)(2)(I)).
Claims 1 and 12 recite a method of comparing said statistical value with a predetermined threshold that under its broadest reasonable interpretation enumerates a mathematical concept. A human can perform the calculation using words or using mathematical symbols to compare a value with a threshold. Therefore, the step of comparing a statistical value with a threshold is nothing more than a mathematical concept (MPEP 2106.04(a)(2)(I)).
Claims 1 and 12 recite a method of determining that a covariate shift has occurred when said statistical value exceeds said predetermined threshold that under its broadest reasonable interpretation enumerates a mathematical concept. A human can perform the calculation using words or using mathematical symbols to determine a covariate shift has occurred by comparing the value with the threshold. Therefore, the step of determining a covariate shift has occurred when statistical value exceeds a predetermined threshold is nothing more than a mathematical concept (MPEP 2106.04(a)(2)(I)).
Claims 1 and 12 recite a method of generating from the further training dataset a plurality of candidate training datasets, each generated by applying to the further training dataset a different combination of different temporal windows and different temporal weighting decay rates, wherein each different temporal window specifies a different time range for data samples of the training dataset used, and each different temporal weighting decay rate applies a different decay rate which progressively reduces the impact of each data sample on the training of the AI model the less recent the data sample that under its broadest reasonable interpretation enumerates a mathematical concept. A human can perform the calculation using words or using mathematical symbols to generate a plurality of candidate training datasets that each candidate is a combination of a temporal window and a temporal weighting decay rate. Therefore, the step of generating a plurality of candidate training datasets wherein each candidate is a combination of different temporal windows and different temporal weighting decay rates is nothing more than a mathematical concept (MPEP 2106.04(a)(2)(I)).
Claims 1 and 12 recite a method of evaluating performance of said plurality of candidate training datasets using model simulations trained on each of said candidate training datasets and tested against a benchmark dataset that under its broadest reasonable interpretation enumerates a mathematical concept. A human can perform the calculation using words or using mathematical symbols to evaluate performance by using some simulations and testing against a benchmark. Therefore, the step of evaluating performance using model simulations trained on each candidate training dataset and tested against a benchmark dataset is nothing more than a mathematical concept (MPEP 2106.04(a)(2)(I)).
Claims 1 and 12 recite a method of selecting the candidate training dataset with the combination of temporal window and temporal weighting decay rate that yields the highest performance in said evaluation for retraining said AI model that under its broadest reasonable interpretation enumerates a mathematical concept. A human can perform the calculation using words or using mathematical symbols to select a candidate training dataset that yields highest performance. Therefore, the step of selecting the candidate training dataset that yields the highest performance in the evaluation is nothing more than a mathematical concept (MPEP 2106.04(a)(2)(I)).
Step 2A – Prong 2
Claims 1 and 12 recite further a method of receiving input data intended for said AI model that fails to integrate the abstract idea into a practical application. The step of receiving input data is a form of insignificant input and output solution activities, where receiving input data is necessary for all uses of the judicial exception. This additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (MPEP 2106.05(g)).
Claims 1 and 12 recite further a method of triggering a retraining process for said AI model in response to said determination that a covariate shift has occurred that fails to integrate the abstract idea into a practical application. The step of triggering the retraining process is a form of insignificant input and output solution activities, where triggering retraining process in response to the determination is necessary for all uses of the judicial exception. This additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (MPEP 2106.05(g)).
Claims 1 and 12 recite further a method of retrieving a further training dataset comprising at least training data based on input data and prediction data from operation of the system after the first training dataset was generated that fails to integrate the abstract idea into a practical application. The step of retrieving a training dataset is a form of insignificant input and output solution activities, where retrieving a further training dataset comprising training data based on input data and prediction data is necessary for all uses of the judicial exception. This additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (MPEP 2106.05(g)).
Claims 1 and 12 recite further a method of retraining the AI model with the selected candidate training dataset that fails to integrate the abstract idea into a practical application. The step of retraining the AI model is a form of insignificant input and output solution activities, where retraining the AI model with selected training dataset is necessary for all uses of the judicial exception. This additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (MPEP 2106.05(g)).
Step 2B
The additional elements in step 2A-Prong 2 those are forms of insignificant extra-solution activities, do not amount to significantly more than an abstract idea because the court decision have determined that these additional elements of receiving input data; triggering retraining process in response to the determination; retrieving a further training dataset comprising training data based on input data and prediction data; and retraining the AI model with selected training dataset to be well-understood, routine, and conventional when claimed in a merely generic manner (MPEP 2106.05(d)(II)).
As such, claims 1 and 12 are not patent eligible.
Dependent claims
Claims 2-11 and 13-22 merely narrow the previously recited abstract idea limitations. For the reasons described above with respect to claims 1 and 11, this judicial exception is not meaningfully integrated into a practical application, or significantly more than the abstract idea. The claims disclose similar limitations described for the independent claims above and do not provide anything more than the mental processes that are practically capable of being performed in the human mind with the assistance of pen. Therefore, claims 2-11 and 13-22 also recite abstract ideas that do not integrate into a practical application or amount to significantly more than the judicial exception, and are rejected under U.S.C. 101.
Step 1
Claims 2-11 are drawn to a method and claims 13-22 are drawn to a system that comprises a covariate drift detection unit and a model retraining module to execute the method of claims 2-11. Therefore, each of these claim groups falls under one of four categories of statutory subject matter (process/method, machines/product/apparatus, manufactures, and composition of matter).
Step 2A – Prong 1
Dependent claims 3 and 14 recite further the mathematical process by analysing said data samples from the first training dataset to detect any samples subject to an above-threshold amount of covariate shift relative to the input data; and removing from the further training dataset those detected samples that exceed the above-threshold amount of covariate shiftthat based on one or more features of the ML project (MPEP 2106.04(a)(2)(I)).
Dependent claims 4 and 15 recite further the mathematical process by wherein the statistical value in step b is computed using an L-infinity norm process that based on one or more features of the ML project (MPEP 2106.04(a)(2)(I)).
Dependent claims 5 and 16 recite further the mathematical process by wherein the L-infinity norm process is applied on a data sample-by-data sample basis, comparing each data sample of the input data to corresponding data samples in the first training dataset that based on one or more features of the ML project (MPEP 2106.04(a)(2)(I)).
Dependent claims 6 and 17 recite further the mathematical process by wherein each different temporal weighting decay rate corresponds to a different exponential decay curve that based on one or more features of the ML project (MPEP 2106.04(a)(2)(I)).
Step 2A – Prong 2
Dependent claims 2 and 13 recite further the insignificant extra solution activities by wherein the further training dataset comprises a combination of data samples from the first training dataset and data samples generated from subsequent operation of the system after the AI model was trained on the first training dataset. This additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (MPEP 2106.05(g)).
Dependent claims 7 and 18 recite further the insignificant extra solution activities by wherein the output prediction data comprises classification data associated with a predicted classification of a property of the input data. This additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (MPEP 2106.05(g)).
Dependent claims 8 and 19 recite further the insignificant extra solution activities by wherein the input data is associated with financial transaction data. This additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (MPEP 2106.05(g)).
Dependent claims 9 and 20 recite further the insignificant extra solution activities by wherein the input data comprises data relating to one or more of: invoices, receipts, purchase orders, quotations, contracts, bank statements, credit memos, debit notes, financial reports, expense reports, billing statements, payroll records, and tax forms. This additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (MPEP 2106.05(g)).
Dependent claims 10 and 21 recite further the insignificant extra solution activities by wherein the predicted classification relates to financial accounting classifications. This additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (MPEP 2106.05(g)).
Dependent claims 11 and 22 recite further the insignificant extra solution activities by wherein the predicted classification comprises assigning a General Ledger (GL) code. This additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (MPEP 2106.05(g)).
As such, dependent claims 2-11 and 13-22 are not patent eligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, 6-13, and 17-22 are rejected under 35 U.S.C. 103 as being unpatentable over Singh et al (US 20220024032 A1) hereafter Singh, and further in view of Maier et al (US 12339926 A1) hereafter Maier.
With respect to claim 1, Singh teaches a computer implemented method for covariate drift correction in a system that employs an AI model trained on a first training dataset for generating output prediction data (a computer-implemented method for performing AI/ML model drift detection and correction for RPA includes analyzing the input dataset for an AI/ML model. The method includes analyzing information including statistical distributions of predictions made by the model [par. 0003-0006]), the method comprising:
receiving input data intended for said AI model (the information pertains to input data for an AI/ML model to determine whether data drift has occurred [par. 0003-0006]);
applying a covariate shift quantification process to said input data, wherein said covariate shift quantification process comprises computing a statistical value to quantify the drift in said input data relative to said first training dataset (the method includes analyzing the information including one or more statistical distributions of predictions made by the AI/ML model. The analysis includes determining whether a statistical moment changes, wherein the statistical moment includes a mean, a variance, a skewness, a kurtosis, a covariance, or a combination thereof [par. 0003-0006, 0016-0023]);
comparing said statistical value with a predetermined threshold (based on the analysis of the information, a change condition is found, a change threshold is met or exceeded after mapping one or more statistical distributions to a respective action token by a plurality of RPA robots. A change threshold is met or exceeded if model statistical performance deviates from historical performance by at least a certain amount [par. 0003-0006, 0016-0018, 0066-0068]);
determining that a covariate shift has occurred when said statistical value exceeds said predetermined threshold (a change threshold is met or exceeded when the information is determined that data drift has occurred. The analysis includes measuring one or more distances between unidimensional distributions of an original dataset and a new dataset for covariate drift. The analysis includes determining whether a statistical moment changes that leads to whether the data drift has occurred [par. 0003-0006, 0082-0086]); and
triggering a retraining process for said AI model in response to said determination that a covariate shift has occurred (the method includes retraining AI/ML model after determining whether a change threshold is met or exceeded, verifying that the retrained AI/ML model meets one or more performance thresholds to deploy the retrained AI/ML model. [par. 0003-0006, 0016-0018, 0026, 0066-0069]), wherein the retraining process comprises the steps of:
f. retrieving a further training dataset comprising at least training data based on input data and prediction data from operation of the system after the first training dataset was generated (the information may include what predictions were made (the output of the AI/ML model), how many predictions the AI/ML model made and how many were actually used, one or more statistical distributions of the predictions (normal distribution, a binomial distribution, etc.), and/or the input data was provided to the AI/ML model. The AI/ML model can then be retrained using the collected information and/or other information to attempt to improve the performance of AI/ML model [par. 0066-0069]).
However, Singh does not explicitly disclose g. generating from the further training dataset a plurality of candidate training datasets, each generated by applying to the further training dataset a different combination of different temporal windows and different temporal weighting decay rates, wherein each different temporal window specifies a different time range for data samples of the training dataset used, and each different temporal weighting decay rate applies a different decay rate which progressively reduces the impact of each data sample on the training of the AI model the less recent the data sample; h. evaluating performance of said plurality of candidate training datasets using model simulations trained on each of said candidate training datasets and tested against a benchmark dataset; and i. selecting the candidate training dataset with the combination of temporal window and temporal weighting decay rate that yields the highest performance in said evaluation for retraining said AI model, and j. retraining the AI model with the selected candidate training dataset.
In the same field of endeavor, Maier teaches g. generating from the further training dataset a plurality of candidate training datasets (a method for dynamic model training of algorithmic underwriting predictive ML model accesses data points of a training dataset. The method determines a current data distribution for the selected covariates, when the current data distribution is compared with the historical data distribution to indicate a data distribution shift exceeding a predetermined threshold [col. 1, line 50 – col. 3, line 50]), each generated by applying to the further training dataset a different combination of different temporal windows and different temporal weighting decay rates, wherein each different temporal window specifies a different time range for data samples of the training dataset used, and each different temporal weighting decay rate applies a different decay rate which progressively reduces the impact of each data sample on the training of the AI model the less recent the data sample (temporal weighting is a method where data samples are assigned weights based on their recency, so that the most recent observations have a greater influence on model training or estimation while the older samples contribute less. The predetermined threshold is a value of temporal drift of the current data distribution relative to the historical data distribution. Each of the historical application records includes a time metric falling within a time period of the historical application records, and the test dataset comprises data points including the time metric wherein the time metric falls within a recent time period [col. 1, line 50 – col. 3, line 50]);
h. evaluating performance of said plurality of candidate training datasets using model simulations trained on each of said candidate training datasets and tested against a benchmark dataset (the algorithmic underwriting system continually monitors model inputs and outputs. The system monitors the model performance using test data 240 that has been held back from model training data. Test data represents a separate test dataset of applicants that have previously been processed via algorithmic underwriting during a recent time period. [col. 4, lines 25-50; col. 7, lines 20-55; col. 9, lines 50-65; col. 11, lines 1-45]); and
i. selecting the candidate training dataset with the combination of temporal window and temporal weighting decay rate that yields the highest performance in said evaluation for retraining said AI model (the Covariate Shift Adaptation 160 compares a training dataset including a plurality of historical application records with a test dataset including records 240. The Covariate Shift Adaptation determines a current data distribution based on the selected covariate by applying the predictive ML model to the test dataset [col. 9, lines 50-65; col. 11, lines 1-45]), and
j. retraining the AI model with the selected candidate training dataset (as the covariate shift exceeds the predetermined threshold, the Analytical Engine 114 automatically updates one or more parameters of the predictive ML model and retrains the predictive ML model using the updated parameters [col. 1, line 50 – col. 3, line 50; col. 9, lines 50-65; col. 11, lines 1-45; col. 14, lines 5-50]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have incorporated the concept of dynamic model training of a predictive ML model accesses data points of a training dataset including a plurality of model covariates as suggested by Maier into the concept of AI/ML model drift detection and correction for robotic process automation (RPA) as suggested by Singh because both of the systems addressing the process of training the AI/ML model by comparing the input dataset with the test dataset and determining whether a statistical value meets or exceeds a certain threshold. Doing so would be desirable because the concept of Singh would be more efficient by applying a temporal adjustment to the selected covariate in each of the plurality of historical application records in the model retraining, such that the test dataset comprises data points including time metric that falls within a recent time period (Maier, [col. 1, line 50 – col. 3, line 50]).
With respect to claim 2, the combination of Singh and Maier teaches wherein the further training dataset comprises a combination of data samples from the first training dataset and data samples generated from subsequent operation of the system after the AI model was trained on the first training dataset (Maier, the method improves the predictive ML model by comparing the training dataset to test data samples in order to detect data distribution shifts, wherein the training dataset includes a plurality of historical records and the test dataset comprises data points including the time metric that falls within a recent time period [col. 1, line 50 – col. 3, line 50]).
With respect to claim 6, the combination of Singh and Maier may not teach wherein each different temporal weighting decay rate corresponds to a different exponential decay curve.
With respect to claim 7, the combination of Singh and Maier teaches wherein the output prediction data comprises classification data associated with a predicted classification of a property of the input data (Maier, model covariates may classify the mortality risk factors into clinical risk factors and non-clinical risk factors for applicants for life insurance products or other financial products. The algorithmic underwriting system manages the predictive modeling of mortality risk factors that exclude clinical assessment risk factors for applicants for life insurance or other financial products [col. 7, line 40 – col. 8, line 15]).
With respect to claim 8, the combination of Singh and Maier teaches wherein the input data is associated with financial transaction data (Maier, a sponsoring enterprise for algorithmic underwriting system may be an insurance company or other financial services company which may be represented by insurance agents or advisors. The input data may be retrieved from the current applications database 220 that stores current data on applications for underwriting. User may submit electronic application data via inputs at user device [col. 7, line 40 – col. 8, line 15; col. 9, lines 25-35]).
With respect to claim 9, the combination of Singh and Maier teaches wherein the input data comprises data relating to one or more of: invoices, receipts, purchase orders, quotations, contracts, bank statements, credit memos, debit notes, financial reports, expense reports, billing statements, payroll records, and tax forms (Maier, a sponsoring enterprise for algorithmic underwriting system may be an insurance company or other financial services company which may be represented by insurance agents or advisors. The input data may be retrieved from the current applications database 220 that stores current data on applications for underwriting. User may submit electronic application data via inputs at user device. Public records include attributes that pertain to individual-level records that are filled by a public office, such as addresses, education, licenses, property, assets, and financial disclosures [col. 7, line 40 – col. 8, line 15; col. 9, lines 25-35; col. 12, lines 25-40]).
With respect to claim 10, the combination of Singh and Maier teaches wherein the predicted classification relates to financial accounting classifications (Maier, public records include attributes that pertain to individual-level records that are filled by a public office, such as addresses, education, licenses, property, assets, and financial disclosures. Credit risk attributes may be classified into the number of collections, ratio of amount past due to amount of total balances, and number of open auto finance accounts [col. 12, lines 25-40]).
With respect to claim 11, the combination of Singh and Maier may not teach wherein the predicted classification comprises assigning a General Ledger (GL) code.
With respect to claim 12, it is a system claim that is corresponding to the method of claim 1. Therefore, it is rejected for the same reason as claimed in claim 1 above.
With respect to claim 13, it is a system claim that is corresponding to the method of claim 2. Therefore, it is rejected for the same reason as claimed in claim 2 above.
With respect to claim 17, it is a system claim that is corresponding to the method of claim 6. Therefore, it is rejected for the same reason as claimed in claim 6 above.
With respect to claim 18, it is a system claim that is corresponding to the method of claim 7. Therefore, it is rejected for the same reason as claimed in claim 7 above.
With respect to claim 19, it is a system claim that is corresponding to the method of claim 8. Therefore, it is rejected for the same reason as claimed in claim 8 above.
With respect to claim 20, it is a system claim that is corresponding to the method of claim 9. Therefore, it is rejected for the same reason as claimed in claim 9 above.
With respect to claim 21, it is a system claim that is corresponding to the method of claim 10. Therefore, it is rejected for the same reason as claimed in claim 10 above.
With respect to claim 22, it is a system claim that is corresponding to the method of claim 11. Therefore, it is rejected for the same reason as claimed in claim 11 above.
Claims 3 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Singh et al (US 20220024032 A1) hereafter Singh, further in view of Maier et al (US 12339926 A1) hereafter Maier, as claimed in claim 1 above, and further in view of Balles et al (US 12423960 B1) hereafter Balles.
With respect to claim 3, the combination of Singh and Maier teaches wherein the method further comprises:
analysing said data samples from the first training dataset to detect any samples subject to an above-threshold amount of covariate shift relative to the input data (Maier, the training dataset includes model covariates those may be used to determine the current data distribution. The current data distribution is compared with the historical data distribution to indicate a data distribution shift exceeding a predetermined threshold [col. 1, line 50 – col. 3, line 50]).
However, the combination of Singh and Maier does not explicitly disclose removing from the further training dataset those detected samples that exceed the above-threshold amount of covariate shift.
In the same field of endeavor, Balles teaches removing from the further training dataset those detected samples that exceed the above-threshold amount of covariate shift (when a concept drift is detected with a sufficient likelihood, the sliding window can be emptied to remove the previous data elements from further use in training to eliminate the incorrect samples from use. The user-specified criteria may include one or more threshold values for the data element before indicating that a likely concept drift has occurred [col. 2, lines 20-55; col. 8, line 30 - col. 9, line 30]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have incorporated the concept of drift detection and correction using probabilistic models using primary ML model for existing data elements as suggested by Balles into the combination of Singh and Maier because all of these systems addressing the process of training the AI/ML model by comparing the input dataset with the test dataset and determining whether a statistical value meets or exceeds a certain threshold. Doing so would be desirable because the combination of Singh and Maier would be more efficient by removing the data samples those may exceed the threshold values before indicating that a likely concept drift has occurred, and the user may define a criteria indicating that an automatic update of the training memory and subsequent retraining of the ML model (Balles, [col. 2, lines 20-55; col. 8, line 30 - col. 9, line 30]).
With respect to claim 14, it is a system claim that is corresponding to the method of claim 3. Therefore, it is rejected for the same reason as claimed in claim 3 above.
Claims 4, 5, 15 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Singh et al (US 20220024032 A1) hereafter Singh, further in view of Maier et al (US 12339926 A1) hereafter Maier, as claimed in claim 1 above, and further in view of Weimer et al (US 20240225561 A1) hereafter Weimer.
With respect to claim 4, the combination of Singh and Maier does not explicitly disclose wherein the statistical value in step b is computed using an L-infinity norm process.
In the same field of endeavor, Weimer teaches wherein the statistical value in step b is computed using an L-infinity norm process (test statistic engineering is disclosed for discriminating between and neurologically intact subjects. Motion distribution covariate shift is common in passive monitoring scenarios and captures the effect of any patient-specific tendency in the data. However, the impact of the motion covariate shift will be limited by the patient’s neurological state. A test statistic is generated that represents a non-parametric statistic of distribution equality that equals the maximum absolute deviation of the cumulative distribution functions corresponding to the probability mass functions [par. 0093-0098]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have incorporated the concept of detecting stroke by monitoring of upper limb movements as suggested by Weimer into the combination of Singh and Maier because all of these systems addressing the process of covariate shift to output desire results by comparing specific data elements. Doing so would be desirable because the combination of Singh and Maier would be more efficient by generating a test statistic that employs the maximum absolute deviation (L-infinity norm curve) of cumulative distribution functions to compare with the threshold such that the resulting test has a constant rate (Weimer, [par. 0093-0098]).
With respect to claim 5, the combination of Singh, Maier and Weimer teaches wherein the L-infinity norm process is applied on a data sample-by-data sample basis, comparing each data sample of the input data to corresponding data samples in the first training dataset (Weimer, a test statistic is generated that represents a non-parametric statistic of distribution equality that equals the maximum absolute deviation of the cumulative distribution functions corresponding to the probability mass functions. The test statistic is used among the sampled data points, and a threshold test is developed for the test statistic to generate a constant false alarm rate [par. 0093-0098, 0100-0102]).
With respect to claim 15, it is a system claim that is corresponding to the method of claim 4. Therefore, it is rejected for the same reason as claimed in claim 4 above.
With respect to claim 16, it is a system claim that is corresponding to the method of claim 5. Therefore, it is rejected for the same reason as claimed in claim 5 above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Sasson et al (US 20210035021 A1) disclosed a method of monitoring performance of a machine learning model externally to the machine learning model, comprising: monitoring data elements being fed into a machine learning model trained on a training dataset of historical training data elements, wherein the data elements are each associated with a respective time after the time associated with the training dataset, analyzing the data elements for identifying shift(s) between at least two subsets of the data elements, computing according to the shift(s), measurement(s) denoting an expected effect on output of the model, and detecting a misclassification event by the model when the measurement(s) exceeds a threshold of the model, wherein the monitoring, the analyzing, the computing, and the detecting are performed externally to the model, without accessing at least one of: data stored within the machine learning model, an implementation of the model, and data structures of the model.
Goldszmidt et al (US 20210224687 A1) disclosed a method for triggering an update to a machine-learning model upon detecting that a distribution of particular (e.g., recently collected) input data set is sufficiently different from a distribution training input data set used to train the model. The distributions may be determined to be sufficiently different when a classifier can identify to which distribution individual data elements belong (e.g., to at least a predetermined degree). An update to the machine-learning model can include morphing weights used by the model and/or retraining the model.
Ford et al (US 20240333615 A1) disclosed a method for automating configuration management in cellular networks. A method of a computing device comprises: assigning, based on a correlation analysis, contexts to different time intervals of data, wherein the correlation analysis is performed based on historic time-series data; grouping, based on the assigned contexts, the historic time-series data; identifying context and compute an anomaly score comparing new data and the grouped historic-time series data of the context; indicating an event of anomaly based on a determination that the computed anomaly score exceeds a first threshold that is identified based on a function of per-context data; and computing, based on the event of the anomaly, an aggregate anomaly score or indicate using a value of mean or moving average of a set of latest anomaly scores, for a context-based multivariate anomaly detection.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Quoc Phung whose telephone number is (703) 756 1330. The examiner can normally be reached on Monday through Friday from 9am to 5pm PT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Welch can be reached on 571-272-7212. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Q.L.P./Examiner, Art Unit 2143
/BEAU D SPRATT/Primary Examiner, Art Unit 2143
Prosecution Insights