DETAILED ACTION
Claims 1-12 are pending. Claims 1-12 are considered in this Office action.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 5/6/2023 has been acknowledged. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner. The initialed and dated copy of Applicant’s IDS form 1449 is attached to the instant Office action.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-12 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claims 1, 11, and 12 recite limitations for obtaining data sets, each datum of a data set corresponding to the output values that the prediction algorithm should give in the presence of the input values of the data set, reception of the probability, for each data set, that a data set is observed during use case of the prediction algorithm (Collecting Information, an observation, a Mental Process; a Fundamental Economic Practice, i.e. data analytics; a Certain Method of Organizing Human Activity), collecting the outputs predicted by the prediction algorithm for each data input value of the data sets (Collecting the Information, an observation, a Mental Process; a Fundamental Economic Practice, i.e. data analytics; a Certain Method of Organizing Human Activity), determining the distribution of the prediction precision of the predicted output for each data set, for obtaining determined distributions (Analyzing the Information, an evaluation, a Mental Process; a Fundamental Economic Practice, i.e. data analytics; a Certain Method of Organizing Human Activity), aggregating distributions determined by using an aggregation function using the probabilities received, for obtaining an aggregated distribution of prediction precision (Analyzing the Information, an evaluation, a Mental Process; a Fundamental Economic Practice, i.e. data analytics; a Certain Method of Organizing Human Activity), and applying at least one risk metric to the aggregated distribution of prediction precision, for obtaining at least one indicator of the performance of the prediction algorithm (Analyzing/Transmitting the Information, an evaluation and judgment, a Mental Process; a Fundamental Economic Practice, i.e. data analytics; a Certain Method of Organizing Human Activity), which under their broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. That is, other than reciting using a prediction algorithm having been trained using a machine learning technique and a learning dataset, a computer program product, computer storage medium, and a data processing unit, nothing in the claim element precludes the step from practically being performed or read into the mind for the purposes of analyzing information in a dataset. For example determining the distribution of the prediction precision of the predicted output for each data set, for obtaining determined distributions encompasses a data analyst or supervisor receiving information of datasets and deciding/determining a distribution to be used, an observation, evaluation, and judgment . If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Further, as described above, these processes recite limitations for a Fundamental Economic Process, a “Method of Organizing Human Activity”. Accordingly, the claim recites an abstract idea.
This judicial exception is not integrated into a practical application. In particular, the claim recites the above stated additional elements to perform the abstract limitations as above. The algorithm/learning, computer program product, storage medium, and data processing unit are recited at a high-level of generality (i.e., as a generic processor/module performing a generic computer function of storing, retrieving, sending, and processing data) such that they amounts no more than mere instructions to apply the exception using generic computer components. Even if taken as an additional element, the receiving and transmission steps are insignificant extra-solution activity as these are receiving, storing, and transmitting data as per the MPEP 2106.05(d). Accordingly, these additional elements do not integrate the abstract idea into a practical application because they not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception, when considered both individually and as an ordered combination. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements being used to perform the abstract limitations stated above amounts to no more than mere instructions to apply the exception using generic computer components. Mere instructions to apply an exception using generic computer components cannot provide an inventive concept. The claim is not patent eligible. Applicant’s specification states:
“The system 10 is a desktop computer. In a variant, the system 10 is a computer
mounted on a rack, a laptop, a tablet, a personal digital assistant (PDA) or a smartphone. “
Which is an example of a generic processing unit/computer system, as per the specification above, and from this interpretation, one would reasonably deduce the aforementioned steps are all functions that can be done on generic components, such as a laptop, desktop, tablet, etc., and thus application of an abstract idea on a generic computer, as per the Alice decision and not similar to Berkheimer, but for edification the Applicant’s specification has been used as above satisfying any such requirement. For the collecting and transmission steps that were considered extra-solution activity in Step 2A above, if they were to be considered an additional element, they have been re-evaluated in Step 2B and determined to be well-understood, routine, conventional, activity in the field. The background does not provide any indication that the additional elements, such as the data processing unit, medium, etc., or the receiving and transmitting steps, are anything other than a generic, and the MPEP Section 2106.05(d) indicates that mere collection or receipt, storing, or transmission of data is a well‐understood, routine, and conventional function when it is claimed in a merely generic manner (as it is here). For these reasons, there is no inventive concept. The claim is not patent eligible.
Claims 2-10 contain the identified abstract ideas, further narrowing them, with the no new additional elements to be considered under prong 2 as part of the Alice analysis of the MPEP for a practical or under 2B, and thus not significantly more for the same reasons and rationale as above.
After considering all claim elements, both individually and in combination, Examiner has determined that the claims are directed to the above abstract ideas and do not amount to significantly more. Therefore, the claims and dependent claims are rejected under 35 U.S.C. 101 as being directed to non-statutory subject matter. See Alice Corporation Pty. Ltd. v. CLS Bank International, No. 13–298.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-12 are rejected under 35 U.S.C. 103 as being unpatentable over Bradley (U.S. Publication No. 2022/024,5492) in view of Adaptive (NPL - Adaptive Sampling for Stochastic Risk-Averse Learning - 2020).
Regarding Claims 1, 11, and 12, Bradley, a system and method for constructing a statistical model and evaluating model performance, teaches the method including the steps of:
obtaining data sets, each datum of a data set corresponding to the output values that the prediction algorithm should give in the presence of the input values of the data set, reception of the probability, for each data set, that a data set is observed during use case of the prediction algorithm,
collecting the outputs predicted by the prediction algorithm for each data input value of the data sets ([0106] outputs are collected from the prediction algorithm)
aggregating distributions determined by using an aggregation function using the probabilities received, for obtaining an aggregated distribution of prediction precision ([0050] the distribution prediction is aggregated as in [0051] in quantiles), and
applying at least one risk metric to the aggregated distribution of prediction precision, for obtaining at least one indicator of the performance of the prediction algorithm ([0050-51] a risk metric is used in the risk model prediction algorithm to determine/obtain performance)
Although Bradley teaches predictions and distributions which are determined as above, it does not explicitly state determining the distribution of the prediction precision of the predicted output for each data set, for obtaining determined distributions.
Adaptive, NPL for a system and method for adaptive sampling for stochastic risk-averse learning, teaches determining the distribution of the prediction precision of the predicted output for each data set, for obtaining determined distributions as on pgs. 19-20 where distributions are determined with precision.
It would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine determined distributions of Bradley with the determined distributions of Adaptive as they are both analogous art along with the claimed invention which all teach solutions to determine distributions in sampling data, and the combination would lead to an improved system which would optimize and improve robustness of distribution shifts in deep learning data and therefore improve accuracy of the model as taught on pg. 9 of Adaptive.
Examiner notes Bradley teaches a computer program product, storage medium, and data processing unit ([0116] processor/processing unit, [0117] product and medium)
Regarding Claim 2, Bradley teaches wherein a risk metric is a quantile metric and an indicator of the performance of the prediction algorithm is the value of a quantile of predetermined level ([0069] each quantile is predetermined as a threshold is set)
Regarding Claim 3, Bradley teaches wherein a risk metric is a conditional expectation and an indicator of the performance of the prediction algorithm is a value of the conditional expectation ([0051] the risk model uses risk metrics which are expectations for the claims which use the confidence level).
Regarding Claim 4, Bradly teaches wherein the prediction precision is calculated using an evaluation metric, the evaluation metric being an average of the absolute prediction error, a quantile metric, or an empirical moment of the distribution of the prediction precision ([0067] an absolute error is determine, along with a [0051] quantile metrics)
Regarding Claim 5, Bradley teaches wherein the prediction precision is calculated using a reference prediction algorithm ([0111-112] a model is used for reference)
Regarding Claim 6, Bradley teaches the performance indicator as in the claims above, but it does not explicitly teach a report.
Adaptive teaches reporting of data giving all of the information as on pgs. 7 and 8.
It would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine determined distributions of Bradley with the reporting of determined distributions of Adaptive as they are both analogous art along with the claimed invention which all teach solutions to determine distributions in sampling data, reporting of data is old and well-known, and the combination would lead to an improved system which would optimize and improve robustness of distribution shifts in deep learning data and therefore improve accuracy of the model as taught on pg. 9 of Adaptive.
Regarding Claim 7, Bradley teaches wherein the obtaining step is carried out by generating each data set from a reference data set according to a given probability law ([0050-51] information and data are obtained using a probability, which is a given probability law)
Regarding Claim 8, Bradley teaches wherein the obtaining step is carried out, for each data set, by generating, by means of a generative model of initial data and by selecting the initial data for forming the data set, according to a given probability law ([0050-51] information and data are obtained using a probability, which is a given probability law)
Regarding Claim 9, the combination of Bradley and Adaptive teaches the obtaining step includes the modification of the data sets in the system of the prediction algorithm models as in Claims 1 and 7 above.
Bradley also teaches introduction of parameters which are problems as in [0078-79], but it does not state by introducing imperfections.
Adaptive teaches introduction of adversal/imperfect sets of information as in pgs. 4 and 8 to increase the accuracy.
It would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine determined distributions of Bradley with the reporting of determined distributions of Adaptive as they are both analogous art along with the claimed invention which all teach solutions to determine distributions in sampling data, reporting of data is old and well-known, and the combination would lead to an improved system which would optimize and improve robustness of distribution shifts in deep learning data and therefore improve accuracy of the model as taught on pg. 9 of Adaptive.
Regarding Claim 10, the combination of Bradley and Adaptive teaches the obtaining step includes the modification of the data sets in the system of the prediction algorithm models as in Claims 1 and 7 above.
Bradley also teaches introduction of parameters which are problems as in [0078-79], but it does not state by introducing adverse perturbations aimed at manipulating the outputs of the prediction algorithm.
Adaptive teaches introduction of adversal/imperfect sets of information as in pgs. 4 and 8 to increase the accuracy.
It would be obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine determined distributions of Bradley with the reporting of determined distributions of Adaptive as they are both analogous art along with the claimed invention which all teach solutions to determine distributions in sampling data, reporting of data is old and well-known, and the combination would lead to an improved system which would optimize and improve robustness of distribution shifts in deep learning data and therefore improve accuracy of the model as taught on pg. 9 of Adaptive.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20200117998 A1
WALTERS; Austin et al.
SYSTEMS AND METHODS TO IDENTIFY NEURAL NETWORK BRITTLENESS BASED ON SAMPLE DATA AND SEED GENERATION
US 20240193165 A1
SPANNHAKE, II; Ernst Wilhelm et al.
SYSTEMS AND METHODS FOR AGGREGATING TIME-SERIES DATA STREAMS BASED ON POTENTIAL STATE CHARACTERISTICS FOLLOWING AGGREGATION
US 20220405860 A1
JUBAN; Romain Florian et al.
ARTIFICIAL INTELLIGENCE TRANSACTION RISK SCORING AND ANOMALY DETECTION
US 20220245492 A1
Bradley; William Francis et al.
CONSTRUCTING A STATISTICAL MODEL AND EVALUATING MODEL PERFORMANCE
US 20210224922 A1
JUBAN; Romain Florian et al.
SYSTEMS AND METHODS FOR ANTI-MONEY LAUNDERING ANALYSIS
US 20210081859 A1
Goel; Shubham et al.
ARCHITECTURE, ENGINEERING AND CONSTRUCTION (AEC) RISK ANALYSIS SYSTEM AND METHOD
US 20190066133 A1
COTTON; Peter
SYSTEM AND METHOD FOR PROVIDING DATA SCIENCE AS A SERVICE
US 20180349817 A1
Goel; Shubham et al.
ARCHITECTURE, ENGINEERING AND CONSTRUCTION (AEC) RISK ANALYSIS SYSTEM AND METHOD
US 20180060744 A1
Achin; Jeremy et al.
SYSTEMS FOR SECOND-ORDER PREDICTIVE DATA ANALYTICS, AND RELATED METHODS AND APPARATUS
US 20170351241 A1
Bowers; Matthew et al.
PREDICTIVE AND PRESCRIPTIVE ANALYTICS FOR SYSTEMS UNDER VARIABLE OPERATIONS
US 20170116530 A1
Modarresi; Kourosh
GENERATING PREDICTION MODELS IN ACCORDANCE WITH ANY SPECIFIC DATA SETS
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOSEPH M WAESCO whose telephone number is (571)272-9913. The examiner can normally be reached on 8 AM - 5 PM M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, BETH BOSWELL can be reached on (571) 272-6737. The fax phone number for the organization where this application or proceeding is assigned is 571-273-1348.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JOSEPH M WAESCO/Primary Examiner, Art Unit 3625B 1/6/2026