DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-20 are presented for examination.
Response to Amendment
Applicant’s amendment has obviated the claim objections. Therefore, those objections are withdrawn.
Claim Rejections - 35 USC § 112
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The term “improve” in claims 1, 8, and 15 is a relative term which renders the claims indefinite. The term “improve” is not defined by the claims, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. In particular, neither the claims nor the specification explicitly defines what the improvement is, how the improvement is measured, or what the baseline for determining whether there has been improvement is.
All claims dependent on a claim rejected hereunder are also rejected for being dependent on a rejected base claim.
Claim Rejections - 35 USC § 101
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The analysis of the claims will follow the 2019 Revised Patent Subject Matter Eligibility Guidance, 84 Fed. Reg. 50 (“2019 PEG”).
Claim 1
Step 1: The claim recites a method; therefore, it is directed to the statutory category of processes.
Step 2A Prong 1: The claim recites, inter alia:
[D]ynamically detecting drift in … one or more machine learning models generating a plurality of predictions and deployed in a computing system: This limitation could encompass the mental detection of drift in the models by observing their outputs.
[D]etecting non-stationary data traffic in a real-time encrypted and encapsulated data stream: This limitation could encompass mentally detecting non-stationary data traffic in the data stream by visual observation. Note that the claim does not define the encryption, so this could encompass encryption schemes that are comprehensible to a human user.
[C]ollecting a plurality of metrics of the one or more machine learning models based on the drift: This limitation could encompass mentally gathering data about the models.
[S]ampling the real-time encrypted data stream based on the collected plurality of metrics to create retraining data, wherein the retraining data are controlled by a minimum required data sample size and by a budget parameter representing a maximum number of trained models and scoring endpoints in use: This limitation could encompass mentally sampling the data stream to create a dataset under the claimed constraints.
[D]etermining the minimum required data sample size by extrapolating a learning curve based on a cross-validation performance of the one or more machine learning models: This limitation could encompass mentally determining the sample size by mentally extrapolating the learning curve based on the recited data.
[C]omputing information metrics from the plurality of metrics based on how informative the sampled real-time encrypted data stream is to improve model performance, wherein the information metrics include a distribution distance: This limitation could encompass mentally computing the information metrics including a distribution distance. Alternatively, this recites a mathematical concept. Note that the “improve model performance” language is intended-result language that does not alter the analysis.
[A]diusting a sampling rate for collecting the retraining data based on the distribution distance from the retraining data and the original training dataset: This limitation could encompass mentally adjusting the rate at which the (mental) sampling takes place based on a distribution distance.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The claim further recites that the method is “computer-implemented” and further recites “training one or more machine learning models with an original training dataset” and “training one or more additional machine learning models based on the created retraining data”. However, the first of these additional elements is a mere instruction to apply the judicial exception using a generic computer programmed with a generic class of computer algorithm, MPEP § 2106.05(f), and the second and third are mere restrictions of the judicial exception to the field of use or technological environment of training machine learning models. MPEP § 2106.05(h).
Step 2B: The claim does not contain significantly more than the judicial exception. The analysis at this step mirrors that of step 2A, prong 2. As an ordered whole, the claim is directed to a mentally performable process of detecting drift in a machine learning model and sampling data therefrom. Nothing in the claim provides significantly more than this. As such, the claim is not patent eligible.
Claim 2
Step 1: A process, as above.
Step 2A Prong 1: The claim recites the same judicial exceptions as in claim 1.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The claim further recites “collecting from the group consisting of model predictions, target data, sampled data, and one or more model parameters of the one or more machine learning models prior to detecting the drift.” This limitation recites the insignificant extra-solution activity of mere data gathering and output. MPEP § 2106.05(g).
Step 2B: The claim does not contain significantly more than the judicial exception. The claim further recites “collecting from the group consisting of model predictions, target data, sampled data, and one or more model parameters of the one or more machine learning models prior to detecting the drift.” This limitation recites the well-understood, routine, and conventional activity of receiving or transmitting data over a network. MPEP § 2106.05(d)(II); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network).
Claim 3
Step 1: A process, as above.
Step 2A Prong 1: The claim recites, inter alia, “updating an ensemble of training data based on collecting the plurality of metrics of the one or more machine learning models”. This limitation could encompass mentally updating the training dataset based on the metrics collected.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The claim further recites that the “ensemble of training data is used to perform an operation selected from the group consisting of training the one or more additional machine learning models or retraining the one or more machine learning models.” This limitation amounts to a mere restriction of the judicial exception to the field of use or technological environment of training a machine learning model. MPEP § 2106.05(h).
Step 2B: The claim does not contain significantly more than the judicial exception. The claim further recites that the “ensemble of training data is used to perform an operation selected from the group consisting of training the one or more additional machine learning models or retraining the one or more machine learning models.” This limitation amounts to a mere restriction of the judicial exception to the field of use or technological environment of training a machine learning model. MPEP § 2106.05(h).
Claim 4
Step 1: A process, as above.
Step 2A Prong 1: The claim recites “adjusting collection of the plurality of metrics and data of the one or more machine learning models based on dynamically detecting the drift.” This limitation could encompass the mental adjustment of the rate of (mental) collection of the data based on detecting the drift.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. See claim 1 analysis.
Step 2B: The claim does not contain significantly more than the judicial exception. See claim 1 analysis.
Claim 5
Step 1: A process, as above.
Step 2A Prong 1: The claim recites “detecting the drift exceeds a drift threshold, wherein the drift is selected from the group consisting of a data drift and a concept drift.” This limitation could encompass mentally determining that the drift exceeds the threshold.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. See claim 1 analysis.
Step 2B: The claim does not contain significantly more than the judicial exception. See claim 1 analysis.
Claim 6
Step 1: A process, as above.
Step 2A Prong 1: The claim recites “responsive to deploying the one or more additional machine learning models, tracking performance of the one or more additional machine learning models”. This limitation could encompass mentally tracking the performance of the models by visually inspecting their outputs.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. See claim 1 analysis.
Step 2B: The claim does not contain significantly more than the judicial exception. See claim 1 analysis.
Claim 7
Step 1: A process, as above.
Step 2A Prong 1: The claim recites “terminating use of the one or more machine models based on training the one or more additional machine learning models.” This limitation could encompass mentally deciding not to use a particular model anymore after it has been trained in a certain way.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. See claim 1 analysis.
Step 2B: The claim does not contain significantly more than the judicial exception. See claim 1 analysis.
Claims 8-14
Step 1: The claims recite a system comprising one or more computers with executable instructions; therefore, the claims are directed to the statutory category of machines.
Step 2A Prong 1: The claims recite the same judicial exceptions as in claims 1-7, respectively.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The analysis at this step mirrors that of claims 1-7, respectively, except insofar as these claims are directed to a “computer system in a computing environment, comprising: a processor set; one or more computer-readable storage media; and program instructions stored on the one or more computer-readable storage media to cause the processor set to perform [the] operations”. However, this amounts to a mere instruction to apply the judicial exception using a generic computer. MPEP § 2106.05(f).
Step 2B: The claim does not contain significantly more than the judicial exception. The analysis at this step mirrors that of claims 1-7, respectively, except insofar as these claims are directed to a “computer system in a computing environment, comprising: a processor set; one or more computer-readable storage media; and program instructions stored on the one or more computer-readable storage media to cause the processor set to perform [the] operations”. However, this amounts to a mere instruction to apply the judicial exception using a generic computer. MPEP § 2106.05(f).
Claims 15-20
Step 1: The claims are directed to a computer program product comprising one or more computer readable storage media. Paragraph 95 of the specification defines computer readable storage media as excluding transitory signals per se. Therefore, the claims are directed to the statutory category of articles of manufacture.
Step 2A Prong 1: The claims recite the same judicial exceptions as in claims 1-5 and 6-7, respectively.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The analysis at this step mirrors that of claims 1-5 and 6-7, respectively, except insofar as these claims are directed to a “computer program product in a computing environment, the computer program product comprising: one or more computer-readable storage media, and program instructions stored on the one or more computer-readable storage media to perform [the] operations”. However, this amounts to a mere instruction to apply the judicial exception using a generic computer. MPEP § 2106.05(f).
Step 2B: The claim does not contain significantly more than the judicial exception. The analysis at this step mirrors that of claims 1-5 and 6-7, respectively, except insofar as these claims are directed to a “computer program product in a computing environment, the computer program product comprising: one or more computer-readable storage media, and program instructions stored on the one or more computer-readable storage media to perform [the] operations”. However, this amounts to a mere instruction to apply the judicial exception using a generic computer. MPEP § 2106.05(f).
Response to Arguments
Applicant's arguments filed March 19, 2026 (“Remarks”) have been fully considered but they are not persuasive.
Applicant first argues that the claims as amended are eligible under 35 USC § 101 because they are allegedly directed to an improvement in technology by preventing inconsistent or decreased application or service-level functionality or complete machine learning model failure. Remarks at 10-13. However, none of these purported benefits is reflected in the claims themselves. Moreover, Applicant identifies no additional elements, beyond those constituting the judicial exception itself, that would allegedly provide that improvement to technology. As noted in the rejection, the claim as a whole is directed to a mentally performable process of detecting drift in a machine learning model and sampling data therefrom, and the only additional elements are the recitations of generic computer implementation and model training, neither of which amounts to significantly more than the judicial exception for the reasons given in the rejection.
Applicant further alleges that the use of the word “improve” is definite under 35 USC § 112(b) on the ground that the specification allegedly renders the claim definite. Remarks at 13-15. However, the cited portions of the specification do not answer the key question raised in the rejection: over what baseline is the model performance being improved? In the absence of a recitation of the baseline, an ordinary artisan reading the claims has no basis for comparison to determine whether an “improvement” has taken place. Nor, for that matter, do the claims even specify what metric is used to measure the “improvement” or how one would determine whether there has been an improvement. Is the improvement an improvement in accuracy? Precision? Recall? F1 score? ROC-AUC? Processor and/or memory usage? Training time? (One must keep in mind that an improvement in some of these metrics may cause a decrease in others.) And even if the claim did specify what metric of “improvement” is intended, the fundamental question remains: improved over what? One cannot train a single model, see that the model provides 95% accuracy on a validation dataset, and proclaim that that represents an “improvement”, because if there is no baseline model, there is no basis for comparison to determine whether an improvement has taken place. The rejection is maintained and will continue to be maintained until Applicant either specifies the baseline and metric for improvement (to the extent supported by the specification) or Applicant deletes the “improve” language.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RYAN C VAUGHN whose telephone number is (571)272-4849. The examiner can normally be reached M-R 7:00a-5:00p ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kamran Afshar, can be reached at (571) 272-7796. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RYAN C VAUGHN/Primary Examiner, Art Unit 2125
Prosecution Insights