Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment/Arguments
1. Applicant’s arguments filed on February 27, 2026 regarding the rejection under 35 U.S.C. 101 have been fully considered but are not persuasive. With respect to Applicant’s arguments on pages 12-15 under section 1 (“Claims 1, 11, and 20 Recite Limitations That Cannot Practically Be Performed In the Human Mind and Do Not Recite A Mental Process”), the arguments have been consider but are not persuasive. Applicant’s arguments primarily focus on whether the claimed operations allegedly cannot be performed mentally due to the volume of data, floating-point arithmetic, optimization algorithms, and computational scale involved. However, the relevant inquiry under Step 2A Prong One is not whether computerized implementation may be advantageous for speed, scale, or efficiency, but whether the claims recite acts of observation, evaluation, comparison, judgement, and mathematical analysis. Further, the Office Action additionally identified the claims as reciting mathematical concepts under Step 2A Prong One. MPEP 2106.04(a)(2)(III)(C) explains that claims may still recite mental processes even when performed on a computer, within a computer environment, or using a computer as a tool to perform evaluative analysis.
Applicant argues that fitting multiple versions of a time-series model allegedly cannot be performed in the human mind because the operations may involve optimization algorithms, iterative numerical solvers, floating-point arithmetic, and large datasets. However, the Office Action did not rely merely on the physical act of a computer executing code. Rather, the rejection identified the underlying claimed operations as evaluative and comparative decision-making, including applying parameter combinations, evaluating prediction results, comparing model performance, minimizing prediction error, and selecting preferred model configurations based on comparative results., A person could review forecasting outputs, compare prediction performance between model versions, evaluate whether one model performs better than another, and select or exclude parameter combinations based on those comparative evaluations, including with the assistance of basic computational tools such as spreadsheets or calculators. Such operations constitute mental evaluative processes under Step 2A Prong One, even if a computer may perform them more efficiently across larger datasets.
Applicant additionally explains that generating correlogram data involves apply ACF/PACF analysis, calculating means, covariances, variances, correlations, subtraction, multiplication, summation, and generating correlation coefficients across lag values. These arguments are likewise not persuasive because the claims recite acts of reviewing and evaluating the resulting correlogram values, comparing values to thresholds, selecting candidate parameter values, and excluding candidate parameter values based on comparative analysis. Such operations constitute acts of observation, evaluation, comparison, and judgement that can practically be performed mentally or with the assistance of basic computational tools such as spreadsheets or calculators. Further, Applicant’s own explanation confirms that the claims additionally recite mathematical calculations and statistical relationships, which fall within the mathematical concepts grouping of abstract ideas under Step 2A Prong One. MPEP 2106.04(a)(2(I) expressly defines mathematical concepts as including mathematical relationships and mathematical calculations.
Applicant’s reliance on Synopsys is also unpersuasive. While Synopsys explained that the particular claims at issue that were interpreted as encompassing pure mental steps because they do not require computer implementation, the present Office Action does not rely solely on the premise that the claims are performed entirely in the human mind without computational assistance. Rather, the Office Action identified evaluative and comparative operations that can practically be performed mentally, with pen and paper, or with the assistance of basic computational tools such as spreadsheets or calculators. Here, the claims recite operations including reviewing correlogram values, comparing values to thresholds, determining relative distances from thresholds, selecting candidate parameter values, excluding candidate value parameter values, evaluating forecasting results, and selecting preferred model configurations based on comparative statistical relationships. Such evaluative and comparative operations fall within the mental process grouping of abstract ideas.
Accordingly, Applicant’s arguments do not show error in the Step 2A Prong One analysis.
With respect to Applicant’s arguments on pages 15-19 under section 2 (“Claims 1, 11, and 20, When Considered As A Whole, Integrate Any Judicial Exception Into A Practical Application”), the arguments have been considered but are not persuasive. Applicant argues that the claims allegedly improve computer functionality, computer performance, data storage, data structures, and machine-learning training by reducing parameter-search space complexity using correlogram-derived threshold analysis. However, the alleged improvements identified by Applicant arise from reducing or refining the underlying mathematical and evaluative analysis itself, rather than from any technological improvement to computer functionality or another technology. The claims do not recite an improvement to processor architecture, memory architecture, network functionality, database technology, machine-learning hardware, or other computer technology. Rather, the claims recite mathematical/statistical analysis and evaluative parameter-selection logic applied to historical time-series data using generic computer components.
Applicant argues that the claims improve computational performance by reducing iterative training cycles and narrowing parameter search space. However, reducing the number of candidate parameter evaluations, reducing the number of model-fitting iterations, or reducing the number of mathematical comparisons performed merely reflects optimization of the underlying abstract mathematical analysis itself. Improving the efficiency of performing an abstract idea does not integrate the abstract idea into a practical application. Here, the alleged reduction in processor utilization, training time, and memory consumption flows directly from performing fewer parameter evaluations and fewer statistical calculations, rather than from any improvement to computer technology itself.
Applicant additionally argues that the claims improve data storage and data structures because the correlogram-derived threshold analysis allegedly reduces storage of unnecessary intermediate models and produces a more targeted parameter search structure. These arguments are likewise not persuasive because the claims do not recite a specific improvement to how data is physically stored, retrieved, compressed, indexed, organized, or managed by a computer system. Rather, Applicant’s alleged improvements merely reflect storing fewer intermediate results after narrowing the underlying mathematical search space. Similarly, the alleged “improved data structure” is merely a refined set of candidate parameters derived from statistical relationships with the data itself. Limiting or narrowing abstract analytical results does not constitute an improvement to computer data structures or storage technology.
Applicant’s reliance on Ex parte Desjardins is also unpersuasive. In Desjardins, the claims were determined to reflect a specific improvement to how machine-learning models operated and retained previously learned tasks, and the improvement was reflected in the claim language itself. Here, by contrast, the claims are directed to generating correlogram data, comparing correlogram values to thresholds, evaluating distances from thresholds, selecting candidate parameter values, excluding candidate parameter values, fitting multiple model versions, and evaluating forecasting performance. These operations constitute mathematical calculations, statistical analysis, and evaluative decision-making applied to historical data. The alleged improvements identified by Applicant merely reflect improving the efficiency or accuracy of the underlying abstract mathematical analysis itself rather than improving computer functionality or another technology.
Applicant additionally argues that Desjardins broadly applies to software-based logical structures and computational processes. However, merely reciting software-based logic, statistical analysis, or computational efficiency does not integrate an abstract idea into a practical application absent a specific technological improvement reflected in the claims themselves. Here, the claims recite generic computer implementation of mathematical/statistical forecasting analysis and parameter-selection logic. The claims do not recite a specialized machine-learning architecture, specialized training mechanism, specialized hardware configuration, or other technological improvement comparable to the circumstances discussed in Desjardins, Enfish, or McRO.
Accordingly, Applicant’s arguments do not show error in the Step 2A Prong Two analysis because the claims merely use generic computer components as tools to perform mathematical/statistical analysis and evaluative parameter-selection operations more efficiently. Any alleged improvements to computational efficiency, storage usage, parameter selection, or forecasting accuracy arise from refinement of the underlying abstract idea itself and does not constitute integration into a practical application.
With respect to the Applicant’s arguments on pages 20-22 under section A (“The Examiner has mischaracterized several limitations as ‘mental processes’” presented as part of the Applicant’s Step 2B analysis; the arguments have been considered but are not persuasive. Applicant’s arguments again primarily rely on the position that the claimed operations allegedly cannot practically be performed mentally because they involve floating pointing arithmetic, optimization algorithms, matrix decompositions, large datasets, or machine-resident data structures. However, the relevant inquiry under Step 2B is not whether computerized implementation may be advantageous for speed, scale, precision, or efficiency, but whether the additional elements amount to significantly more than recited judicial exception. Here, the claims merely recite generic computer implementation of mathematical analysis and evaluative data processing operations. Further, the Office Action additionally identified the claims as reciting mathematical concepts and mental processes under Step 2A Prong One. MPEP 2106.04(a)(2)(III)(C) expressly explains that claims may still recite mental processes even when performed on a computer, within a computer environment, or using a computer as a tool to perform evaluative analysis. Further, mathematical calculations and mathematical relationships independently constitute abstract ideas under the mathematical concepts grouping.
Applicant first argues that “applying a sample set of historical time-series data to a conversion function to generate a set of correlogram data comprising a plurality of values” allegedly cannot practically be performed mentally because the operation may involve ACF/PACF calculations, means, covariances, variances, correlations, subtraction, multiplication, summation, floating-point arithmetic, and repeated operations across lag values. These arguments are not persuasive. The claims recite mathematical/statistical analysis involving generation and evaluation of correlation values across lag values. Such operations constitute mathematical calculations and mathematical relationships under Step 2A Prong One. Further, the resulting correlogram values may be reviewed, evaluated, compared to thresholds, and used to select or exclude candidate parameters through evaluative and comparative reasoning that can practically be performed mentally or with the assistance of basic computational tools such as spreadsheets or calculators. The fact that a computer may perform these calculations more efficiently or across larger datasets does not remove the recited mathematical concepts and evaluative operations from the abstract-idea analysis. Merely performing such calculations using generic computer implementation does not amount to significantly more than the recited judicial exception under Step 2B.
Applicant next argues that “fitting multiple versions of the first time-series model to the training data set” allegedly cannot practically be performed mentally because the operation may involve fitting multiple ARMIA variants, optimization algorithms, matrix decompositions, iterative numerical solvers, and repeated parameter combinations across large datasets. However, the Office Action did not rely merely on the physical act of numerical computation performed by a processor. Rather, the rejection identified the underlying claimed operations as evaluative and comparative analysis including applying different parameter combinations, evaluating prediction results, comparing model performance, minimizing prediction error, and selecting preferred model configurations based on comparative outcomes. A person can review forecasting outputs, compare predictive performance between model variants, evaluation whether one configuration performs better than another, and select or exclude parameter combinations based on those evaluations, including with the assistance of spreadsheets, calculators, or other basic computational tools. Such evaluative operations constitute mental process under Step 2A Prong One, and generic computer implementation of those operations do not amount to significantly more under Step 2B.
Applicant additionally argues that “evaluation performances of multiple versions based on predictions generated by the multiple versions from the test data set” allegedly cannot practically be performed mentally because the operation may involve prediction generation, application of coefficient matrices, floating-point arithmetic, and performance metrics such MSE and AIC/BIC calculations. These arguments are likewise not persuasive because the claims recite evaluating forecasting results, comparing prediction performance metrics, determining relative model performance, and selecting preferred model versions based on comparative statistical relationships. Such operations constitute acts of evaluation, comparison, judgement, and mathematical analysis that can practically be performed mentally or with the assistance of spreadsheets, calculators, or similar computational tools. Further, Applicant’s own explanation confirms that the limitation recites mathematical calculations and statistical relationships, which independently falls within the mathematical concepts grouping of abstract ideas. Implementing such analysis using generic computer components does not amount to significantly more than the recited judicial exception.
Applicant further argues that “dividing a first set of historical time-series data associated with the first system component into a training data set and a test data set” allegedly lacks a meaningful human analog because the data is machine-generated, machine-resident, and stored within machine data structures. These arguments are not persuasive. Dividing data into categories, subsets, or groups for later evaluation or comparison constitutes a fundamental data organization and information-analysis activity that can practically be performed mentally or with the assistance of basic computational tools. The fact that the claimed data may originate from monitored computer systems or may be stored electronically does not transform the recited partitioning operation into a technological improvement or amount to significantly more than the judicial exception. Further, merely operating on data stored within a computer environment does not negate that the claims recite evaluative and organizational operations falling within the mental process grouping.
With respect to Applicant’s arguments on pages 23-27 under section B (“The Additional Elements, Individually and in Combination, Amount to Significantly More Than Any Judicial Exception”) presented as part of the Applicant’s Step 2B analysis, the arguments have been considered but are not persuasive. Applicant argues that the claims allegedly improve the technology of time-series model training and forecasting, recite a non-conventional combination of operations, and add meaningful limitations beyond generally linking a judicial exception to a technological environment. However, the alleged improvements identified by Applicant arise from refining and narrowing the underlying mathematical/statistical analysis itself, rather than from any improvement to computer technology, processor functionality, storage architecture, networking functionality, database technology, machine-learning hardware, or another technological field. The claims merely use generic computer components as tools to perform mathematical/statistical analysis, evaluative parameter-selection logic, and forecasting analysis more efficiently.
Applicant first argues that the claims allegedly improve the technology of time-series model training and forecasting by generating a targeted search space using correlogram-derived threshold analysis and threshold-distance relationships. Applicant further argues that conventional grid search techniques allegedly waste computational resources by evaluating parameter combinations unrelated to the temporal dependency structure of the underlying data. These arguments are not persuasive. The alleged improvements identified by Applicant merely reflect refining the underlying abstract mathematical/statistical analysis itself by reducing the number of candidates parameter evaluations and narrowing the parameter search space based on statistical relationships identified within the data. Improving the efficiency of performing an abstract mathematical analysis does not constitute a technological improvement to computer functionality or another technology. Here, the claims recite generating correlograms values, comparing correlogram values to thresholds, evaluating distances from thresholds, selecting candidate parameter values, excluding candidate parameter values, fitting model versions, and evaluating forecasting performance. Such operations constitute mathematical calculations, statistical analysis, evaluative comparison, and parameter-selection logic. The alleged reduction in processor utilization, memory usage, training cycles, or search-space size flows directly from performing fewer abstract analytical operations rather than from any improvement to computer technology itself.
Applicant additionally argues that the claims recite a “specific structured process” rather than merely instructions to apply an abstract idea on a computer. However, the claimed “structured process” itself merely combines abstract mathematical and evaluative operations including generating correlogram data, evaluating threshold relationships, comparing distance from thresholds, selecting parameters, evaluating forecasting performance, and selecting preferred model versions based on comparative outcomes. The fact that the claims recite a sequence itself is directed to mathematical/statistical analysis and evaluative decision-making. Further, the claims merely use generic computer implementation as a tool to performed the recited analysis and forecasting operations.
Applicant next argues that the Examiner allegedly failed to establish that the claimed operations as well-understood, routine, or conventional under Berkheimer. These arguments are likewise not persuasive. Berkheimer does not stand for the proposition that any allegedly non-conventional abstract idea automatically amounts to significantly more under Step 2B. Rather, the relevant inquiry remains whether the additional elements amount to an inventive concept beyond the judicial exception itself. Even assuming arguendo that Applicant’s particular arrangement of correlogram analysis, threshold comparison, threshold-distance evaluation, parameter filtering, model fitting, forecasting evaluation, and model selection was not previously widespread, the allegedly non-conventionality resides in the abstract mathematical/statistical analysis itself rather than in an improvement to computer technology. The claims do not recite a specialized processor architecture, specialized memory structure, specialized machine-learning architecture, specialized networking configuration, or other technological improvement comparable to the circumstances discussed in BASCOM, Enfish, CellzDirect, or Amdocs.
Applicant further argues that the Examiner improperly analyzed the claim elements individually rather than as an ordered combination. However, the Office Action considered the claims both individually and as an ordered combination. When considered as a whole, the claims still merely recite a sequence of mathematical/statistical calculations and evaluative operations directed to generating and analyzing correlogram values, selecting candidate parameters based on threshold relationships and threshold-distance comparisons, fitting model versions, evaluating forecasting performance, and selecting preferred model configurations. The ordered combination merely organizes and refines the underlying abstract analysis itself and does not transform the claims into a technological improvement or inventive concept under Step 2B.
Applicant additionally argues that the claims add meaningful limitations beyond generally linking a judicial exception to a technological environment because the claims allegedly define a specific machine-implemented forecasting pipeline operating on machine-collected time-series data. These arguments are not persuasive. Merely performing mathematical/statistical analysis on machine-generated or machine-stored data within a computer environment does not amount to significantly more than the recited judicial exception. The claims recite generic computer implementation of abstract analytical operations involving data collection, statistical evaluation, threshold comparison, parameter filtering, model evaluation, and forecasting analysis. The fact that the claims operate on electronically stored data or produce a stored forecasting model does not transform the recited abstract analysis into a technological improvement.
Applicant’s reliance on Amdocs is also unpersuasive. In Amdocs, the claims were directed to a specific distributed network architecture that provided a technological solution to a technological problem arising in computer-network accounting systems. Here, by contrast, the alleged improvement resides in the underlying statistical forecasting analysis itself. The claims do not recite a non-generic technological architecture or unconventional computer functionality, but instead recite generic computer implementation of mathematical/statistical forecasting analysis and evaluative parameter-selection logic.
Accordingly, Applicant’s arguments on pages 23-27 do not show error in the Step 2B analysis because the claims – including newly added claims 21 and 22 – merely use generic computer components as tools to perform mathematical/statistical analysis, comparative evaluation, mental decision-making, and parameter-selection logic. In particular, claims 21 and 22 recite identifying correlogram values relative to a threshold, comparing respective distances from the threshold, selecting one value, and refraining from including another value based on the comparative threshold analysis. Such operations constitute mathematical calculations, evaluative comparison, mental decision-making, and parameter-selection logic that can practically be performed mentally or with the assistance of basic computational tools such as spreadsheets or calculators, and therefore do not amount to an inventive concept or significantly more than the recited judicial exception.
Accordingly, for at least the reasons set forth above, claims 1-22 are directed to judicial exceptions including mental processes and mathematical concepts under Step 2A Prong One, fail to integrate the judicial exception into a practical application under Step 2A Prong Two, and do not recite additional elements amounting to significantly more than the judicial exception under Step 2B. Therefore, the rejection of claims 1-22 under 35 U.S.C. 101 is maintained.
2. Applicant’s arguments filed on February 27, 2026 regarding the rejection under 35 U.S.C. 103 have been fully considered but are not persuasive. With respect to Applicant’s arguments on pages 27-31 traversing the rejection of claims 1-22, the arguments have been considered but are not persuasive. Applicant primarily argues the combination of Achin and Baldan allegedly fails to teach selecting candidate parameter values based on both (1) a relationship to a threshold and (2) comparative distances from the threshold. Applicant further argues that Baldan merely “observes” significant lag values in correlogram plots rather than actually using threshold-relative significance relationships for forecasting-model parameter selection. However, Applicant’s arguments improperly isolate individual disclosures from Baldan while disregarding the reference as a whole and the understanding of a POSITA interpreting a correlogram analysis under the broadest reasonable interpretation of the claims.
Applicant acknowledges that Baldan expressly states: “In the PACF we see that lags 1, 2, and 5 are significant.” Baldan further discloses correlogram plots including dashed confidence-bound lines representing statistical significance thresholds. The identified significant lang values therefore correspond to correlogram values that exceed dashed threshold lines. Importantly, Baldan’s forecasting methodology described in Section 5.2 expressly explains that the ACF and PACF plots are visually examined “looking for significant lags,” and further states “[t]he significant lags found will be used to configure forecasting methods.” Baldan additionally explains that the forecasting methods evaluate whether the identified lags are “reasonable” and “effective” for the forecasting problem. Thus, Baldan does not merely “observe” significant lag values in insolation, but instead identifies significant lag values from correlogram analysis and operationally uses those lag values to configure and evaluate forecasting models. Under the broadest reasonable interpretation, identifying threshold-exceeding lag values for use in forecasting-model configuration reasonably teaches selecting candidate parameter values according to their relationship to the threshold.
Applicant further argues that Baldan’s discussion of lag value 5 allegedly demonstrates that Baldan merely uses a fixed default lag cap rather than the threshold-relative lag selection. These arguments are not persuasive. Baldan expressly explains in Section 5.2 that the forecasting methodology involves visually examining ACF/PACF plots “looking for significant lags,” where “[t]he significant lags found will be used to configure the forecasting methods.” Baldan further explains that a default maximal lag value 5 is used only “[if] the plots do not show significant lags.” Thus, Baldan’s disclosure of lag value 5 does not replace the correlogram-based lag selection analysis, but instead serves as a fallback exploration strategy where significant lag relationships are not visually identified from the correlogram data. Further, Section 7.1 applies this methodology and explains that ‘most of the series’ autocorrelation can be covered using the first 5 lags,” while additionally explaining that ARIMA and ETS assess whether selected lag choices are ‘reasonable.’ Thus, Baldan does not merely apply an arbitrary fixed lag cap independent of the correlogram analysis, but instead uses correlogram-derived autocorrelation analysis to guide lag-value selection and forecasting-model configuration. Further, Baldan’s ARIMA results themselves vary among multiple lag structures, including [Table 3] ARIMA(4,1,4), ARIMA(1,1,2), ARIMA(5,1,2), and ARIMA(5,1,3), demonstrating that Baldan is not merely rigidly selecting a single fixed lag value independent of the earlier correlogram analysis.
Applicant additionally argues that the claims allegedly require a distinct filtering rule based on specifically on “distance from the threshold,” and that Baldan allegedly lacks any disclosure of using threshold distance as part of parameter selection. However, Applicant’s arguments improperly impose an unduly narrow interpretation of the claims. The claims broadly recite selecting candidate parameters values based on a first distance of a first value from the threshold and a second distance of a second value from the threshold. Baldan’s correlogram plots expressly disclose significant lag values exceeding a dashed confidence-bound threshold lines by differing magnitudes, where some values extend only slightly above the threshold while others extend substantially farther beyond the threshold. A POSITA would have understood that the differing exceedance magnitudes shown in the correlogram necessarily correspond to differing distances relative to the threshold lines and reflect differing degrees of autocorrelation significance.
Further, Applicant’s arguments improperly attempt to separate statistical significance from threshold-relative magnitude analysis. In correlogram analysis, statistical significance is determined relative to the confidence-bound thresholds shown by the dashed lines. A POSITA interpreting the correlogram would have understood that values exhibiting magnitudes of threshold exceedance represent differing strengths of autocorrelation and may therefore reasonably guide parameter-selection decisions for forecasting-model configuration. Under the broadest reasonable interpretation, evaluating correlogram values according to their significance relative to the dashed threshold lines reasonably teaches selecting candidate lag parameters according to their relationship to, and relative distance from, the threshold.
Applicant further argues that the Office Action improperly relied on inherency because Baldan allegedly could determine “significance” using alternative criteria. These arguments are likewise not persuasive because the rejection does not rely solely on inherency. Rather, the rejection relies on Baldan’s explicit disclosure of correlogram significance analysis together with the understanding of a POSITA interpreting correlogram plots and forecasting-model parameter selection. Obviousness does not require ipsissimis verbis disclosure of the exact claimed language. Here, Baldan expressly discloses threshold-based correlogram analysis, identified significant lag values, differing magnitudes of threshold exceedance, and use of those significant lag values to configure forecasting methods. A POSITA would have reasonably understood and found it obvious to use the threshold-relative significance relationships shown in the correlogram when selecting or excluding candidate lag parameters for forecasting-model configuration.
Although Applicant does not separately argue newly added claims 21 and 22, these claims likewise do not render the rejection improper. A POSITA would have understood that selecting or excluding lag parameters according to differing significance relationships relative to the correlogram thresholds constitutes a routine forecasting-model optimization technique directed to improving parameter selection efficiency and reducing unnecessary parameter exploration.
Accordingly, Applicant’s arguments do not show error in the rejection because Baldan teaches threshold-based correlogram significance analysis and forecasting-model configuration using identified significant lag values, while Achin teaches systematic exploration and evaluation of predictive model parameter spaces. The combination of Achin and Baldan therefore teaches or at least renders obvious the claimed threshold-relative candidate-parameter selection limitations. A POSITA would have been motivated to incorporate Baldan’s correlogram-derived lag-selection analysis into Achin’s predictive model exploration framework in order to improve parameter-search efficiency, reduce unnecessary exploration of unsuitable lag parameters, reduce computational overhead, and improving forecasting-model performance.
Applicant’s arguments are not persuasive for all the reasons discussed above. Baldan teaches threshold-based correlogram significance analysis and identification of significant lag relationships for forecasting-model configuration, which Achin teaches systematic exploration and evaluation of predictive model parameter spaces. The combination of Achin and Baldan therefore teaches or at least renders obvious the limitations of independent claims 1, 11, and 20. Claims 2-10 and 12-19 depend therefrom and are likewise unpatentable over the cited combination. Further, newly added claims 21 and 22 likewise do not render the rejection improper for at least the reasons discussed above. Accordingly, the rejection under 35 U.S.C. 103 is maintained.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-22 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
101 Subject Matter Eligibility Analysis
Step 1: Claims 1-22 are within the four statutory categories (a process, machine, manufacture or composition of matter).
Claims 1-10 and 2-22 are directed to storage mediums and processors which are machines.
Claims 11-19 are directed to a method consisting of a series of steps, meaning that it is directed to the statutory category of process.
Step 2A Prong One, Step 2A Prong Two, and Step 2B Analysis:
Step 2A Prong One asks if the claim recites a judicial exception (abstract idea, law of nature, or natural phenomenon). If the claim recites a judicial exception, analysis proceeds to Step 2A Prong Two, which asks if the claim recites additional elements that integrate the abstract idea into a practical application. If the claim does not integrate the judicial exception, analysis proceeds to Step 2B, which asks if the claim amounts to significantly more than the judicial exception. If the claim does not amount to significantly more than the judicial exception, the claim is not eligible subject matter under 35 U.S.C. 101.
None of the claims represent an improvement to technology.
Regarding claim 1, the following claim elements are abstract ideas:
dividing a first set of historical time-series data associated with the first system component into a training data set and a test data set (This is an abstract idea of a “mental process.” It merely involves taking a collection of past measurements and separating them into two groups according the chosen rule, such as using the earlier entries for “training” and the later entries for “testing.” A person could manually review a list of historical values, mark the first portion as “training,” and the remaining portion as “test,” using mental judgement or simple tools like pen and paper. Because this partitioning is nothing more than organizing data into subsets based on human-selected criteria, it can practically be performed in the human mind and therefore falls within the mental processing grouping.);
determining a search space of parameters for the first time-series model based on the first set of historical time-series data, at least by (This is an abstract idea of a “mental process.” It involves reviewing historical data and deciding which parameter values should be considered when building a predictive model. A person could mentally examine a list of past measurements, note potential lag values, trends, or seasonal patterns, and then choose a range of parameter settings to test – such as which lags, differencing levels, or smoothing factors might be appropriate. This selection of a “search space” is simply human judgement applied to data interpretation and model planning, which can be practically performed in the human mind or with pen and paper.):
applying a sample set of the historical time-series data to a conversion function to generate a set of correlogram data comprising a plurality of values (This is an abstract idea of a “mental process.” It amounts to performing mathematical statistical calculations – such as computing correlations at different lags – and writing down the results. A person could review a series of past measurements, manually apply the correlation formulas step-by-step (e.g., by calculating how strongly each lagged value relates to the current value), and record the resulting numbers in a table. This is simply mathematical analysis and interpretation that can be performed mentally or with pen and paper, and therefore falls within the mental process and mathematical concept grouping of abstract ideas.);
comparing the plurality of values to a threshold (This is an abstract idea of a “mental process.” It involves reviewing numerical values and judging whether each value is above, below, or equal to a reference line or threshold. A person could simply look at a list of correlation values, visually compare each number to a chosen cutoff (e.g., 0.2 or a confidence level), and decide which ones meet the condition. Such comparison and evaluation of numbers can be performed mentally or with pen and paper, and therefore falls within the mental process grouping of abstract ideas.);
selecting, as candidate parameter values, a set of one or more values based on (a) the set of one or more values being equal to, or greater than, the threshold, and (b) a distance of each of the one or more values to the threshold (This is an abstract idea of a “mental process.” It involves reviewing numerical values, determining which one meets a condition relative to a threshold, and then choosing those whose values lie closest to that threshold. A person could mentally examine a list of correlation values, cross out those below the cutoff, note which remaining values exceed the threshold by the smallest amount, and select those as “candidates.” This selection based on relative distance and comparison is a cognitive judgement that can be performed in the human mind or with simple tools such as pen and paper, and thus falls within the mental process grouping of abstract ideas.);
fitting multiple versions of the first time-series model to the training data set, wherein the multiple versions of the first time-series model comprise different combinations of the parameters in the search space, including different candidate parameter values obtained from the set of correlogram data (This is an abstract idea of a “mental process.” It involves applying different mathematical models or parameter combinations to the same data and judging how well each model fits. A person could mentally or manually test multiple parameter settings – such as different lag values, differencing orders, or smoothing factors – calculate predicted values for each setting, and note how closely the predictions match historical data. This evaluation and comparison of alternate model formulations is purely mathematical analysis that can be performed in the human mind or with pen and paper.);
evaluating performances of the multiple versions based on predictions generated by the multiple versions from the test data set; (This is an abstract idea of a “mental process.” It consists of comparing predicted values to actual historical values and judging which model performs better. A person could manually compute prediction errors – such as by subtracting each predicted value from the actual value – list the errors for each model, and mentally determine which set of errors is smallest. This act of reviewing predictions, calculating accuracy, and deciding which model fits best can be performed in the human mind or with pen and paper.) and
selecting a version of the first time-series model with a highest performance in the multiple versions for use in forecasting the first set of metrics (This is an abstract idea of a “mental process.” It involves reviewing the calculated performance scores of several models and choosing the ones that performs best. A person could manually compare accuracy values – such as error rates or deviations – for each model tested, mentally identify the model with the smallest error or highest score, and decide to use that model going forward. This type of comparative evaluation and decision-making can be performed in the human mind or with pen and paper.); and
The following claim elements are additional elements which, taken alone or in combination with the other elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
A non-transitory computer readable medium (This is a high-level recitation of generic computer components for performing the abstract idea. See MPEP 2106.05.)
one or more hardware processors (This is a high-level recitation of generic computer components for performing the abstract idea. See MPEP 2106.05.),
training a first time-series model to predict a first set of metrics for a first system component in the monitored computer system at least by (The step of “training” a model is merely an instruction to apply the abstract idea and does provide a meaningful limitation. See MPEP 2106.05(f).)
accessing historical time-series data comprising metrics collected from a monitored computer system (This represents a well-understood, routine, and conventional computer operation of retrieving data from memory or a monitored system, see MPEP 2106.05(d)(II)(iv).);
storing the selected version of the first time-series model for use in forecasting the first set of metrics (This step is merely a generic data operation that amounts to storing and retrieving information in memory, which has been recognized by the courts as well-understood, routine, and conventional computer activity.).
Regarding claim 2, the rejection of claim 1 is incorporated herein. Further, claim 2 recites the following additional elements, which taken alone or in combination with other elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
wherein the first time-series model comprises a plurality of parameters, including a first parameter of a first parameter type, the first parameter type corresponding to an autoregressive function applied to the sample of the historical time-series data (This is merely an instruction to apply the abstract idea using a particular model configuration. It simply identifies that the model uses an autoregressive-type parameter, which is a conventional modeling choice and does not impose any meaningful limitation or integrate the judicial exception into a practical application. See MPEP 2106.05(f).), and
wherein the multiple versions of the first time-series model comprise instances of the first timeseries model including different candidate parameter values for the first parameter of the first parameter type (This is merely an instruction to apply the abstract idea with different parameter settings. It simply directs using multiple instances of the same model, each configured with various candidate values for the autoregressive-type parameter. This does not add any meaningful limitation or integrate the judicial exception into a practical application.).
Regarding claim 3, the rejection of claim 2 is incorporated herein. Further, claim 3 recites the following abstract idea:
wherein the second parameter of the second parameter type corresponds to a differencing function applied to a first training data set to generate a modified set of stationary data (This is an abstract idea of a “mental process.” It involves generating a modified, “stationary” set of data by applying a differencing function to historical values. A person could manually list the values, subtract each prior value from the next, and produce the transformed series using simple arithmetic. Because this transformation – applying a differencing step and generating the resulting stationary data – can be performed in the human mind or with pen and paper, it falls within the mental process grouping of abstract ideas.).
The following claim elements are additional elements which, taken alone or in combination with the other elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
wherein the first time-series model comprises at least a second parameter of a second parameter type (This is merely an instruction to apply the abstract idea using a model that includes an additional parameter type. It simply directs that the model be configured with another conventional parameter and does not impose any meaningful limitation or integrate the judicial exception into a practical application.),
Regarding claim 4, the rejection of claim 1 is incorporated herein. Further, claim 4 recites the following abstract ideas:
applying an autocorrelation function (ACF) to a sample set of the historical time-series data (This is an abstract idea of a mathematical concept and a mental process. It involves performing mathematical analysis – computing correlation values across different lags of the data – which is a type of mathematical relationship and algorithm. A person could manually tabulate the time-series values, shift the series by successive lags, multiply corresponding values, average the products, and thereby calculate the ACF values using pen and paper. Because this step recites mathematical calculations and can be performed mentally or with basic manual computation, it falls within both the mathematical concept and mental process groupings of abstract ideas.);
based on a result of applying the ACF: selecting a first set of parameter values of a first parameter type as the candidate parameter values for training the multiple versions of the first time-series model (This is an abstract idea of a mental process. It involves reviewing ACF – output – i.e., examining correlation values at different lags – then choosing which lags should serve as parameter values. A person could mentally look at the computed ACF values, note which ones cross a significant line, and decide which lags to keep as candidate parameters. This type of analysis and rule-based selection – observing numerical values, comparing them to a threshold, and choosing parameters accordingly – can be performed mentally or with basic tools such as pen and paper or simple calculator.),
The following claim elements are additional elements which, taken alone or in combination with the other elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
training the first time-series model with respective candidate parameter values to generate the multiple versions of the first time-series model (The step of “training” a model is merely an instruction to apply the abstract idea and does provide a meaningful limitation. See MPEP 2106.05(f).).
Regarding claim 5, the rejection of claim 4 is incorporated herein. Further, claim 5 recites the following abstract ideas:
identifying a first set of correlation values in the correlogram data meeting or exceeding a particular confidence threshold, the first set of correlation values corresponding to a respective second set of first parameter values of the first parameter type, wherein the first set of correlation values comprises a plurality of correlation values (This is an abstract idea of a mental process and mathematical concept. It involves evaluating numerical correlation values – i.e., mathematical results produced by autocorrelation calculations – and determining which of those values listed in a table, compare each numerical value to the threshold line, and mark which ones exceed the cutoff. This activity consists of mathematical evaluation (comparing correlation magnitudes to a significant level) and cognitive judgement that can be performed in the human mind or with basic tools such as pencil and paper, graph paper, or a simple calculator.);
identifying a second set of correlation values exceeding the particular confidence threshold, the second set of correlation values corresponding to a respective third set of first parameter values of the first parameter type (This is an abstract idea of a mental process. It involves observing correlation values, determining which ones rise above the confidence threshold, and associating those values with corresponding parameter choices. A person could manually review a list or plot of correlation values, note which ones exceed the threshold line, and match those values to their respective lags using simple reasoning or basic tolls such as pencil and paper. This process – examining numerical values, comparing them to a cutoff, and identifying the related parameters – can be carried out in the human mind or with minimal manual aids.); and
based on determining the first set of correlation values exceeds the particular confidence threshold by an amount less than each of the second set of correlation values: selecting the second set of first parameter values as the candidate parameter values, (This is an abstract idea of a mental process. It involves comparing two groups of numerical correlation values, judging which group exceeds the threshold by a smaller or larger margin, and then choosing the corresponding parameter values based on the comparison. A person could mentally review the correlation values in each group, assess how far each value sits above the confidence line, determine which group shows the greater margin, and then manually select the associated parameters – all using simple observation and arithmetic. This type of value-comparison and rule-based selection can be performed in the human mind or with basic tools such as pencil and paper.) and
The following claim elements are additional elements which, taken alone or in combination with the other elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
omitting the third set of parameter values from among the candidate set of parameter values (This limitation is merely instructions to apply to the abstract idea by excluding certain parameter values after the comparison, without adding any meaningful limitation. See MPEP 2106.05(f).)
Regarding claim 6, the rejection of claim 5 is incorporated herein. Further, claim 6 recites the following abstract ideas:
presenting the candidate parameter values of the first parameter type as recommendations for applying to the first time-series model (This is an abstract idea of a mental process. It involves reviewing selected parameter values and conveying them as recommendations, which a person could mentally determine and communicate – e.g., by noting which values appear suitable and writing or stating them. This type of evaluation and presentation can be performed in the human mind or with basic tolls such as pen and paper.).
Regarding claim 7, the rejection of claim 1 is incorporated herein. Further, claim 7 recites the following abstract ideas:
identifying a plurality of system components associated with a particular workload of the monitored computer system, wherein the first system component is among the plurality of system components (This is abstract idea of a mental process. It involves examining information about the system and determining which components relate to a particular workload – something a person could do mentally by reviewing a list of components, noting their roles, and grouping those involved in the workload. Such identification and categorization can be performed in the human mind or with basic tools such as pen and paper, and therefore falls within the mental-process groupings of abstract ideas.),
The following claim elements are additional elements which, taken alone or in combination with the other elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
wherein the metrics collected from the monitored computer system include workload values for the plurality of system components (This is merely an instruction to apply the abstract idea to particular data – i.e., to treat workload values as the metrics being used – without adding any meaningful limitation.),
training a second a second time-series model to predict a second set of metrics for a second system component among the plurality of system components (The step of “training” a model is merely an instruction to apply the abstract idea and does provide a meaningful limitation. See MPEP 2106.05(f).).
Regarding claim 8, the rejection of claim 7 is incorporated herein. Further, claim 8 recites the following additional elements, which taken alone or in combination with other elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
wherein the second time-series model is of a same type as the first time-series model (This is mere instructions to apply the abstract idea by specifying that both models use the same model type, which does not add any meaningful limitation or integrate the abstract idea into a practical application.), and
wherein a second set of parameters associated with the second time-series model is different from a first set of parameters associated with the selected version of the first time-series model (This is merely an instruction to apply the abstract idea by specifying that two models use different parameters parameter sets, which is an insignificant extra-solution detail about how the abstract comparison is carried out.).
Regarding claim 9, the rejection of claim 7 is incorporated herein. Further, claim 9 recites the following additional elements, which taken alone or in combination with other elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
wherein the second time-series model is of a different type as the first time-series model (This is merely an instruction to apply the abstract idea by specifying that the second model uses a different model type, which does not add any meaningful limitation or change how the abstract analysis is performed.).
Regarding claim 10, the rejection of claim is incorporated herein. Further, claim 9 recites the following additional elements, which taken alone or in combination with other elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
responsive to receiving a request to forecast workload values for the particular workload (This is merely receiving a request, which constitutes well-understood, routine, and conventional computer activity involving basic data transmission. See MPEP 2106.05(d)(II)(i).)
accessing a first set of time-series data associated with the first system component (This is a well-understood, routine, and conventional data-retrieval operation involving simply obtaining existing data from memory or storage. See MPEP 2106.05(d)(II)(iv).);
accessing a second set of time-series data associated with the second system component (This is a well-understood, routine, and conventional data-retrieval operation involving simply obtaining existing data from memory or storage. See MPEP 2106.05(d)(II)(iv).);
applying the first set of time-series data to the selected version of the first time-series model to generate a first prediction for the first system component (This is merely an instruction to apply the abstract idea by using the selected model to make a prediction from input data. It reflects routine model execution and does not add a meaningful limitation.);
applying the second set of time-series data to the second time-series model to generate a second prediction for the second component (This is merely an instruction to apply the abstract idea by using the selected model to make a prediction from input data. It reflects routine model execution and does not add a meaningful limitation.); and
presenting the first prediction and the second prediction in response to the request to forecast the workload values for the particular workload (This is merely an instruction to apply the abstract idea by displaying or reporting the results of the prediction, which constitutes insignificant extra-solution activity and routine output presentation. It does not add any meaningful limitation or integrate the abstract idea into a practical application.).
Regarding claim 11, the rejection of claim 1 is incorporated herein. Claim 11 recites the method form of the same operations set forth in claim 1. Therefore, the same subject matter analysis utilized for claim 1, as described above, is equally applicable to claim 11.
Regarding claim 12, the rejection of claim 11 is incorporated herein. The claim recites similar limitations as corresponding to claim 2. Therefore, the same subject matter analysis was utilized for claim 2, as described above, is equally applicable to claim 12.
Therefore claim 12, is ineligible.
Regarding claim 13, the rejection of claim 12 is incorporated herein. The claim recites similar limitations as corresponding to claim 3. Therefore, the same subject matter analysis was utilized for claim 3, as described above, is equally applicable to claim 13.
Therefore claim 13, is ineligible.
Regarding claim 14, the rejection of claim 11 is incorporated herein. The claim recites similar limitations as corresponding to claim 4. Therefore, the same subject matter analysis was utilized for claim 4, as described above, is equally applicable to claim 14.
Therefore claim 14, is ineligible.
Regarding claim 15, the rejection of claim 14 is incorporated herein. The claim recites similar limitations as corresponding to claim 5. Therefore, the same subject matter analysis was utilized for claim 5, as described above, is equally applicable to claim 15.
Therefore claim 15, is ineligible.
Regarding claim 16, the rejection of claim 15 is incorporated herein. The claim recites similar limitations as corresponding to claim 6. Therefore, the same subject matter analysis was utilized for claim 6, as described above, is equally applicable to claim 16.
Therefore claim 16, is ineligible.
Regarding claim 17, the rejection of claim 11 is incorporated herein. The claim recites similar limitations as corresponding to claim 7. Therefore, the same subject matter analysis was utilized for claim 7, as described above, is equally applicable to claim 17.
Therefore claim 17, is ineligible.
Regarding claim 18, the rejection of claim 17 is incorporated herein. The claim recites similar limitations as corresponding to claim 8. Therefore, the same subject matter analysis was utilized for claim 8, as described above, is equally applicable to claim 18.
Therefore claim 18, is ineligible.
Regarding claim 19, the rejection of claim 17 is incorporated herein. The claim recites similar limitations as corresponding to claim 9. Therefore, the same subject matter analysis was utilized for claim 9, as described above, is equally applicable to claim 19.
Therefore claim 19, is ineligible.
Regarding claim 20, the following claim elements are abstract ideas:
dividing a first set of historical time-series data associated with the first system component into a training data set and a test data set (This is an abstract idea of a “mental process.” It merely involves taking a collection of past measurements and separating them into two groups according the chosen rule, such as using the earlier entries for “training” and the later entries for “testing.” A person could manually review a list of historical values, mark the first portion as “training,” and the remaining portion as “test,” using mental judgement or simple tools like pen and paper. Because this partitioning is nothing more than organizing data into subsets based on human-selected criteria, it can practically be performed in the human mind and therefore falls within the mental processing grouping.);
determining a search space of parameters for the first time-series model based on the first set of historical time-series data, at least by (This is an abstract idea of a “mental process.” It involves reviewing historical data and deciding which parameter values should be considered when building a predictive model. A person could mentally examine a list of past measurements, note potential lag values, trends, or seasonal patterns, and then choose a range of parameter settings to test – such as which lags, differencing levels, or smoothing factors might be appropriate. This selection of a “search space” is simply human judgement applied to data interpretation and model planning, which can be practically performed in the human mind or with pen and paper.):
applying a sample set of the historical time-series data to a conversion function to generate a set of correlogram data comprising a plurality of values (This is an abstract idea of a “mental process.” It amounts to performing mathematical statistical calculations – such as computing correlations at different lags – and writing down the results. A person could review a series of past measurements, manually apply the correlation formulas step-by-step (e.g., by calculating how strongly each lagged value relates to the current value), and record the resulting numbers in a table. This is simply mathematical analysis and interpretation that can be performed mentally or with pen and paper, and therefore falls within the mental process and mathematical concept grouping of abstract ideas.);
comparing the plurality of values to a threshold (This is an abstract idea of a “mental process.” It involves reviewing numerical values and judging whether each value is above, below, or equal to a reference line or threshold. A person could simply look at a list of correlation values, visually compare each number to a chosen cutoff (e.g., 0.2 or a confidence level), and decide which ones meet the condition. Such comparison and evaluation of numbers can be performed mentally or with pen and paper, and therefore falls within the mental process grouping of abstract ideas.);
selecting, as candidate parameter values, a set of one or more values based on (a) the set of one or more values being equal to, or greater than, the threshold, and (b) a distance of each of the one or more values to the threshold (This is an abstract idea of a “mental process.” It involves reviewing numerical values, determining which one meets a condition relative to a threshold, and then choosing those whose values lie closest to that threshold. A person could mentally examine a list of correlation values, cross out those below the cutoff, note which remaining values exceed the threshold by the smallest amount, and select those as “candidates.” This selection based on relative distance and comparison is a cognitive judgement that can be performed in the human mind or with simple tools such as pen and paper, and thus falls within the mental process grouping of abstract ideas.);
fitting multiple versions of the first time-series model to the training data set, wherein the multiple versions of the first time-series model comprise different combinations of the parameters in the search space, including different candidate parameter values obtained from the set of correlogram data (This is an abstract idea of a “mental process.” It involves applying different mathematical models or parameter combinations to the same data and judging how well each model fits. A person could mentally or manually test multiple parameter settings – such as different lag values, differencing orders, or smoothing factors – calculate predicted values for each setting, and note how closely the predictions match historical data. This evaluation and comparison of alternate model formulations is purely mathematical analysis that can be performed in the human mind or with pen and paper.);
evaluating performances of the multiple versions based on predictions generated by the multiple versions from the test data set; (This is an abstract idea of a “mental process.” It consists of comparing predicted values to actual historical values and judging which model performs better. A person could manually compute prediction errors – such as by subtracting each predicted value from the actual value – list the errors for each model, and mentally determine which set of errors is smallest. This act of reviewing predictions, calculating accuracy, and deciding which model fits best can be performed in the human mind or with pen and paper.) and
selecting a version of the first time-series model with a highest performance in the multiple versions for use in forecasting the first set of metrics (This is an abstract idea of a “mental process.” It involves reviewing the calculated performance scores of several models and choosing the ones that performs best. A person could manually compare accuracy values – such as error rates or deviations – for each model tested, mentally identify the model with the smallest error or highest score, and decide to use that model going forward. This type of comparative evaluation and decision-making can be performed in the human mind or with pen and paper.); and
The following claim elements are additional elements which, taken alone or in combination with the other elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
one or more processors (This is a high-level recitation of generic computer components for performing the abstract idea. See MPEP 2106.05.);
memory storing instructions (This is a high-level recitation of generic computer components for performing the abstract idea. See MPEP 2106.05.).
accessing historical time-series data comprising metrics collected from a monitored computer system (This represents a well-understood, routine, and conventional computer operation of retrieving data from memory or a monitored system, see MPEP 2106.05(d)(II)(iv).);
training a first time-series model to predict a first set of metrics for a first system component in the monitored computer system at least by (The step of “training” a model is merely an instruction to apply the abstract idea and does provide a meaningful limitation. See MPEP 2106.05(f).);
storing the selected version of the first time-series model for use in forecasting the first set of metrics (This step is merely a generic data operation that amounts to storing and retrieving information in memory, which has been recognized by the courts as well-understood, routine, and conventional computer activity.).
Regarding claim 21, the rejection of claim 1 is incorporated herein. Further, claim 21 recites the following abstract ideas:
wherein selecting the candidate parameter values comprises: identifying a first value in the correlogram data that is equal to, or greater than, the threshold; identifying a second value in the correlogram data that is equal to, or greater than, the threshold (This is an abstract idea of a mental process. It involves visually reviewing correlogram data, where the threshold is represented by a dotted line, and observing which values meet or exceed the threshold based on their distance relative to the dotted line. A person could visually compare the correlogram values to the threshold line and identify the values satisfying the threshold condition.);
based on a first distance of the first value from the threshold and a second distance of the second value from the threshold: selecting the first value as a first candidate parameter value among the candidate parameter values (This is an abstract idea of a mental process. It involves visually comparing the distances of multiple correlogram values from a threshold line and selecting one value based on comparison. A person could observe that one value is closer to the dashed/dotted threshold line than another value and mentally decide to select that value as a candidate parameter value using observation and judgement.); and
The following claim elements are additional elements which, taken alone or in combination with the other elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
refraining from including the second value among the candidate parameter values (This limitation amounts to mere instructions to apply the abstract idea and insignificant extra-solution activity because it merely excludes data from further consideration after the mental evaluation and comparison has already been performed.).
Regarding claim 22, the rejection of claim 1 is incorporated herein. Further, claim 22 recites the following abstract ideas:
wherein selecting the candidate parameter values comprises: identifying a first value in the correlogram data that is equal to, or greater than, the threshold; identifying a second value in the correlogram data that is equal to, or greater than, the threshold (This is an abstract idea of a mental process. It involves visually reviewing correlogram data, where the threshold is represented by a dotted line, and observing which values meet or exceed the threshold based on their distance relative to the dotted line. A person could visually compare the correlogram values to the threshold line and identify the values satisfying the threshold condition.);
based on a first distance of the first value from the threshold and a second distance of the second value from the threshold: selecting the first value as a first candidate parameter value among the candidate parameter values (This is an abstract idea of a mental process. It involves visually comparing the distances of multiple correlogram values from a threshold line and selecting one value based on comparison. A person could observe that one value is closer to the dashed/dotted threshold line than another value and mentally decide to select that value as a candidate parameter value using observation and judgement.); and
The following claim elements are additional elements which, taken alone or in combination with the other elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
refraining from including the second value among the candidate parameter values, wherein the first distance is smaller than the second distance (This limitation amounts to mere instructions to apply the abstract idea and insignificant extra-solution activity because it merely excludes data from further consideration after the mental evaluation and comparison has already been performed.).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-22 are rejected under the 35 U.S.C. 103 as being unpatentable over Achin et al., (Pub. No.: US 20180046926 A1 (Filed: 2017)). in view of Baldan et al., (NPL: “A Forecasting Methodology for Workload Forecasting in Cloud Systems” (Published: February 2016)).
Regarding claim 1, Achin teaches the following limitations:
A non-transitory computer readable medium comprising instructions which, when executed by one or more hardware processors, causes performance of operations comprising (Achin, paragraph [0477] “some embodiments may be embodied as a computer readable medium (or multiple computer readable media) (e.g., a computer memory, one or more floppy discs, compact discs, optical discs, magnetic tapes, flash memories, circuit configurations in Field Programmable Gate Arrays or other semiconductor devices, or other tangible computer storage medium) encoded with one or more programs that, when executed on one or more computers or other processors, perform methods that implement the various embodiments discussed above…The terms “program” or “software” are used herein in a generic sense to refer to any type of computer code or set of computer-executable instructions that can be employed to program a computer or other processor”)
accessing historical time-series data comprising metrics collected from a monitored computer system (Achin, paragraph [0042] “ Performing the predictive modeling procedure may include performing the pre-processing task, including: (a) obtaining time-series data including one or more data sets, wherein each data set includes a plurality of observations, wherein each observation includes (1) an indication of a time associated with the observation and (2) respective values of one or more variables; (b) determining a time interval of the time-series data; (c) identifying one or more of the variables as targets, and identifying zero or more other variables as features;”)
training a first time-series model to predict a first set of metrics for a first system component in the monitored computer system at least by (Achin, paragraph [0015] “predictive modeling method including performing a predictive modeling procedure, including: (a) obtaining time-series data including one or more data sets, wherein each data set includes a plurality of observations, wherein each observation includes (1) an indication of a time associated with the observation and (2) respective values of one or more variables…(c) identifying one or more of the variables as targets, and identifying zero or more other variables as features; (d) determining a forecast range…associated with a prediction problem represented by the time-series data, wherein the forecast range indicates a duration of a period for which values of the targets are to be predicted” (e) generating training data from the time-series data, wherein the training data include a first subset of the observations of at least one of the data sets, wherein the first subset of the observations includes training-input and training-output collections of the observations, wherein the times associated with the observations in the training-input and training-output collections correspond, respectively, to a training-input time range and a training-output time range,”):
dividing a first set of historical time-series data associated with the first system component into a training data set and a test data set (Achin, paragraph [0015] “(e) generating training data from the time-series data, wherein the training data include a first subset of the observations of at least one of the data sets, wherein the first subset of the observations includes training-input and training-output collections of the observations, wherein the times associated with the observations in the training-input and training-output collections correspond, respectively, to a training-input time range and a training-output time range, wherein the skip range separates an end of the training-input time range from a beginning of the training-output time range, and wherein a duration of the training-output time range is at least as long as the forecast range; (f) generating testing data from the time-series data, wherein the testing data include a second subset of the observations of at least one of the data sets, wherein the second subset of the observations includes testing-input and testing-validation collections of the observation…(g) fitting a predictive model to the training data; and (h) testing the fitted model on the testing data.”);
determining a search space of parameters for the first time-series model based on the first set of historical time-series data, at least by (Achin, paragraph [0100] “ in some embodiments a predictive modeling system 100 includes a predictive modeling exploration engine 110…The exploration engine 110 may implement a search technique (or “modeling methodology”) for efficiently exploring the predictive modeling search space (e.g., potential combinations of pre-processing steps, modeling algorithms, and post-processing steps) to generate a predictive modeling solution suitable for a specified prediction problem. The search technique may include an initial evaluation of which predictive modeling techniques are likely to provide suitable solutions for the prediction problem. In some embodiments, the search technique includes an incremental evaluation of the search space (e.g., using increasing fractions of a dataset), and a consistent comparison of the suitability of different modeling solutions for the prediction problem (e.g., using consistent metrics). In some embodiments, the search technique adapts based on results of prior searches, which can improve the effectiveness of the search technique over time.”):
evaluating performances of the multiple versions based on predictions generated by the multiple versions from the test data set (Achin, paragraph [0349] “As with cross-sectional models (see, e.g., the description of step 350 of the method 300, above), the engine 110 may iterate through the dataset, training each model on a small fraction of the training window, evaluating its performance on that fraction, then deciding whether to continue testing the model on additional data based on its performance.” – describes evaluating the performance of a predictive model on validation windows (which correspond to test data). The reference teaches iteratively generating multiple versions of the model by training on different fractions of the training window and then “evaluating its performance” on each corresponding validation fraction. Each fraction produces distinct fitted versions of the model, and the performance evaluation is explicitly based on predictions made for the validation (test ) portion of the time series data. Achin further explains that validation windows can be defined in weeks, months, or years and increase across multiple fractions, providing the multiple versions required by the claim.) ; and
selecting a version of the first time-series model with a highest performance in the multiple versions for use in forecasting the first set of metrics (Achin, paragraph [0404] “the system 100 selects predictive models for blending based on the model-specific predictive values, and blends the selected models (e.g., at step 432 of the method 400)… the system 100 may select “complementary top models” for blending…The system 100 may classify a model as a “top” model if the accuracy of the model is greater than a threshold accuracy, if the model has one of the N highest accuracy values among the fitted models…the system 100 may use the above-described classification techniques to select two or more complementary top models for blending.” – teaches selecting one or more predictive models based on performance metrics, including selecting models that have “one of the N highest accuracy values among the fitted models.” Accuracy is a direct measure of performance. Identifying the “top” model(s) among many fitted models necessarily involves comparing multiple versions and choosing the version with the highest performance. This satisfies the requirement of selecting the highest-performing version of the time-series model for use in forecasting the metrics.); and
storing the selected version of the first time-series model for use in forecasting the first set of metrics (Achin, paragraph [0320] “ File Storage 830. This module provides an interface to and manages storage mechanisms for files. Types of data stored via this module include uploaded datasets, derived data, model computations, and predictions. This module may layer a file directory and naming convention on top of cloud storage. Additionally, when cloud workers access this module, they may also temporarily cache the stored files in their local storage.”).
However, Achin does not teach but, Achin in view of Baldan teaches the following limitations:
applying a sample set of the historical time-series data to a conversion function to generate a set of correlogram data comprising a plurality of values (Achin, paragraph [0026] “ the actions of the method further include down-sampling the training data prior to fitting the predictive model to the training data. In some embodiments, down-sampling the training data includes: removing, from the training data, all observations obtained from at least one of the data sets. In some embodiments, down-sampling the training data includes setting a down-sampled time interval of the training data to an integer multiple of the time-interval of the time series data;”) Baldan, page 937, section 7 “Fig. 5 depicts the complete workload system series for the Google dataset hourly CPU load…However, in the ACF and PACF plots this pattern is not reflected, which suggests that the cycles change in length. In the PACF we see that lags 1, 2, and 5 are significant. In the differenced version, we get significant lags up to 12 and 16, respectively, in the ACF and PACF plots…we use a lag value of 24 to evaluate daily patterns in the seasonal study…We also use a value of 5, as this is a maximal lag choice commonly used in time series forecasting (as already discussed in Section 5.2) and as most of the series’ auto-correlation can be covered using the first five lags.” – shows that Baldan applies the historical time-series data to the auto-correlation and partial auto-correlation functions (conversion functions) generating ACF and PACF correlograms that include multiple correlation values across different lag intervals, meeting the limitation of “generating a set of correlogram (Fig 5) data comprising a plurality of values.).;
comparing the plurality of values to a threshold (Baldan, section 7.1, Fig. 5 “In the PACF we see that lags 1, 2, and 5 are significant. In the differenced version, we get significant lags up to 12 and 16, respectively, in the ACF and PACF plots.” – teaches comparing a plurality of correlation values to a significance threshold. Baldan explains that “lags 1, 3, and 5 are significant,” which inherently requires comparing the correlation values at those lags to the correlogram’s statistical confidence bounds (the dashed horizontal threshold likes in Fig. 5). The plotted values exceeding the threshold are identified as significant. Thus, Baldan teaches “comparing the plurality of values to a threshold.);
selecting, as candidate parameter values, a set of one or more values based on (a) the set of one or more values being equal to, or greater than, the threshold, and (b) a distance of each of the one or more values to the threshold (Baldan, section 7.1, “Fig. 5 depicts the complete workload system series for the Google dataset hourly CPU load.2…In the PACF we see that lags 1, 2, and 5 are significant. In the differenced version, we get significant lags up to 12 and 16, respectively, in the ACF and PACF plots.” – teaches selecting candidate parameter values based on whether their correlation values exceed a statistical threshold. Baldan states that “lags 1, 3, and 5 are significant,” which occurs only when the correlation values at those lags are equal to or greater than the significant threshold shown by the dashed confidence bounds in the correlogram. Because identifying significance is the ACF/PACF plot is inherently based on the distance between each correlation value and threshold lines, Baldan also teaches selecting values based on the distance of each correlation value to the threshold (Fig. 5). Thus, Baldan teaches selecting candidate parameter values based on both (a) exceeding a threshold and (b) the distance to the threshold.);
fitting multiple versions of the first time-series model to the training data set, wherein the multiple versions of the first time-series model comprise different combinations of the parameters in the search space, including different candidate parameter values obtained from the set of correlogram data (Achin, paragraph [0101] “The exploration engine 110 may use the library 130 of modeling techniques to evaluate potential modeling solutions in the search space. In some embodiments, the modeling technique library 130 includes machine-executable templates encoding complete modeling techniques. A machine-executable template may include one or more predictive modeling algorithms…a machine-executable template further includes one or more pre-processing and/or post-processing steps suitable for use with the template's algorithm(s). The algorithm(s), pre-processing steps, and/or post-processing steps may be parameterized.” Paragraph [0102] “The exploration engine 110 may uses the computational resources of a distributed computing system to explore the search space or portions thereof… interfaces that facilitate the evaluation of predictive modeling solutions in accordance with the search plan” Baldan, section 7.1, Fig. 5 “we see that lags 1, 2, and 5 are significant. In the differenced version, we get significant lags up to 12 and 16, respectively, in the ACF and PACF plots.” – Achin teaches exploring a parameterized model search space and evaluation multiple model versions with different parameter values. Baldan provides the specific candidate parameter values (significant ACF/PACF lags) that populate the search space.);
Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, having a combination of Achin and Baldan before them, to incorporate the time-series parameter-selection techniques of Baldan into the predictive-model generation framework of Achin. One would have been motivated to make such a combination in order to achieve a predictable, superior result in forecasting complex workload time series.
Regarding claim 2, Achin in view of Baldan teaches all the elements of claim 1, therefore is rejected for the same reasons as those presented for claim 1. Achin in view of Baldan further teaches the limitation:
wherein the first time-series model comprises a plurality of parameters, including a first parameter of a first parameter type, the first parameter type corresponding to an autoregressive function applied to the sample of the historical time-series data (Baldan, section 5.1 “In this section, we describe the different forecasting algorithms employed in this framework. Since there is no single universally valid model and the time series to be analyzed have different properties, we use a wide range of forecasting models to obtain the fittest model according to their features.” Section 5.1.1 “ARIMA models are denoted as ARIMA(p, d, q), where p is the number of lagged values considered in the autoregressive part, q is the number of lagged values considered for the moving average part, and d is the number of differences considered.” – teaches that ARIMA time-series model includes parameters (p, d, q). where p is a parameter corresponding to an autoregressive function applied to prior time-series samples, thus meeting the limitation’s requirement of a first parameter of a first parameter type corresponding to autoregressive function.), and
wherein the multiple versions of the first time-series model comprise instances of the first timeseries model including different candidate parameter values for the first parameter of the first parameter type (Baldan, section 5.1.1 “ARIMA models are denoted as ARIMA(p, d, q), where p is the number of lagged values considered in the autoregressive part, q is the number of lagged values considered for the moving average part, and d is the number of differences considered. There is an established methodology to choose all possible parameters of an ARIMA model fully automatically.” Achin, paragraph [0206] “ exploration engine 110 may “blend” multiple models using different mathematical combinations to create new models (e.g., using stepwise selection of models to include in the blender).” Paragraph [0207] “blending may work better when a large variety of candidate models are available to blend...Predictive modeling system 100 may deliver a substantial head start by automatically producing a wide variety of models and maintaining metadata describing how the candidate models differ.” – Baldan teaches that the first time-series model (e.g., ARIMA) has parameters including an autoregressive parameter p, and that different parameter values are automatically selected; Achin teaches generating a wide variety of candidate models and multiple versions of models that differ in parameters.).
Regarding claim 3, Achin in view of Baldan teaches all the elements of claim 2, therefore is rejected for the same reasons as those presented for claim 2. Achin in view of Baldan further teaches the limitation:
wherein the first time-series model comprises at least a second parameter of a second parameter type, wherein the second parameter of the second parameter type corresponds to a differencing function applied to a first training data set to generate a modified set of stationary data (Baldan, section 5.1.1 “Combining an ARMA model and differencing, we obtain the non-seasonal ARIMA model [28]. It is a workhorse in time series analysis and modelling. ARIMA models are denoted as ARIMA(p, d, q), where p is the number of lagged values considered in the autoregressive part, q is the number of lagged values considered for the moving average part, and d is the number of differences considered.” – teaches that the ARIMA model includes the d parameter, representing how many differencing operations are applied to the training data (“d is the number of differences considered”). Differencing is the standard transformation used in ARIMA to remove trends and convert the raw series into stationary data. Therefore, Baldan teaches a second parameter type corresponding to a differencing function applied to the training data to generate a modified, stationary dataset.).
Regarding claim 4, Achin in view of Baldan teaches all the elements of claim 1, therefore is rejected for the same reasons as those presented for claim 1. Achin in view of Baldan further teaches the limitation:
wherein determining the search space of parameters for the first time-series model comprises: applying an autocorrelation function (ACF) to a sample set of the historical time-series data (Baldan, section 5.1.1 “ARIMA models are denoted as ARIMA(p, d, q), where p is the number of lagged values considered in the autoregressive part, q is the number of lagged values considered for the moving average part, and d is the number of differences considered. There is an established methodology to choose all possible parameters of an ARIMA model fully automatically. In R [29], a language and environment for statistical computing and graphics, this methodology, known as the Hyndman-Khandakar algorithm [5], is implemented in the auto.arima function from the forecast package [27].” – teaches determining the parameter search space for ARIMA by using Hyndman-Khandakar algorithm, which is the standard automatic model-selection method applied to a sample set of historical time-series data (“established methodology to choose all possible parameters…fully automatically”). The Hyndman-Khandakar procedure applies autocorrelation analysis, including autocorrelation function (ACF), to sample data in order to identify candidate values for the autoregressive and moving average parameters – thus teaching determine a parameter search space by applying ACF to historical data.);
based on a result of applying the ACF: selecting a first set of parameter values of a first parameter type as the candidate parameter values for training the multiple versions of the first time-series model (Baldan, section 5.1.1. “ARIMA models are denoted as ARIMA(p, d, q), where p is the number of lagged values considered in the autoregressive part…There is an established methodology to choose all possible parameters of an ARIMA model fully automatically… known as the Hyndman-Khandakar algorithm” Section 5.2, Fig. 3 “By plotting a series and its ACF, we can look for significant lags to model our series and extract other information from patterns like stationarity, seasonality and trends.” – teaches applying the ACF to the historical data to identify significant lag values, which corresponds to the autoregressive parameter p of the ARIMA model. These lags form the first set of candidate parameter values for training multiple versions of the time-series model.),
training the first time-series model with respective candidate parameter values to generate the multiple versions of the first time-series model (Baldan, section 6.2 “Furthermore, we also apply all forecasting methods with all parameter combinations defined in the parameter grids to train on all time series, to verify on the training set that the initially chosen parameters are reasonable choices. In this way, we avoid over-fitting without loosing data for an additional validation set. The models with the chosen parameter configurations are executed over the respective test data.”).
Regarding claim 5, Achin in view of Baldan teaches all the elements of claim 4, therefore is rejected for the same reasons as those presented for claim 4. Achin in view of Baldan further teaches the limitation:
wherein selecting the first set of parameter values of the first parameter type is based at least on: identifying a first set of correlation values in the correlogram data meeting or exceeding a particular confidence threshold, the first set of correlation values corresponding to a respective second set of first parameter values of the first parameter type, wherein the first set of correlation values comprises a plurality of correlation values (Baldan, section 7.1 “Fig. 5 depicts the complete workload system series for the Google dataset hourly CPU load…we see that lags 1, 2, and 5 are significant. In the differenced version, we get significant lags up to 12 and 16, respectively, in the ACF and PACF plots….Significant lags are only present up to lag 10.” – Fig. 5 contains an ACF correlogram where two dashed horizontal lines represent the particular confidence threshold used to evaluate autocorrelation. In the correlogram, multiple vertical bars extend above these dashed confidence bounds. Because these bars visibly exceed the confidence threshold and are identified in the quoted material as “significant,” they constitute a plurality of threshold-exceeding correlation values, which corresponds directly to the claimed first set of correlation values meeting or exceeding a threshold.)
identifying a second set of correlation values exceeding the particular confidence threshold, the second set of correlation values corresponding to a respective third set of first parameter values of the first parameter type (Baldan, section 7.1 “Fig. 5 depicts the complete workload system series for the Google dataset hourly CPU load…we see that lags 1, 2, and 5 are significant. In the differenced version, we get significant lags up to 12 and 16, respectively, in the ACF and PACF plots….Significant lags are only present up to lag 10.” - Figure 5 also shows the ACF correlogram for the “differenced” version of the same time-series data. In that correlogram, additional vertical bars extend beyond the same dashed confidence threshold lines. These additional threshold-exceeding values form a second set of correlation values distinct from the first. Each of these threshold-exceeding correlations correspond to a different lag index, and those lag indices correspond directly to a respective third set of first parameter-type-values (autoregressive lag parameters). Thus, the figure provides a second set of correlation values required by the claim, each mapping to its own lag-based parameter value.); and
based on determining the first set of correlation values exceeds the particular confidence threshold by an amount less than each of the second set of correlation values: selecting the second set of first parameter values as the candidate parameter values, and omitting the third set of parameter values from among the candidate set of parameter values (Baldan, section 7.1 “Fig. 5 depicts the complete workload system series for the Google dataset hourly CPU load…we see that lags 1, 2, and 5 are significant. In the differenced version, we get significant lags up to 12 and 16, respectively, in the ACF and PACF plots….Significant lags are only present up to lag 10.” – In Fig. 5, the significant autocorrelation bars rise above the dashed confidence bounds by different amounts. Some bars extend only slightly above the threshold, while others extend much higher above it. This visual distinction provides the basis for separating the significant correlation values into two groups according to how much each value exceeds a threshold. Under the claim language, the parameter values associated with the bars that rise above the threshold form the selected set, and the parameter values associated with the bars that rise only slightly above the threshold form the omitted set. Fig. 5 therefore supplies the differing degrees of threshold exceedance needed to perform the selection and omission steps recited in the limitation.).
Regarding claim 6, Achin in view of Baldan teaches all the elements of claim 5, therefore is rejected for the same reasons as those presented for claim 5. Achin in view of Baldan further teaches the limitation:
presenting the candidate parameter values of the first parameter type as recommendations for applying to the first time-series model (Achin, paragraph [0207] “Predictive modeling system 100 may deliver a substantial head start by automatically producing a wide variety of models and maintaining metadata describing how the candidate models differ.” paragraph [0208] “First, user interface 120 may provide an interactive model comparison, including several different alternative measures of candidate model fit and graphics such as dual lift charts, so that users can easily identify accurate and complementary models…modeling system 100 gives the user the option of choosing specific candidate models and blending techniques or automatically fitting some or all of the blending techniques in the modeling technique library using some or all of the candidate models.” paragraph [0209] “the user interface 120 presents the final results to the user. Based on this presentation, the user may refine the dataset (e.g., by returning to step 412), adjust the allocation of resources to executing modeling techniques (e.g., by returning to step 444), modify one or more of the modeling techniques to improve accuracy (e.g., by returning to step 430), alter the dataset (e.g., by returning to step 402), etc.” – Achin generates multiple candidate model configurations and makes their differing parameter settings available in the user interface. The system compares these configurations, shows how the parameter values differ, and presents the configuration information to the user for selection. By displaying these differing parameter values as part of the recommended configurations shown to the user, it meets the requirement of presenting the candidate parameter values of the first parameter type as recommendations for applying to the first-time series model.).
Regarding claim 7, Achin in view of Baldan teaches all the elements of claim 1, therefore is rejected for the same reasons as those presented for claim 1. Achin in view of Baldan further teaches the limitation:
identifying a plurality of system components associated with a particular workload of the monitored computer system, wherein the first system component is among the plurality of system components, wherein the metrics collected from the monitored computer system include workload values for the plurality of system components (Baldan, section 6.1 “To evaluate the proposal we have gathered datasets from four different cloud platforms…Log files are composed of historical resource usage data. Measurements are usually taken at one second intervals…In the case of the Google dataset, the task resource usage table contains the following fields: start and end time, job ID, task index, machine ID CPU usage (mean), memory usage, assigned memory, unmapped page cache memory usage, page cache memory usage, maximum memory usage, disk I/ O time (mean), local disk space used (mean), CPU usage (max), disk IO time (max), Cycles Per Instruction (CPI), Memory Accesses per Instruction (MAI), sampling rate and aggregation type. The other three datasets have a similar log format…The resulting series represents the workload of the system per minute.” – the monitoring system collects “historical resource usage data” for many distinct system components, including CPU usage, memory usage, disk I/O, CPI, and MAI. These multiple component metrics are aggregated into a time series that “represents the workload of the system per minute.”),
training a second a second time-series model to predict a second set of metrics for a second system component among the plurality of system components (Baldan, section 6.2 “we also apply all forecasting methods with all parameter combinations defined in the parameter grids to train on all time series, to verify on the training set that the initially chosen parameters are reasonable choices. In this way, we avoid over-fitting without loosing data for an additional validation set. The models with the chosen parameter configurations are executed over the respective test data. See Table 2 for the parameter configurations.” Section 6.1 “the task resource usage table contains…CPU usage (mean), memory usage…disk I/O time (mean)… Memory Accesses per Instruction (MAI)” – describes collecting separate time-series for many system components. Section 6.2 explains that the system trains forecasting models “on all time series,” meaning each component’s metrics receives its own trained model training process.).
Regarding claim 8, Achin in view of Baldan teaches all the elements of claim 7, therefore is rejected for the same reasons as those presented for claim 7. Achin in view of Baldan further teaches the limitation:
wherein the second time-series model is of a same type as the first time-series model (Baldan, section 5.1.1 “First, we describe thoroughly the ARIMA and ETS methods, which can be considered the de-facto standard models in time series forecasting [27]. Besides these state-of-the-art models, we also use an additional set of general regression methods for comparison purposes.” – methodology routinely applies the same model type – such as ARIMA – to multiple different time-series problems.) and wherein a second set of parameters associated with the second time-series model is different from a first set of parameters associated with the selected version of the first time-series model (Baldan, section 5.1.1. “ARIMA models are denoted as ARIMA(p, d, q), where p is the number of lagged values considered in the autoregressive part, q is the number of lagged values considered for the moving average part, and d is the number of differences considered.“ Page 936, Table 3 shows the same model type (ARIMA) is trained using different parameter sets for different datasets (each data corresponds to a different workload/system component). The listed ARIMA models – ARIMA(4,1,4) ARIMA(1,1,2) ARIMA(5,1,2) ARIMA(5,1,3) – are all ARIMA models but with different values for p, d, and q.).
Regarding claim 9, Achin in view of Baldan teaches all the elements of claim 7, therefore is rejected for the same reasons as those presented for claim 7. Achin in view of Baldan further teaches the limitation:
wherein the second time-series model is of a different type as the first time-series model (Baldan, section 5.1 “In this section, we describe the different forecasting algorithms employed in this framework. Since there is no single universally valid model and the time series to be analyzed have different properties, we use a wide range of forecasting models to obtain the fittest model according to their features.” Section 5.1.2 “Besides these state-of-the-art forecasting models, we also use a set of general regression models such as… Linear Auto-Regression…Lasso… Multi-Adaptive Regression Splines.” Section 5.1.3 describes different ML model types used in times series analysis such as Multi-Layer Perceptron with Back-Propagation, Elman Recurrent Network, and Support Vector Regression. – Baldan applies different types of time-series models (ARIMA vs. ETS vs. regression vs. neural networks) to the same workload, it teaches that a second time-series model may be a different type than the first time series model.).
Regarding claim 10, Achin in view of Baldan teaches all the elements of claim 9, therefore is rejected for the same reasons as those presented for claim 9. Achin in view of Baldan further teaches the limitation:
responsive to receiving a request (Achin paragraph [0041] “In some embodiments, the actions of the method further include determining a time resolution of the time-series data. In some embodiments, the targets are identified based on user input.”) to forecast workload values for the particular workload: accessing a first set of time-series data associated with the first system component (Baldan, section 6.1 “The datasets are original logs of these clusters, which we have preprocessed to extract the data relevant for the CPU usage, namely core-second values per minute. Log files are composed of historical resource usage data…the task resource usage table contains the following fields: start and end time, job ID, task index, machine ID CPU usage (mean), memory usage… As noted in Section 3, we focus on CPU usage. Data are processed so that the total number of cores used by all the tasks at a given one-second period is summed. The resulting series represents the workload of the system per minute.” – one the user identifies the forecasting target (request), the system accesses the corresponding workload time-series for the relevant system component.)
accessing a second set of time-series data associated with the second system component (Baldan, section 3, “In elastic cloud systems, the main variables to consider are usually related to the use of the main resources offered by the platform, like CPU and memory utilization, disk space, bandwidth, etc. These variables must be properly studied, forecasted and managed so that the system can make appropriate provisioning and management of resources at any moment…The study and subsequently defined methodology can be derived in a similar fashion for other resources…The values of CPU rate at each second composes a time series, which is fed to our forecasting module.” – identifies multiple system components – CPU, memory usage, and bandwidth – and explains each component’s measurements form its own time-series used for forecasting. Because the methodology applies separately to each resource, the reference inherently includes a first time series for one component, different time-series for another component.) ;
applying the first set of time-series data to the selected version of the first time-series model to generate a first prediction for the first system component (Section 6.1 “we focus on CPU usage. Data are processed so that the total number of cores used by all the tasks at a given one-second period is summed. The resulting series represents the workload of the system per minute.” Section 6.2 “The models with the chosen parameter configurations are executed over the respective test data…For every dataset the first 80 percent of data are used for model training, and the remaining 20 percent compose the test set… In addition, regarding the forecasting horizon, since the most common usage scenario is one-step-ahead prediction, every model has been designed for forecasting with this horizon.” – CPU usage time-series data is used as the model input, the model is trained on this data, and the selected model is then “executed over the…test data” to produce a forecast.);
applying the second set of time-series data to the second time-series model to generate a second prediction for the second component (Baldan, section 6.1 “To evaluate the proposal we have gathered datasets from four different cloud platforms. These are from Google clusters [40], LANL Origin 2000 Cluster (Nirvana), University Gaia Cluster of Luxemburg, and Sharnet Whale, [41]…. Log files are composed of historical resource usage data… the task resource usage table contains the following fields: start and end time, job ID, task index, machine ID CPU usage (mean), memory usage… The other three datasets have a similar log format.” Section 6.2 “The models with the chosen parameter configurations are executed over the respective test data… the forecasting horizon, since the most common usage scenario is one-step-ahead prediction, every model has been designed for forecasting with this horizon.” – describes multiple datasets (Google, LANL, Sharnet), each representing a different system component workload trace. Section 6 states that resource-usage measurements for each dataset are aggregated into separate time-series, and the selected forecasting models are “executed over the respective test data,” meaning each model is applied to the time-series of its corresponding dataset.) ; and
presenting the first prediction and the second prediction in response to the request to forecast the workload values for the particular workload (Achin, paragraph [0052] “the actions of the method further include displaying, via a graphical user interface, graphical content identifying the first and second features of the initial dataset and the model-specific predictive values of the first and second features.” Baldan, section 6.1 “the task resource usage table contains the following fields: start and end time, job ID, task index, machine ID CPU usage (mean), memory usage, assigned memory, unmapped page cache memory usage, page cache memory usage, maximum memory usage, disk I/O time (mean), local disk space used (mean)… As noted in Section 3, we focus on CPU usage. Data are processed so that the total number of cores used by all the tasks at a given one-second period is summed. The resulting series represents the workload of the system per minute.” Section 3 “CPU load and memory use can be seen as the most important and limited resources in a computer system… The values of CPU rate at each second composes a time series, which is fed to our forecasting module.” – Achin teaches presenting predictive outputs to a user through a graphical user interface. Baldan teaches that workload forecasting produces predicted workload values (e.g., CPU workload).).
Regarding claim 11, Achin teaches the following limitations:
accessing historical time-series data comprising metrics collected from a monitored computer system (Achin, paragraph [0042] “ Performing the predictive modeling procedure may include performing the pre-processing task, including: (a) obtaining time-series data including one or more data sets, wherein each data set includes a plurality of observations, wherein each observation includes (1) an indication of a time associated with the observation and (2) respective values of one or more variables; (b) determining a time interval of the time-series data; (c) identifying one or more of the variables as targets, and identifying zero or more other variables as features;”)
training a first time-series model to predict a first set of metrics for a first system component in the monitored computer system at least by (Achin, paragraph [0015] “predictive modeling method including performing a predictive modeling procedure, including: (a) obtaining time-series data including one or more data sets, wherein each data set includes a plurality of observations, wherein each observation includes (1) an indication of a time associated with the observation and (2) respective values of one or more variables…(c) identifying one or more of the variables as targets, and identifying zero or more other variables as features; (d) determining a forecast range…associated with a prediction problem represented by the time-series data, wherein the forecast range indicates a duration of a period for which values of the targets are to be predicted” (e) generating training data from the time-series data, wherein the training data include a first subset of the observations of at least one of the data sets, wherein the first subset of the observations includes training-input and training-output collections of the observations, wherein the times associated with the observations in the training-input and training-output collections correspond, respectively, to a training-input time range and a training-output time range,”):
dividing a first set of historical time-series data associated with the first system component into a training data set and a test data set (Achin, paragraph [0015] “(e) generating training data from the time-series data, wherein the training data include a first subset of the observations of at least one of the data sets, wherein the first subset of the observations includes training-input and training-output collections of the observations, wherein the times associated with the observations in the training-input and training-output collections correspond, respectively, to a training-input time range and a training-output time range, wherein the skip range separates an end of the training-input time range from a beginning of the training-output time range, and wherein a duration of the training-output time range is at least as long as the forecast range; (f) generating testing data from the time-series data, wherein the testing data include a second subset of the observations of at least one of the data sets, wherein the second subset of the observations includes testing-input and testing-validation collections of the observation…(g) fitting a predictive model to the training data; and (h) testing the fitted model on the testing data.”);
determining a search space of parameters for the first time-series model based on the first set of historical time-series data, at least by (Achin, paragraph [0100] “ in some embodiments a predictive modeling system 100 includes a predictive modeling exploration engine 110…The exploration engine 110 may implement a search technique (or “modeling methodology”) for efficiently exploring the predictive modeling search space (e.g., potential combinations of pre-processing steps, modeling algorithms, and post-processing steps) to generate a predictive modeling solution suitable for a specified prediction problem. The search technique may include an initial evaluation of which predictive modeling techniques are likely to provide suitable solutions for the prediction problem. In some embodiments, the search technique includes an incremental evaluation of the search space (e.g., using increasing fractions of a dataset), and a consistent comparison of the suitability of different modeling solutions for the prediction problem (e.g., using consistent metrics). In some embodiments, the search technique adapts based on results of prior searches, which can improve the effectiveness of the search technique over time.”):
evaluating performances of the multiple versions based on predictions generated by the multiple versions from the test data set (Achin, paragraph [0349] “As with cross-sectional models (see, e.g., the description of step 350 of the method 300, above), the engine 110 may iterate through the dataset, training each model on a small fraction of the training window, evaluating its performance on that fraction, then deciding whether to continue testing the model on additional data based on its performance.” – describes evaluating the performance of a predictive model on validation windows (which correspond to test data). The reference teaches iteratively generating multiple versions of the model by training on different fractions of the training window and then “evaluating its performance” on each corresponding validation fraction. Each fraction produces distinct fitted versions of the model, and the performance evaluation is explicitly based on predictions made for the validation (test ) portion of the time series data. Achin further explains that validation windows can be defined in weeks, months, or years and increase across multiple fractions, providing the multiple versions required by the claim.) ; and
selecting a version of the first time-series model with a highest performance in the multiple versions for use in forecasting the first set of metrics (Achin, paragraph [0404] “the system 100 selects predictive models for blending based on the model-specific predictive values, and blends the selected models (e.g., at step 432 of the method 400)… the system 100 may select “complementary top models” for blending…The system 100 may classify a model as a “top” model if the accuracy of the model is greater than a threshold accuracy, if the model has one of the N highest accuracy values among the fitted models…the system 100 may use the above-described classification techniques to select two or more complementary top models for blending.” – teaches selecting one or more predictive models based on performance metrics, including selecting models that have “one of the N highest accuracy values among the fitted models.” Accuracy is a direct measure of performance. Identifying the “top” model(s) among many fitted models necessarily involves comparing multiple versions and choosing the version with the highest performance. This satisfies the requirement of selecting the highest-performing version of the time-series model for use in forecasting the metrics.); and
storing the selected version of the first time-series model for use in forecasting the first set of metrics (Achin, paragraph [0320] “ File Storage 830. This module provides an interface to and manages storage mechanisms for files. Types of data stored via this module include uploaded datasets, derived data, model computations, and predictions. This module may layer a file directory and naming convention on top of cloud storage. Additionally, when cloud workers access this module, they may also temporarily cache the stored files in their local storage.”).
However, Achin does not teach but, Achin in view of Baldan teaches the following limitations:
applying a sample set of the historical time-series data to a conversion function to generate a set of correlogram data comprising a plurality of values (Achin, paragraph [0026] “ the actions of the method further include down-sampling the training data prior to fitting the predictive model to the training data. In some embodiments, down-sampling the training data includes: removing, from the training data, all observations obtained from at least one of the data sets. In some embodiments, down-sampling the training data includes setting a down-sampled time interval of the training data to an integer multiple of the time-interval of the time series data;”) Baldan, page 937, section 7 “Fig. 5 depicts the complete workload system series for the Google dataset hourly CPU load…However, in the ACF and PACF plots this pattern is not reflected, which suggests that the cycles change in length. In the PACF we see that lags 1, 2, and 5 are significant. In the differenced version, we get significant lags up to 12 and 16, respectively, in the ACF and PACF plots…we use a lag value of 24 to evaluate daily patterns in the seasonal study…We also use a value of 5, as this is a maximal lag choice commonly used in time series forecasting (as already discussed in Section 5.2) and as most of the series’ auto-correlation can be covered using the first five lags.” – shows that Baldan applies the historical time-series data to the auto-correlation and partial auto-correlation functions (conversion functions) generating ACF and PACF correlograms that include multiple correlation values across different lag intervals, meeting the limitation of “generating a set of correlogram (Fig 5) data comprising a plurality of values.).;
comparing the plurality of values to a threshold (Baldan, section 7.1, Fig. 5 “In the PACF we see that lags 1, 2, and 5 are significant. In the differenced version, we get significant lags up to 12 and 16, respectively, in the ACF and PACF plots.” – teaches comparing a plurality of correlation values to a significance threshold. Baldan explains that “lags 1, 3, and 5 are significant,” which inherently requires comparing the correlation values at those lags to the correlogram’s statistical confidence bounds (the dashed horizontal threshold likes in Fig. 5). The plotted values exceeding the threshold are identified as significant. Thus, Baldan teaches “comparing the plurality of values to a threshold.);
selecting, as candidate parameter values, a set of one or more values based on (a) the set of one or more values being equal to, or greater than, the threshold, and (b) a distance of each of the one or more values to the threshold (Baldan, section 7.1, “Fig. 5 depicts the complete workload system series for the Google dataset hourly CPU load.2…In the PACF we see that lags 1, 2, and 5 are significant. In the differenced version, we get significant lags up to 12 and 16, respectively, in the ACF and PACF plots.” – teaches selecting candidate parameter values based on whether their correlation values exceed a statistical threshold. Baldan states that “lags 1, 3, and 5 are significant,” which occurs only when the correlation values at those lags are equal to or greater than the significant threshold shown by the dashed confidence bounds in the correlogram. Because identifying significance is the ACF/PACF plot is inherently based on the distance between each correlation value and threshold lines, Baldan also teaches selecting values based on the distance of each correlation value to the threshold (Fig. 5). Thus, Baldan teaches selecting candidate parameter values based on both (a) exceeding a threshold and (b) the distance to the threshold.);
fitting multiple versions of the first time-series model to the training data set, wherein the multiple versions of the first time-series model comprise different combinations of the parameters in the search space, including different candidate parameter values obtained from the set of correlogram data (Achin, paragraph [0101] “The exploration engine 110 may use the library 130 of modeling techniques to evaluate potential modeling solutions in the search space. In some embodiments, the modeling technique library 130 includes machine-executable templates encoding complete modeling techniques. A machine-executable template may include one or more predictive modeling algorithms…a machine-executable template further includes one or more pre-processing and/or post-processing steps suitable for use with the template's algorithm(s). The algorithm(s), pre-processing steps, and/or post-processing steps may be parameterized.” Paragraph [0102] “The exploration engine 110 may uses the computational resources of a distributed computing system to explore the search space or portions thereof… interfaces that facilitate the evaluation of predictive modeling solutions in accordance with the search plan” Baldan, section 7.1, Fig. 5 “we see that lags 1, 2, and 5 are significant. In the differenced version, we get significant lags up to 12 and 16, respectively, in the ACF and PACF plots.” – Achin teaches exploring a parameterized model search space and evaluation multiple model versions with different parameter values. Baldan provides the specific candidate parameter values (significant ACF/PACF lags) that populate the search space.);
Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, having a combination of Achin and Baldan before them, to incorporate the time-series parameter-selection techniques of Baldan into the predictive-model generation framework of Achin. One would have been motivated to make such a combination in order to achieve a predictable, superior result in forecasting complex workload time series.
Regarding claim 12, Achin in view of Baldan teaches all the elements of claim 11, therefore is rejected for the same reasons as those presented for claim 11, the claim recites similar limitations corresponding to claim 2 and is rejected for similar reasons as claim 2 using similar teachings and rationale.
Regarding claim 13, Achin in view of Baldan teaches all the elements of claim 12, therefore is rejected for the same reasons as those presented for claim 12, the claim recites similar limitations corresponding to claim 3 and is rejected for similar reasons as claim 3 using similar teachings and rationale.
Regarding claim 14, Achin in view of Baldan teaches all the elements of claim 11, therefore is rejected for the same reasons as those presented for claim 11, the claim recites similar limitations corresponding to claim 4 and is rejected for similar reasons as claim 4 using similar teachings and rationale.
Regarding claim 15, Achin in view of Baldan teaches all the elements of claim 14, therefore is rejected for the same reasons as those presented for claim 14, the claim recites similar limitations corresponding to claim 5 and is rejected for similar reasons as claim 5 using similar teachings and rationale.
Regarding claim 16, Achin in view of Baldan teaches all the elements of claim 15, therefore is rejected for the same reasons as those presented for claim 15, the claim recites similar limitations corresponding to claim 6 and is rejected for similar reasons as claim 6 using similar teachings and rationale.
Regarding claim 17, Achin in view of Baldan teaches all the elements of claim 11, therefore is rejected for the same reasons as those presented for claim 11, the claim recites similar limitations corresponding to claim 7 and is rejected for similar reasons as claim 7 using similar teachings and rationale.
Regarding claim 18, Achin in view of Baldan teaches all the elements of claim 17, therefore is rejected for the same reasons as those presented for claim 17, the claim recites similar limitations corresponding to claim 8 and is rejected for similar reasons as claim 8 using similar teachings and rationale.
Regarding claim 19, Achin in view of Baldan teaches all the elements of claim 17, therefore is rejected for the same reasons as those presented for claim 17, the claim recites similar limitations corresponding to claim 9 and is rejected for similar reasons as claim 9 using similar teachings and rationale.
Regarding claim 20, Achin teaches the following limitations:
A system comprising: one or more processors; and memory storing instructions that, when executed by the one or more processors, cause the system to perform operations comprising (Achin paragraph [0060] “including a memory configured to store processor-executable instructions; and a processor configured to execute the processor-executable instructions, wherein executing the processor-executable instructions”):
accessing historical time-series data comprising metrics collected from a monitored computer system (Achin, paragraph [0042] “ Performing the predictive modeling procedure may include performing the pre-processing task, including: (a) obtaining time-series data including one or more data sets, wherein each data set includes a plurality of observations, wherein each observation includes (1) an indication of a time associated with the observation and (2) respective values of one or more variables; (b) determining a time interval of the time-series data; (c) identifying one or more of the variables as targets, and identifying zero or more other variables as features;”)
training a first time-series model to predict a first set of metrics for a first system component in the monitored computer system at least by (Achin, paragraph [0015] “predictive modeling method including performing a predictive modeling procedure, including: (a) obtaining time-series data including one or more data sets, wherein each data set includes a plurality of observations, wherein each observation includes (1) an indication of a time associated with the observation and (2) respective values of one or more variables…(c) identifying one or more of the variables as targets, and identifying zero or more other variables as features; (d) determining a forecast range…associated with a prediction problem represented by the time-series data, wherein the forecast range indicates a duration of a period for which values of the targets are to be predicted” (e) generating training data from the time-series data, wherein the training data include a first subset of the observations of at least one of the data sets, wherein the first subset of the observations includes training-input and training-output collections of the observations, wherein the times associated with the observations in the training-input and training-output collections correspond, respectively, to a training-input time range and a training-output time range,”):
dividing a first set of historical time-series data associated with the first system component into a training data set and a test data set (Achin, paragraph [0015] “(e) generating training data from the time-series data, wherein the training data include a first subset of the observations of at least one of the data sets, wherein the first subset of the observations includes training-input and training-output collections of the observations, wherein the times associated with the observations in the training-input and training-output collections correspond, respectively, to a training-input time range and a training-output time range, wherein the skip range separates an end of the training-input time range from a beginning of the training-output time range, and wherein a duration of the training-output time range is at least as long as the forecast range; (f) generating testing data from the time-series data, wherein the testing data include a second subset of the observations of at least one of the data sets, wherein the second subset of the observations includes testing-input and testing-validation collections of the observation…(g) fitting a predictive model to the training data; and (h) testing the fitted model on the testing data.”);
determining a search space of parameters for the first time-series model based on the first set of historical time-series data, at least by (Achin, paragraph [0100] “ in some embodiments a predictive modeling system 100 includes a predictive modeling exploration engine 110…The exploration engine 110 may implement a search technique (or “modeling methodology”) for efficiently exploring the predictive modeling search space (e.g., potential combinations of pre-processing steps, modeling algorithms, and post-processing steps) to generate a predictive modeling solution suitable for a specified prediction problem. The search technique may include an initial evaluation of which predictive modeling techniques are likely to provide suitable solutions for the prediction problem. In some embodiments, the search technique includes an incremental evaluation of the search space (e.g., using increasing fractions of a dataset), and a consistent comparison of the suitability of different modeling solutions for the prediction problem (e.g., using consistent metrics). In some embodiments, the search technique adapts based on results of prior searches, which can improve the effectiveness of the search technique over time.”):
evaluating performances of the multiple versions based on predictions generated by the multiple versions from the test data set (Achin, paragraph [0349] “As with cross-sectional models (see, e.g., the description of step 350 of the method 300, above), the engine 110 may iterate through the dataset, training each model on a small fraction of the training window, evaluating its performance on that fraction, then deciding whether to continue testing the model on additional data based on its performance.” – describes evaluating the performance of a predictive model on validation windows (which correspond to test data). The reference teaches iteratively generating multiple versions of the model by training on different fractions of the training window and then “evaluating its performance” on each corresponding validation fraction. Each fraction produces distinct fitted versions of the model, and the performance evaluation is explicitly based on predictions made for the validation (test ) portion of the time series data. Achin further explains that validation windows can be defined in weeks, months, or years and increase across multiple fractions, providing the multiple versions required by the claim.) ; and
selecting a version of the first time-series model with a highest performance in the multiple versions for use in forecasting the first set of metrics (Achin, paragraph [0404] “the system 100 selects predictive models for blending based on the model-specific predictive values, and blends the selected models (e.g., at step 432 of the method 400)… the system 100 may select “complementary top models” for blending…The system 100 may classify a model as a “top” model if the accuracy of the model is greater than a threshold accuracy, if the model has one of the N highest accuracy values among the fitted models…the system 100 may use the above-described classification techniques to select two or more complementary top models for blending.” – teaches selecting one or more predictive models based on performance metrics, including selecting models that have “one of the N highest accuracy values among the fitted models.” Accuracy is a direct measure of performance. Identifying the “top” model(s) among many fitted models necessarily involves comparing multiple versions and choosing the version with the highest performance. This satisfies the requirement of selecting the highest-performing version of the time-series model for use in forecasting the metrics.); and
storing the selected version of the first time-series model for use in forecasting the first set of metrics (Achin, paragraph [0320] “ File Storage 830. This module provides an interface to and manages storage mechanisms for files. Types of data stored via this module include uploaded datasets, derived data, model computations, and predictions. This module may layer a file directory and naming convention on top of cloud storage. Additionally, when cloud workers access this module, they may also temporarily cache the stored files in their local storage.”).
However, Achin does not teach but, Achin in view of Baldan teaches the following limitations:
applying a sample set of the historical time-series data to a conversion function to generate a set of correlogram data comprising a plurality of values (Achin, paragraph [0026] “ the actions of the method further include down-sampling the training data prior to fitting the predictive model to the training data. In some embodiments, down-sampling the training data includes: removing, from the training data, all observations obtained from at least one of the data sets. In some embodiments, down-sampling the training data includes setting a down-sampled time interval of the training data to an integer multiple of the time-interval of the time series data;”) Baldan, page 937, section 7 “Fig. 5 depicts the complete workload system series for the Google dataset hourly CPU load…However, in the ACF and PACF plots this pattern is not reflected, which suggests that the cycles change in length. In the PACF we see that lags 1, 2, and 5 are significant. In the differenced version, we get significant lags up to 12 and 16, respectively, in the ACF and PACF plots…we use a lag value of 24 to evaluate daily patterns in the seasonal study…We also use a value of 5, as this is a maximal lag choice commonly used in time series forecasting (as already discussed in Section 5.2) and as most of the series’ auto-correlation can be covered using the first five lags.” – shows that Baldan applies the historical time-series data to the auto-correlation and partial auto-correlation functions (conversion functions) generating ACF and PACF correlograms that include multiple correlation values across different lag intervals, meeting the limitation of “generating a set of correlogram (Fig 5) data comprising a plurality of values.).;
comparing the plurality of values to a threshold (Baldan, section 7.1, Fig. 5 “In the PACF we see that lags 1, 2, and 5 are significant. In the differenced version, we get significant lags up to 12 and 16, respectively, in the ACF and PACF plots.” – teaches comparing a plurality of correlation values to a significance threshold. Baldan explains that “lags 1, 3, and 5 are significant,” which inherently requires comparing the correlation values at those lags to the correlogram’s statistical confidence bounds (the dashed horizontal threshold likes in Fig. 5). The plotted values exceeding the threshold are identified as significant. Thus, Baldan teaches “comparing the plurality of values to a threshold.);
selecting, as candidate parameter values, a set of one or more values based on (a) the set of one or more values being equal to, or greater than, the threshold, and (b) a distance of each of the one or more values to the threshold (Baldan, section 7.1, “Fig. 5 depicts the complete workload system series for the Google dataset hourly CPU load.2…In the PACF we see that lags 1, 2, and 5 are significant. In the differenced version, we get significant lags up to 12 and 16, respectively, in the ACF and PACF plots.” – teaches selecting candidate parameter values based on whether their correlation values exceed a statistical threshold. Baldan states that “lags 1, 3, and 5 are significant,” which occurs only when the correlation values at those lags are equal to or greater than the significant threshold shown by the dashed confidence bounds in the correlogram. Because identifying significance is the ACF/PACF plot is inherently based on the distance between each correlation value and threshold lines, Baldan also teaches selecting values based on the distance of each correlation value to the threshold (Fig. 5). Thus, Baldan teaches selecting candidate parameter values based on both (a) exceeding a threshold and (b) the distance to the threshold.);
fitting multiple versions of the first time-series model to the training data set, wherein the multiple versions of the first time-series model comprise different combinations of the parameters in the search space, including different candidate parameter values obtained from the set of correlogram data (Achin, paragraph [0101] “The exploration engine 110 may use the library 130 of modeling techniques to evaluate potential modeling solutions in the search space. In some embodiments, the modeling technique library 130 includes machine-executable templates encoding complete modeling techniques. A machine-executable template may include one or more predictive modeling algorithms…a machine-executable template further includes one or more pre-processing and/or post-processing steps suitable for use with the template's algorithm(s). The algorithm(s), pre-processing steps, and/or post-processing steps may be parameterized.” Paragraph [0102] “The exploration engine 110 may uses the computational resources of a distributed computing system to explore the search space or portions thereof… interfaces that facilitate the evaluation of predictive modeling solutions in accordance with the search plan” Baldan, section 7.1, Fig. 5 “we see that lags 1, 2, and 5 are significant. In the differenced version, we get significant lags up to 12 and 16, respectively, in the ACF and PACF plots.” – Achin teaches exploring a parameterized model search space and evaluation multiple model versions with different parameter values. Baldan provides the specific candidate parameter values (significant ACF/PACF lags) that populate the search space.);
Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, having a combination of Achin and Baldan before them, to incorporate the time-series parameter-selection techniques of Baldan into the predictive-model generation framework of Achin. One would have been motivated to make such a combination in order to achieve a predictable, superior result in forecasting complex workload time series.
Regarding claim 21, Achin in view of Baldan teaches all the elements of claim 1, therefore is rejected for the same reasons as those presented for claim 1. Achin in view of Baldan further teaches the limitation:
wherein selecting the candidate parameter values comprises: identifying a first value in the correlogram data that is equal to, or greater than, the threshold; identifying a second value in the correlogram data that is equal to, or greater than, the threshold (Baldan, [section 7.1, Fig. 5] “In the PACF we see that lags 1, 2, and 5 are significant. In the differenced version, we get significant lags up to 12 and 16, respectively, in the ACF and PACF plots.” – Fig. 5 of Baldan includes ACF and PACF correlogram plots with dashed horizontal confidence-bound lines representing the statistical significance threshold for the plotted correlation values. The correlation bars at lags 1, 2, and 5 in the PACF plot visibly exceed the confidence-bound threshold, and Baldan expressly identifies these as “significant.” A correlation value that exceeds a confidence-bound threshold corresponds to a value equal to, or greater than, the threshold in the correlogram data. Accordingly, Baldan teaches identifying at least a first and a second correlogram values that meet or exceed the threshold, satisfying the claimed requirement of selecting candidate parameter values based on threshold-exceeding correlogram values.);
based on a first distance of the first value from the threshold and a second distance of the second value from the threshold: selecting the first value as a first candidate parameter value among the candidate parameter values (Baldan, [section 7.1, Fig. 5] “In the PACF we see that lags 1, 2, and 5 are significant. In the differenced version, we get significant lags up to 12 and 16, respectively, in the ACF and PACF plots.” – Fig. 5 of Baldan includes ACF and PACF correlogram plots with dashed horizontal confidence-bound threshold lines representing statistical significance thresholds. The plotted correlation bars exceed the threshold by different magnitudes, where some bars extend only slightly above the threshold while others extend substantially beyond it. These differing distances visually indicate differing strengths of autocorrelation among the significant lag values identified by Baldan. A POSITA would have understood that, when selecting lag parameters for time-series forecasting, correlation values exhibiting greater exceedance above the significance threshold reflect strong and more reliable autocorrelation relationships, and therefore would reasonably by selected as candidate lag parameters. Conversely, weaker threshold-exceeding correlations would reasonably be excluded from the candidate set to reduce unnecessary parameter exploration and improve model performance. Thus, Baldan’s correlogram analysis, combined with ordinary skill in the art, teaches selecting one threshold-exceeding value and refraining from including another based on their respective distances above the threshold.);
Regarding claim 21, Achin in view of Baldan teaches all the elements of claim 1, therefore is rejected for the same reasons as those presented for claim 1. Achin in view of Baldan further teaches the limitation:
wherein selecting the candidate parameter values comprises: identifying a first value in the correlogram data that is equal to, or greater than, the threshold; identifying a second value in the correlogram data that is equal to, or greater than, the threshold (Baldan, [section 7.1, Fig. 5] “In the PACF we see that lags 1, 2, and 5 are significant. In the differenced version, we get significant lags up to 12 and 16, respectively, in the ACF and PACF plots.” – Fig. 5 of Baldan includes ACF and PACF correlogram plots with dashed horizontal confidence-bound lines representing the statistical significance threshold for the plotted correlation values. The correlation bars at lags 1, 2, and 5 in the PACF plot visibly exceed the confidence-bound threshold, and Baldan expressly identifies these as “significant.” A correlation value that exceeds a confidence-bound threshold corresponds to a value equal to, or greater than, the threshold in the correlogram data. Accordingly, Baldan teaches identifying at least a first and a second correlogram values that meet or exceed the threshold, satisfying the claimed requirement of selecting candidate parameter values based on threshold-exceeding correlogram values.);
based on a first distance of the first value from the threshold and a second distance of the second value from the threshold: selecting the first value as a first candidate parameter value among the candidate parameter values; and refraining from including the second value among the candidate parameter values, wherein the first distance smaller than the second distance (Baldan, [section 7.1, Fig. 5] “In the PACF we see that lags 1, 2, and 5 are significant. In the differenced version, we get significant lags up to 12 and 16, respectively, in the ACF and PACF plots.” – Fig. 5 of Baldan includes ACF and PACF correlogram plots with dashed horizontal confidence-bound threshold lines representing statistical significance thresholds. The plotted correlation bars exceed the threshold by different magnitudes, where some bars extend only slightly above the threshold value while others extend substantially beyond it. These differing distances visually indicate differing levels of autocorrelations among the significant lag values identified by Baldan. A POSITA would have understood that, when multiple correlogram values exceed a significance threshold by different distances, comparative evaluation of those distances is a routine part forecasting-model parameter selection. In particular, selecting threshold-exceeding value that lies closer to the significance boundary (i.e., having a smaller distance) while excluding a value that lies farther above the threshold can be desirable to reduce model complexity, avoid overfitting, and prioritize minimally sufficient autocorrelation signals. Forecasting practitioners do not automatically include every large correlogram spike; instead, the routinely balance statistical significance against parsimony and model stability. Thus, Baldan’s correlogram analysis, combined with ordinary skill in the art, would have suggested selecting the threshold-exceeding value having the smaller distance from the threshold and refraining from including the threshold-exceeding value having the larger distance.).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Daravanh Phakousonh whose telephone number is (571)272-6324. The examiner can normally be reached Mon - Thurs 7 AM - 5 PM, Every other Friday 7 AM - 4PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Li B Zhen can be reached at 571-272-3768. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Daravanh Phakousonh/Examiner, Art Unit 2121
/Li B. Zhen/Supervisory Patent Examiner, Art Unit 2121
Prosecution Insights