DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-2, 4-7, 9-13 and 20 is/are rejected under 35 U.S.C. 102 (a)(1) as being anticipated by Jiang et al. (“An improved Stacking framework for stock index prediction by leveraging tree-based ensemble models and deep learning algorithms”, Physica A 541 (2020) 122272).
Regarding claim 1.
Jiang teaches a method comprising: obtaining, by a computer system, a data set comprising time series data (see page 14, “exponentially smoothed technical indicators and macroeconomic features are employed as input variables to forecast the movement of the three major stock price indices in U.S. namely S&P500, Dow 30 and Nasdaq, from the period of Jan 2003 to Apr 2019. Four tree-based ensemble models, called RF, ERT, XGBoost and LightGBM, and four deep learning models, called RNN, BRNN, LSTM and GRU, are stacked as base classifiers in the first layer. Cross-validation method is then deployed to iteratively collect the prediction probabilities of each base learner and generate the input for the next layer classifier to prevent overfitting. In the second layer, logistic regression, as well as its variation which utilize a Lasso penalty term in the loss function, namely Meta-LR and Meta-LassoLR, respectively, are applied as meta-classifier to identify the unique learning patterns of the base learners.”, also see figure 5 on page 9, input block contains time series ,
PNG
media_image1.png
44
364
media_image1.png
Greyscale
, and table 3 on page 10, input
PNG
media_image2.png
21
932
media_image2.png
Greyscale
, also see pagr 8, “RNN and BRNN can extract the information from the time series data of arbitrary length.”);
partitioning, by the computer system, the data set into a plurality of segments (see page 9, section 4.3, “The first step of our improved Stacking algorithm is to gather each base classifiers’ output into a new set of data. According with the concept of cross-validation method, the training dataset is firstly split into k folds. During the k successive rounds, k − 1 folds are utilized to form the first-level classifiers and then applied to the remaining 1 validation subset that is not utilized for training in each iteration. The resulting prediction probabilities, along with the true instances’ classes, are then merged and provided as input data to the level-2 classifier. In the second step, a specific learning algorithm is employed as meta-classifier to extract the information and combine the learning patterns from the base classifiers. The framework and pseudo codes of Stacking algorithm are illustrated in Fig. 5 and Table 3, respectively.”, also table 3,
PNG
media_image3.png
50
932
media_image3.png
Greyscale
);
for each segment of the plurality of segments, creating, by the computer system, a plurality of segment groups, each segment group comprising a time series training data set and a time series test data set, thereby producing a plurality of time series training data sets and a plurality of time series test data sets (see page 9, section 4.3, “The first step of our improved Stacking algorithm is to gather each base classifiers’ output into a new set of data. According with the concept of cross-validation method, the training dataset is firstly split into k folds. During the k successive rounds, k − 1 folds are utilized to form the first-level classifiers and then applied to the remaining 1 validation subset that is not utilized for training in each iteration. The resulting prediction probabilities, along with the true instances’ classes, are then merged and provided as input data to the level-2 classifier. In the second step, a specific learning algorithm is employed as meta-classifier to extract the information and combine the learning patterns from the base classifiers. The framework and pseudo codes of Stacking algorithm are illustrated in Fig. 5 and Table 3, respectively.”, also see figure 5, training folds and validation folds for training and testing data set across kth iteration, also see page 3, section 3.1);
training, by the computer system, each machine learning model of a plurality of machine learning models using the plurality of time series training data sets, thereby producing a plurality of trained machine learning models (see page 14, “exponentially smoothed technical indicators and macroeconomic features are employed as input variables to forecast the movement of the three major stock price indices in U.S. namely S&P500, Dow 30 and Nasdaq, from the period of Jan 2003 to Apr 2019. Four tree-based ensemble models, called RF, ERT, XGBoost and LightGBM, and four deep learning models, called RNN, BRNN, LSTM and GRU, are stacked as base classifiers in the first layer. Cross-validation method is then deployed to iteratively collect the prediction probabilities of each base learner and generate the input for the next layer classifier to prevent overfitting. In the second layer, logistic regression, as well as its variation which utilize a Lasso penalty term in the loss function, namely Meta-LR and Meta-LassoLR, respectively, are applied as meta-classifier to identify the unique learning patterns of the base learners.”, also see table 3, steps 4-6 for example);
determining, by the computer system, using the plurality of trained machine learning models, a plurality of time series forecast data sets that correspond to the plurality of time series test data sets (see page 14, “exponentially smoothed technical indicators and macroeconomic features are employed as input variables to forecast the movement of the three major stock price indices in U.S. namely S&P500, Dow 30 and Nasdaq, from the period of Jan 2003 to Apr 2019. Four tree-based ensemble models, called RF, ERT, XGBoost and LightGBM, and four deep learning models, called RNN, BRNN, LSTM and GRU, are stacked as base classifiers in the first layer. Cross-validation method is then deployed to iteratively collect the prediction probabilities of each base learner and generate the input for the next layer classifier to prevent overfitting. In the second layer, logistic regression, as well as its variation which utilize a Lasso penalty term in the loss function, namely Meta-LR and Meta-LassoLR, respectively, are applied as meta-classifier to identify the unique learning patterns of the base learners.”, also see table 3, steps 10 for example);
stacking, by the computer system, the plurality of time series forecast datasets according to time for each machine learning model, thereby creating a plurality of stacked time series forecast data sets corresponding to the plurality of machine learning models (see page 9, section 4.3, “The first step of our improved Stacking algorithm is to gather each base classifiers’ output into a new set of data. According with the concept of cross-validation method, the training dataset is firstly split into k folds. During the k successive rounds, k − 1 folds are utilized to form the first-level classifiers and then applied to the remaining 1 validation subset that is not utilized for training in each iteration. The resulting prediction probabilities, along with the true instances’ classes, are then merged and provided as input data to the level-2 classifier. In the second step, a specific learning algorithm is employed as meta-classifier to extract the information and combine the learning patterns from the base classifiers. The framework and pseudo codes of Stacking algorithm are illustrated in Fig. 5 and Table 3, respectively.”, also see figure 5, output vectors from each iteration combined/stacked sequentially into all level 1 prediction probability);
and training, by the computer system, an ensemble machine learning model to generate a combined forecast using the plurality of stacked time series forecast data sets from the plurality of trained machine learning models and the data set comprising time series data (see page 14, “exponentially smoothed technical indicators and macroeconomic features are employed as input variables to forecast the movement of the three major stock price indices in U.S. namely S&P500, Dow 30 and Nasdaq, from the period of Jan 2003 to Apr 2019. Four tree-based ensemble models, called RF, ERT, XGBoost and LightGBM, and four deep learning models, called RNN, BRNN, LSTM and GRU, are stacked as base classifiers in the first layer. Cross-validation method is then deployed to iteratively collect the prediction probabilities of each base learner and generate the input for the next layer classifier to prevent overfitting. In the second layer, logistic regression, as well as its variation which utilize a Lasso penalty term in the loss function, namely Meta-LR and Meta-LassoLR, respectively, are applied as meta-classifier to identify the unique learning patterns of the base learners.”, also see table 3, steps 13-19 for example and figure 5, combining level 1 and level 2 into final prediction).
Regarding claim 2.
Jiang discloses the method of claim 1,
Jiang further discloses wherein the data set comprises a sequence
of time series data, wherein the plurality of segments comprise a plurality of non- overlapping subsequences of time series data from the sequence of time series data,
and wherein the plurality of time series training data sets and the plurality of time series
test data sets comprise a plurality of subsequences of time series data from the plurality of non-overlapping subsequences of time series data (see page 9, section 4.3, “The first step of our improved Stacking algorithm is to gather each base classifiers’ output into a new set of data. According with the concept of cross-validation method, the training dataset is firstly split into k folds. During the k successive rounds, k − 1 folds are utilized to form the first-level classifiers and then applied to the remaining 1 validation subset that is not utilized for training in each iteration. The resulting prediction probabilities, along with the true instances’ classes, are then merged and provided as input data to the level-2 classifier. In the second step, a specific learning algorithm is employed as meta-classifier to extract the information and combine the learning patterns from the base classifiers. The framework and pseudo codes of Stacking algorithm are illustrated in Fig. 5 and Table 3, respectively.”, also see figure 5, training folds and validation folds for training and testing data set across kth iteration, wherein division is distinct non-overlapping blocks representing the folds, table 3,
PNG
media_image3.png
50
932
media_image3.png
Greyscale
also see page 3, section 3.1).
Regarding claim 4.
Jiang discloses the method of claim 1,
Jiang further discloses wherein the ensemble machine learning
model comprises a neural network, a linear regression model, or a logistic regression model (see page 14, “In the second layer, logistic regression, as well as its variation which utilize a Lasso penalty term in the loss function, namely Meta-LR and Meta-LassoLR, respectively, are applied as meta-classifier to identify the unique learning patterns of the base learners.”).
Regarding claim 5.
Jiang discloses the method of claim 1,
Jiang further discloses wherein the ensemble machine learning model comprises a combination model, wherein the ensemble machine learning model generates a combined forecast by generating a weighted combination of a plurality of base model forecast generated by the plurality of trained machine learning models, and wherein training the ensemble machine learning model comprises determining a plurality of weights corresponding to the plurality of trained machine learning models, the weighted combination determined, in part, by the plurality of weights (see page 13, “In the last section of the empirical results, we manage to explore the structure of our Stacking framework. With Meta- LR and its regularized version, regression coefficients, i.e., the learning weights of each base classifier, can be extracted, as is demonstrated in Table 6… Meta-LassoLR model shrinks the weights of BRNN and GRU, as well as RF to zero in predicting S&P500 and Nasdaq, respectively. In other words, the meta-classifier disregards the resulting predictions of some specific base learners since it considers this will not be helpful to enhance the prediction accuracy. Moreover, Meta-Lasso LR eliminates all the forecasting results except for ERT and Light GBM when it refers to the Dow 30 stock index. All the above conclusions suggest that Meta-Lasso LR is capable of weighting/selecting the optimal base classifiers for such prediction problem and thus achieve a better level of forecasting performance.”, also see page 9, “We firstly choose the logistic regression as meta-classifier since it is one of the most basic and traditional ways of Stacking [44,45]. This stacked model is then extended by deploying a Lasso (least absolute shrinkage and selection operator) based regularization term [46] for considering the multicollinearity of the prediction probabilities extracting from the base learners. Finally, we attempt to combine the original features with the predictions of the base classifiers in the second-level training process”).
Regarding claim 6.
Jiang discloses the method of claim 1,
Jiang further discloses wherein each time series training data set corresponds to a training time period, wherein each time series test data set corresponds to a testing time period, and wherein the computer system creates the plurality of segment groups, such that each training time period immediately precedes a corresponding testing time period (see figure 5 and page 3, “This study uses first 2400 days (approximately 58.91% of the entire data) as the training set ranging from Jan 2003 to Aug 2012. It is worth noting that we deliberately set aside 20 days between the training set and the testing set to prevent information leakage. Therefore, our testing set contains 1654 days (approximately 40.6% of the entire data) ranging from Sep 2012 to Apr 2019.”).
Regarding claim 7.
Jiang discloses the method of claim 1,
Jiang further discloses wherein each time series forecast data set of the plurality of time series forecast data sets correspond to a respective time period, and wherein the plurality of time series forecast data sets in each segment do not overlap in time as a result of no two or more of the time series forecast data sets corresponding to overlapping time periods (see page 9, section 4.3, “The first step of our improved Stacking algorithm is to gather each base classifiers’ output into a new set of data. According with the concept of cross-validation method, the training dataset is firstly split into k folds. During the k successive rounds, k − 1 folds are utilized to form the first-level classifiers and then applied to the remaining 1 validation subset that is not utilized for training in each iteration. The resulting prediction probabilities, along with the true instances’ classes, are then merged and provided as input data to the level-2 classifier. In the second step, a specific learning algorithm is employed as meta-classifier to extract the information and combine the learning patterns from the base classifiers. The framework and pseudo codes of Stacking algorithm are illustrated in Fig. 5 and Table 3, respectively.”, also table 3,
PNG
media_image3.png
50
932
media_image3.png
Greyscale
also see figure 5, training folds and validation folds for training and testing data set across kth iteration, wherein division is distinct non-overlapping blocks representing the folds).
Regarding claim 9.
Jiang discloses the method of claim 1,
Jiang further discloses wherein the plurality of time series forecast data sets comprise a plurality of pluralities of base model forecast data sets, each plurality of base model forecast data sets corresponding to a different machine learning model of the plurality of machine learning models, wherein each base model forecast data set comprises a chronologically ordered sequence of time series forecast data values associated with a sequence of forecast timestamps or a sequence of forecast indices, and wherein stacking the plurality of time series forecast data sets according to time for each machine learning model comprises: determining, by the computer system, for a corresponding plurality of base
model forecast data sets, a chronological ordering of base model forecast data sets within that plurality of base model forecast data sets; and combining, by the computer system, the corresponding plurality of base model forecast data sets according to the chronological ordering, thereby stacking the plurality of time series forecast data sets according to time for each machine learning model (see page 9, section 4.3, “The first step of our improved Stacking algorithm is to gather each base classifiers’ output into a new set of data. According with the concept of cross-validation method, the training dataset is firstly split into k folds. During the k successive rounds, k − 1 folds are utilized to form the first-level classifiers and then applied to the remaining 1 validation subset that is not utilized for training in each iteration. The resulting prediction probabilities, along with the true instances’ classes, are then merged and provided as input data to the level-2 classifier. In the second step, a specific learning algorithm is employed as meta-classifier to extract the information and combine the learning patterns from the base classifiers. The framework and pseudo codes of Stacking algorithm are illustrated in Fig. 5 and Table 3, respectively.”, also table 3,
PNG
media_image4.png
72
560
media_image4.png
Greyscale
, also see figure 5. Probability -> all level 1 prediction and level 2 meta classifier).
Regarding claim 10.
Jiang discloses the method of claim 9,
Jiang further discloses wherein combining the corresponding
plurality of base model forecast data sets comprises:
concatenating the corresponding plurality of base model forecast data sets
together such that the plurality of base model forecast data sets are in the chronological ordering (see page 9, section 4.3, “The first step of our improved Stacking algorithm is to gather each base classifiers’ output into a new set of data. According with the concept of cross-validation method, the training dataset is firstly split into k folds. During the k successive rounds, k − 1 folds are utilized to form the first-level classifiers and then applied to the remaining 1 validation subset that is not utilized for training in each iteration. The resulting prediction probabilities, along with the true instances’ classes, are then merged and provided as input data to the level-2 classifier. In the second step, a specific learning algorithm is employed as meta-classifier to extract the information and combine the learning patterns from the base classifiers. The framework and pseudo codes of Stacking algorithm are illustrated in Fig. 5 and Table 3, respectively.”, also table 3,
PNG
media_image4.png
72
560
media_image4.png
Greyscale
, also see figure 5. Probability -> all level 1 prediction and level 2 meta classifier).
Regarding claim 11.
Jiang discloses the method of claim 1,
Jiang further discloses the method further comprising: receiving, by the computer system, a request from a requestor to generate a requested time series forecast data set corresponding to a request data set; obtaining, by the computer system, the request data set;
determining, by the computer system, using the plurality of trained machine learning models and the request data set, a plurality of requested time series forecast data sets; generating, by the computer system, the requested time series forecast data set using the ensemble machine learning model and the plurality of requested time series forecast data sets; and
providing, by the computer system, to the requestor, the requested time series forecast data set (see page 9, section 4.3, “The first step of our improved Stacking algorithm is to gather each base classifiers’ output into a new set of data. According with the concept of cross-validation method, the training dataset is firstly split into k folds. During the k successive rounds, k − 1 folds are utilized to form the first-level classifiers and then applied to the remaining 1 validation subset that is not utilized for training in each iteration. The resulting prediction probabilities, along with the true instances’ classes, are then merged and provided as input data to the level-2 classifier. In the second step, a specific learning algorithm is employed as meta-classifier to extract the information and combine the learning patterns from the base classifiers. The framework and pseudo codes of Stacking algorithm are illustrated in Fig. 5 and Table 3, respectively.”, also see figure 5, training folds and validation folds for training and testing data set across kth iteration, also see table 3).
Regarding claim 12.
Jiang discloses the method of claim 11,
Jiang further discloses wherein the computer system trains the ensemble machine learning model to generate a combined forecast using one or more external training features in addition to the plurality of stacked time series forecast data sets and the data set comprising time series data, wherein the computer system generates the requested time series forecast data set using the ensemble machine learning model using one or more external features in addition to the plurality of requested time series forecast data sets, and wherein the method further comprises: obtaining the one or more external training features; and obtaining the one or more external features (see page 14, “exponentially smoothed technical indicators and macroeconomic features are employed as input variables to forecast the movement of the three major stock price indices in U.S. namely S&P500, Dow 30 and Nasdaq, from the period of Jan 2003 to Apr 2019. Four tree-based ensemble models, called RF, ERT, XGBoost and LightGBM, and four deep learning models, called RNN, BRNN, LSTM and GRU, are stacked as base classifiers in the first layer. Cross-validation method is then deployed to iteratively collect the prediction probabilities of each base learner and generate the input for the next layer classifier to prevent overfitting. In the second layer, logistic regression, as well as its variation which utilize a Lasso penalty term in the loss function, namely Meta-LR and Meta-LassoLR, respectively, are applied as meta-classifier to identify the unique learning patterns of the base learners.”, also see page 13, “our results indicate that implementing a Lasso based regularization term improves all of the three evaluation metrics over the LR based meta-classifier. However, meta-classifier fails to provide better prediction performance
when it is fitted both on the predictions of the base learners and the original features (i.e., Meta-LassoLR∗). This may be related to the aspect of information extraction and we will further analyze in detail in the next section.”, also see figure 10 caption).
Regarding claim 13.
Jiang discloses the method of claim 11,
Jiang further discloses wherein the requested time series forecast data set corresponds to forecasted demand for a service or corresponds to a forecasted service time corresponding to that service, or wherein the requested time series forecast data set comprises a first statistic used to reduce a variance in a second statistic using variance reduction techniques (see page 14, “exponentially smoothed technical indicators and macroeconomic features are employed as input variables to forecast the movement of the three major stock price indices in U.S. namely S&P500, Dow 30 and Nasdaq, from the period of Jan 2003 to Apr 2019.”).
Regarding claim 20, claim 20 recites a system to perform the method recited in claim 1. Therefore the rejection of claim 1 above applies equally here.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 3, 8 and 14-16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jiang et al. (“An improved Stacking framework for stock index prediction by leveraging tree-based ensemble models and deep learning algorithms”, Physica A 541 (2020) 122272) in view of Rahimi et al. (US 20240311650 A1).
Regarding claim 3.
Jiang discloses the method of claim 1,
Jiang do not teach claim 3,
Rahimi teaches wherein training, by the computer system, the ensemble machine learning model to generate a combined forecast using the plurality of stacked time series forecast data sets from the plurality of trained machine learning models and the data set comprising the time series data comprises: determining, by the computer system, for each trained machine learning model, a set of error terms comprising at least one error term by comparing a stacked time series forecast data set corresponding to that machine learning model to an actual time series data set derived from the data set comprising the time series data, thereby determining a plurality of sets of error terms; and updating a parameter set associated with the ensemble machine learning model based on the plurality of sets of error terms (see ¶ 42, “the set of time series data is initially passed through an outlier removal technique 206 which may be configured to ensure that any impossible data or human error is not being fed into the model creation pipeline. Outliers may be initially detected by setting a threshold value for a maximum amount of standard deviation presented, and checking any data points that might be over that margin.”, also see ¶ 45, “a value imputation technique 208 may be performed for data intervals that might be missing or have a “not-a-number” (NAN) value. The value imputation technique 208 is important because there are models that would present an error in training or converge to vanishing or exploding weights if there are missing values in the timeseries. The value imputation technique 208 may be performed in one or more ways. First, a moving average calculation may be used to impute missing or NAN datapoints in the data. However, this calculation does not work well if large intervals of time are missing.”, also see ¶ 63, “it should be appreciated that additional or alternative metrics or techniques for testing the time series data are envisioned (e.g., mean absolute error (MAE), root mean squared error (RMSE), symmetric mean absolute percentage error (SMAPE), and/or others).”, also see ¶ 65).
Both Jiang and Rahimi pertain to the problem of machine learning using time series data, thus being analogous. It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine Jiang and Rahimi to teach the above limitations. The motivation for doing so would be “According to certain aspects, a set of time series data may be prepared and a plurality of features extracted therefrom. A feature vector based on the plurality of features may be generated and input into a classifier model to assess how well each of a plurality of available machine learning models is equipped to analyze the set of time series data and output a time series forecast. In embodiments, a stacking machine learning model may improve the time series forecast by accounting for multiple machine learning models as well as a set of covariates.” (see Rahimi abstract).
Regarding claim 8.
Jiang discloses the method of claim 1,
Jiang do not teach claim 8,
Rahimi teaches wherein the plurality of machine learning models correspond to one or more machine learning model types, wherein the one or more machine learning model types include one or more of: autoregressive (AR) models; moving average (MA) models; autoregressive moving average (ARMA) models; autoregressive integrated moving average (ARIMA) models; seasonal autoregressive integrated moving average (SARIMA) models; autoregressive integrated moving average models with exogenous regressors (ARIMAX); seasonal autoregressive integrated moving average models with exogenous regressors (SARIMAX); and ETS models (see ¶ 55, “It should be appreciated that the model search space 216 may be configured in a modular manner which may enable for the inclusion or removal of one or more models in the model search space 216, for example manually by a user or automatically based on one or more characteristics. For instance, the Seasonal Autoregressive Integrated Moving Average (SARIMA) model is derived from the combination of using autoregressive (AR) and moving average (MA) features to predict data that can present both trends and seasonality. Generally, the SARIMA model accounts for various components including seasonality, trend, stationarity, and integration. Further, the Trigonometric, Box-cox transformation, AutoRegressive Moving Average, Trend components, and Seasonality (TBATS) model is used for complex time series that exhibit multiple seasonality and trend shifts.”).
Both Jiang and Rahimi pertain to the problem of machine learning using time series data, thus being analogous. It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine Jiang and Rahimi to teach the above limitations. The motivation for doing so would be “According to certain aspects, a set of time series data may be prepared and a plurality of features extracted therefrom. A feature vector based on the plurality of features may be generated and input into a classifier model to assess how well each of a plurality of available machine learning models is equipped to analyze the set of time series data and output a time series forecast. In embodiments, a stacking machine learning model may improve the time series forecast by accounting for multiple machine learning models as well as a set of covariates.” (see Rahimi abstract).
Regarding claim 14.
Jiang teaches a method performed by a computing node group comprising an
ensemble computing node and a plurality of base model computing nodes, the method comprising: obtaining, by the computing node group, a data set comprising time series
data (see page 14, “exponentially smoothed technical indicators and macroeconomic features are employed as input variables to forecast the movement of the three major stock price indices in U.S. namely S&P500, Dow 30 and Nasdaq, from the period of Jan 2003 to Apr 2019. Four tree-based ensemble models, called RF, ERT, XGBoost and LightGBM, and four deep learning models, called RNN, BRNN, LSTM and GRU, are stacked as base classifiers in the first layer. Cross-validation method is then deployed to iteratively collect the prediction probabilities of each base learner and generate the input for the next layer classifier to prevent overfitting. In the second layer, logistic regression, as well as its variation which utilize a Lasso penalty term in the loss function, namely Meta-LR and Meta-LassoLR, respectively, are applied as meta-classifier to identify the unique learning patterns of the base learners.”, also see figure 5 on page 9, input block contains time series ,
PNG
media_image1.png
44
364
media_image1.png
Greyscale
, and table 3 on page 10, input
PNG
media_image2.png
21
932
media_image2.png
Greyscale
, also see pagr 8, “RNN and BRNN can extract the information from the time series data of arbitrary length.”);
partitioning, by the computing node group, the data set into a plurality of
segments (see page 9, section 4.3, “The first step of our improved Stacking algorithm is to gather each base classifiers’ output into a new set of data. According with the concept of cross-validation method, the training dataset is firstly split into k folds. During the k successive rounds, k − 1 folds are utilized to form the first-level classifiers and then applied to the remaining 1 validation subset that is not utilized for training in each iteration. The resulting prediction probabilities, along with the true instances’ classes, are then merged and provided as input data to the level-2 classifier. In the second step, a specific learning algorithm is employed as meta-classifier to extract the information and combine the learning patterns from the base classifiers. The framework and pseudo codes of Stacking algorithm are illustrated in Fig. 5 and Table 3, respectively.”, also table 3,
PNG
media_image3.png
50
932
media_image3.png
Greyscale
);
for each segment in the plurality of segments, creating, by the computing
node group, a plurality of segment groups, each segment group comprising a time series training data set and a time series test data set, thereby producing a plurality of time series training data sets and a plurality of time series test data sets (see page 9, section 4.3, “The first step of our improved Stacking algorithm is to gather each base classifiers’ output into a new set of data. According with the concept of cross-validation method, the training dataset is firstly split into k folds. During the k successive rounds, k − 1 folds are utilized to form the first-level classifiers and then applied to the remaining 1 validation subset that is not utilized for training in each iteration. The resulting prediction probabilities, along with the true instances’ classes, are then merged and provided as input data to the level-2 classifier. In the second step, a specific learning algorithm is employed as meta-classifier to extract the information and combine the learning patterns from the base classifiers. The framework and pseudo codes of Stacking algorithm are illustrated in Fig. 5 and Table 3, respectively.”, also see figure 5, training folds and validation folds for training and testing data set across kth iteration, also see page 3, section 3.1);
training, by each base model computing node, at least one respective machine learning model of a plurality of machine learning models using the plurality of time series training data sets, thereby producing a plurality of trained machine learning models corresponding to the plurality of base model computing nodes (see page 14, “exponentially smoothed technical indicators and macroeconomic features are employed as input variables to forecast the movement of the three major stock price indices in U.S. namely S&P500, Dow 30 and Nasdaq, from the period of Jan 2003 to Apr 2019. Four tree-based ensemble models, called RF, ERT, XGBoost and LightGBM, and four deep learning models, called RNN, BRNN, LSTM and GRU, are stacked as base classifiers in the first layer. Cross-validation method is then deployed to iteratively collect the prediction probabilities of each base learner and generate the input for the next layer classifier to prevent overfitting. In the second layer, logistic regression, as well as its variation which utilize a Lasso penalty term in the loss function, namely Meta-LR and Meta-LassoLR, respectively, are applied as meta-classifier to identify the unique learning patterns of the base learners.”, also see table 3, steps 4-6 for example);
determining, by each base model computing node, using at least one respective trained machine learning model, a plurality of time series forecast data sets that correspond to a respective plurality of time series test data sets (see page 14, “exponentially smoothed technical indicators and macroeconomic features are employed as input variables to forecast the movement of the three major stock price indices in U.S. namely S&P500, Dow 30 and Nasdaq, from the period of Jan 2003 to Apr 2019. Four tree-based ensemble models, called RF, ERT, XGBoost and LightGBM, and four deep learning models, called RNN, BRNN, LSTM and GRU, are stacked as base classifiers in the first layer. Cross-validation method is then deployed to iteratively collect the prediction probabilities of each base learner and generate the input for the next layer classifier to prevent overfitting. In the second layer, logistic regression, as well as its variation which utilize a Lasso penalty term in the loss function, namely Meta-LR and Meta-LassoLR, respectively, are applied as meta-classifier to identify the unique learning patterns of the base learners.”, also see table 3, steps 10 for example);
stacking according to time, by the computing node group, a respective plurality of time series forecast data sets for each trained machine learning model, thereby creating a plurality of stacked time series forecast data sets corresponding to the plurality of trained machine learning models (see page 9, section 4.3, “The first step of our improved Stacking algorithm is to gather each base classifiers’ output into a new set of data. According with the concept of cross-validation method, the training dataset is firstly split into k folds. During the k successive rounds, k − 1 folds are utilized to form the first-level classifiers and then applied to the remaining 1 validation subset that is not utilized for training in each iteration. The resulting prediction probabilities, along with the true instances’ classes, are then merged and provided as input data to the level-2 classifier. In the second step, a specific learning algorithm is employed as meta-classifier to extract the information and combine the learning patterns from the base classifiers. The framework and pseudo codes of Stacking algorithm are illustrated in Fig. 5 and Table 3, respectively.”, also see figure 5, output vectors from each iteration combined/stacked sequentially into all level 1 prediction probability);
and training, by the ensemble computing node, an ensemble machine learning model to generate a combined forecast, the ensemble machine learning model trained using the plurality of stacked time series forecast data sets and the time series data (see page 14, “exponentially smoothed technical indicators and macroeconomic features are employed as input variables to forecast the movement of the three major stock price indices in U.S. namely S&P500, Dow 30 and Nasdaq, from the period of Jan 2003 to Apr 2019. Four tree-based ensemble models, called RF, ERT, XGBoost and LightGBM, and four deep learning models, called RNN, BRNN, LSTM and GRU, are stacked as base classifiers in the first layer. Cross-validation method is then deployed to iteratively collect the prediction probabilities of each base learner and generate the input for the next layer classifier to prevent overfitting. In the second layer, logistic regression, as well as its variation which utilize a Lasso penalty term in the loss function, namely Meta-LR and Meta-LassoLR, respectively, are applied as meta-classifier to identify the unique learning patterns of the base learners.”, also see table 3, steps 13-19 for example and figure 5, combining level 1 and level 2 into final prediction).
Jiang do not teach distributing, by the computing node group, the plurality of time series training data sets to the plurality of base model computing nodes.
Rahimi teaches distributing, by the computing node group, the plurality of time series training data sets to the plurality of base model computing nodes (see ¶ 31, “single server computer 115 in FIG. 1A, it should be appreciated that the server computer 115 may be in the form of a distributed cluster of computers, servers, machines, cloud-based services, or the like. In this implementation, the entity may utilize the distributed server computer(s) 115 as part of an on-demand cloud computing platform. Accordingly, when the electronic devices 101, 102, 103 interface with the server computer 115, the electronic devices 101, 102, 103 may actually interface with one or more of a number of distributed computers, servers, machines, or the like, to facilitate the described functionalities.”, also see ¶ 66, “using the trained AutoML model 220 of FIG. 2. In particular, the systems and methods may use the feature vector extracted from the time series data (401; as described with respect to FIG. 2) as inputs, the vector of results from the different models as one-hot class labels, and an automation model 402 that would receive the inputs and classify them to match the correct vector label distribution (403). According to embodiments, the vector label distribution 403 represents the models in order based on performance.”, also see ¶ 106-107).
Both Jiang and Rahimi pertain to the problem of machine learning using time series data, thus being analogous. It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine Jiang and Rahimi to teach the above limitations. The motivation for doing so would be “According to certain aspects, a set of time series data may be prepared and a plurality of features extracted therefrom. A feature vector based on the plurality of features may be generated and input into a classifier model to assess how well each of a plurality of available machine learning models is equipped to analyze the set of time series data and output a time series forecast. In embodiments, a stacking machine learning model may improve the time series forecast by accounting for multiple machine learning models as well as a set of covariates.” (see Rahimi abstract).
Regarding claim 15.
Jiang and Rahimi teaches the method of claim 14,
Jiang further teaches further comprising: receiving, by the computing node group, a request from a requestor to generate a requested time series forecast data set corresponding to a request data set; obtaining, by the computing node group, the request data set; generating a plurality of requested time series forecast data sets; generating, by the ensemble computing node, the requested time series forecast data set using the ensemble machine learning model and the plurality of requested time series forecast data sets; and providing, by the computing node group, to the requestor, the requested time series forecast data set (see page 9, section 4.3, “The first step of our improved Stacking algorithm is to gather each base classifiers’ output into a new set of data. According with the concept of cross-validation method, the training dataset is firstly split into k folds. During the k successive rounds, k − 1 folds are utilized to form the first-level classifiers and then applied to the remaining 1 validation subset that is not utilized for training in each iteration. The resulting prediction probabilities, along with the true instances’ classes, are then merged and provided as input data to the level-2 classifier. In the second step, a specific learning algorithm is employed as meta-classifier to extract the information and combine the learning patterns from the base classifiers. The framework and pseudo codes of Stacking algorithm are illustrated in Fig. 5 and Table 3, respectively.”, also see figure 5, training folds and validation folds for training and testing data set across kth iteration, also see table 3).
Rahimi teaches distributing, by the computing node group, the request data set to the plurality of base model computing nodes (see ¶ 31, “single server computer 115 in FIG. 1A, it should be appreciated that the server computer 115 may be in the form of a distributed cluster of computers, servers, machines, cloud-based services, or the like. In this implementation, the entity may utilize the distributed server computer(s) 115 as part of an on-demand cloud computing platform. Accordingly, when the electronic devices 101, 102, 103 interface with the server computer 115, the electronic devices 101, 102, 103 may actually interface with one or more of a number of distributed computers, servers, machines, or the like, to facilitate the described functionalities.”, also see ¶ 66, “using the trained AutoML model 220 of FIG. 2. In particular, the systems and methods may use the feature vector extracted from the time series data (401; as described with respect to FIG. 2) as inputs, the vector of results from the different models as one-hot class labels, and an automation model 402 that would receive the inputs and classify them to match the correct vector label distribution (403). According to embodiments, the vector label distribution 403 represents the models in order based on performance.”, also see ¶ 106-107).
The motivation utilized in the combination of claim 14, super, applies equally as well to claim 15.
Regarding claim 16.
Jiang and Rahimi teaches the method of claim 14,
Rahimi further teaches wherein the computing node group obtains the data set comprising time series data from a coordinator computer, wherein a distributed computing system comprises the coordinator computer and the computing node group (see ¶ 31, “single server computer 115 in FIG. 1A, it should be appreciated that the server computer 115 may be in the form of a distributed cluster of computers, servers, machines, cloud-based services, or the like. In this implementation, the entity may utilize the distributed server computer(s) 115 as part of an on-demand cloud computing platform. Accordingly, when the electronic devices 101, 102, 103 interface with the server computer 115, the electronic devices 101, 102, 103 may actually interface with one or more of a number of distributed computers, servers, machines, or the like, to facilitate the described functionalities.”, also see ¶ 66, “using the trained AutoML model 220 of FIG. 2. In particular, the systems and methods may use the feature vector extracted from the time series data (401; as described with respect to FIG. 2) as inputs, the vector of results from the different models as one-hot class labels, and an automation model 402 that would receive the inputs and classify them to match the correct vector label distribution (403). According to embodiments, the vector label distribution 403 represents the models in order based on performance.”, also see ¶ 106-107).
The motivation utilized in the combination of claim 14, super, applies equally as well to claim 16.
Allowable Subject Matter
Claims 17-19 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Related prior arts:
DUONG et al. (US 20240220788 A1) teaches dynamic supervised learning (DSL) and dynamic neural distribution function (DNDF) machine learning architectures and platforms. In contrast to existing ML approaches, DNDF accommodates a whole data structure via a neural network distribution function from which a decision boundary is born out. In particular, a neural network learning algorithm is used to extract a decision boundary while a neural distribution function is a neural data distribution approach wherein one or more decision boundaries are extracted among various distributions.
Quanz et al. (US 20220147669 A1) teaches computational efficiency of a computing platform in processing a time series data includes receiving the time series data and grouping it into a hierarchy of partitions of related time series. The hierarchy has different partition levels. A computation capability of a computing platform is determined. A partition level, from the different partition levels, is selected based on the determined computation capability. One or more modeling tasks are defined, each modeling task including a group of time series of the plurality of time series, based on the selected partition level. One or more modeling tasks are executed in parallel on the computing platform by, for each modeling task, training a model using all the time series in the group of time series of the corresponding modeling task.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to IMAD M KASSIM whose telephone number is (571)272-2958. The examiner can normally be reached 10:30AM-5:30PM, M-F (E.S.T.).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael J. Huntley can be reached at (303) 297 - 4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/IMAD KASSIM/Primary Examiner, Art Unit 2129
