DETAILED ACTION
This Office Action is in response to Applicant's Communication received on 10/06/2025 for application number 17/812,282.
Claims 1-5 and 7-20 are presented for examination. Claims 1, 8, and 18 are independent claims.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The amendment filed on 10/06/2025 has been entered.
Claims 1, 8, and 18 have been amended. Claims 6 is canceled. Claims 1-5 and 7-20 are pending in the application.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2 and 4-5 are rejected under 35 U.S.C. 103 as being unpatentable over Yocum et al. (US 2021/0271925 A1 hereinafter Yocum) in view of McCune et al. (US 2019/0383512 A1 hereinafter McCune).
Regarding Claim 1, Yocum teaches a method for selecting a forecasting algorithm for a class ([0023] for a given schedule of individual days/ points in time for which predictions are to be made, the system may select the best model by forecast group, prediction horizon, and model class for a particular day) comprising:
receiving time series data by a computing device, wherein the time series data comprises a plurality of time intervals and each time interval is associated with an interval value ([0013] models are trained to predict time series data, which may include the average handle time and call volume; [0015] the call volume identifies the number of phone calls that are expected to be received during a specified time interval; [0023] for a given schedule of individual days/ points in time for which predictions are to be made, the system may select the best model by forecast group, prediction horizon, and model class for a particular day; predictions are then made with a set of models to generate a piecewise forecast using the best model for a particular day; [0032] the repository (150) is a computing system that may include multiple computing devices in accordance with the computing system (500) and the nodes (522) and (524); the data in the repository (150) include multiple variables, parameters, historical time series data (i.e., time series data comprising plurality of time intervals), and forecasts that are generated (i.e., interval value) and used by the components of the system - thus, comprising plurality of time intervals and each time interval associated with forecasts/ interval value; [0075] the nodes may include functionality to receive requests from the client device and transmit responses to the client device; the client device may be a computing system; [0076] the computing system or group of computing systems include functionality to perform a variety of operations disclosed);
receiving a plurality of forecasting algorithms by the computing device ([0037] the process generates predictions used to generate estimated headcounts from models with multiple horizons and classes; in Step 202, models are obtained corresponding to model horizons; [0043] multiple models may be trained having multiple model horizons and classes; a model discovery service may have a model space of different models that are trained; the model space may include time series models that have different classes, different lags, and be of different types; the different classes of models may include in-season models, out-of-season models, and all-season models; [0076] the computing system or group of computing systems include functionality to perform a variety of operations disclosed);
creating a set of classes by the computing device, wherein each class in the set of classes is associated with a plurality of subsequences of the time series data, and wherein each of the plurality of subsequences comprises a time interval of the plurality of time intervals ([0020] a model class identifies, for a model, the class of the model; the classes may relate to seasonality, which is how the model performs with respect to a particular period of time; example classes may include “all-season”, “in-season”, and “out-of-season” (i.e., set of classes); different classes may be trained and scored differently during training to identify the best performing model - thus, the set of classes trained/ created for the model; [0023] for a given schedule of individual days/ points in time for which predictions are to be made, the system may select the best model by forecast group, prediction horizon, and model class for a particular day; [0037] the models may include lags from target time series; a target time series is a time series for which a model is used to generate predictions; [0039] as a part of generating the values and for each day to be predicted, the model having the model horizon with a lowest value that is greater than or equal to an ordinal value of the day to be predicted may be selected for that day to generate a piecewise forecast for the time series variable being predicted - thus, each class/ particular period of time is associated with plurality of subsequences of the time series data/ individual days comprising plurality of time intervals; [0076] the computing system or group of computing systems include functionality to perform a variety of operations disclosed);
for each class of the set of classes, selecting a forecasting algorithm from the plurality of forecasting algorithms based on the subsequences of the time series data associated with the class by the computing device ([0020] a model class identifies, for a model, the class of the model; the classes may relate to seasonality, which is how the model performs with respect to a particular period of time; example classes may include “all-season”, “in-season”, and “out-of-season”; [0023] for a given schedule of individual days/ points in time for which predictions are to be made, the system may select the best model by forecast group, prediction horizon, and model class for a particular day; [0025] select the best scoring models according to the future time range to forecast; [0039] as a part of generating the values and for each day to be predicted, the model having the model horizon with a lowest value that is greater than or equal to an ordinal value of the day to be predicted may be selected for that day to generate a piecewise forecast for the time series variable being predicted);
receiving a request to forecast the interval value for a future time interval by the computing device ([0025] select the best scoring models according to the future time range to forecast, and provide this selection of models to the demand prediction service; [0058] FIG. 4 illustrates how the piecewise model selection displayed on the graphical user interface (400); [0063] the row (410) displays the date for the predicted days; the dates range from January 5 to January 13 (i.e., future time interval); [0064] the row (412) displays a piecewise prediction forecast generated based on multiple models; [0037] the process generates predictions used to generate estimated headcounts (i.e., forecast interval value) from models with multiple horizons and classes);
determining a class of the set of classes that is associated with the future time interval by the computing device ([0020] a model class identifies, for a model, the class of the model; the classes may relate to seasonality, which is how the model performs with respect to a particular period of time; example classes may include “all-season”, “in-season”, and “out-of-season”; [0023] for a given schedule of individual days/ points in time for which predictions are to be made, the system may select the best model by forecast group, prediction horizon, and model class for a particular day; [0025] select the best scoring models according to the future time range to forecast; [0051] for each of the identified dates, the class that applies to that date is identified; the classes may include in-season, out-of-season, and all-season classes, and one of the classes may be assigned to each date for which a prediction will be generated; [0058] FIG. 4 illustrates how the piecewise model selection displayed on the graphical user interface (400); [0060] the row (404) displays the classes of the models used to generate the predictions of the row (412); [0063] the row (410) displays the date for the predicted days; the dates range from January 5 to January 13 (i.e., future time interval) - thus, determining a class associated with the future time interval);
using the forecasting algorithm selected for the determined class to predict the interval value for the future time interval by the computing device ([0023] for a given schedule of individual days/ points in time for which predictions are to be made, the system may select the best model by forecast group, prediction horizon, and model class for a particular day; [0025] select the best scoring models according to the future time range to forecast; [0037] the process generates predictions used to generate estimated headcounts (i.e., forecast interval value) from models with multiple horizons and classes; [0058] FIG. 4 illustrates how the piecewise model selection displayed on the graphical user interface (400); [0064] the row (412) displays a piecewise prediction forecast generated based on multiple models; the model “T(1,A)” is used to generate the predicted value of “8” for the predicted day “0” that corresponds to the date of January 5 using a horizon of 1 and a class of A;[0042] the estimated headcount generated using the Erlang-C traffic modeling formula with the values from the piecewise forecast and displayed on a client computing device); and
providing the predicted interval value for the future time interval by the computing device ([0037] the process generates predictions used to generate estimated headcounts (i.e., forecast interval value) from models with multiple horizons and classes; [0058] FIG. 4 illustrates how the piecewise model selection displayed on the graphical user interface (400); [0064] the row (412) displays a piecewise prediction forecast generated based on multiple models; the model “T(1,A)” is used to generate the predicted value of “8” for the predicted day “0” that corresponds to the date of January 5 using a horizon of 1 and a class of A; [0042] the estimated headcount generated using the Erlang-C traffic modeling formula with the values from the piecewise forecast and displayed on a client computing device).
However, Yocum fails to expressly teach wherein creating each class comprises: displaying some or all of the plurality of time intervals in a user interface by the computing device; receiving a selection of the plurality of time intervals associated with the class from among the displayed some or all of the plurality of time intervals through the user interface by the computing device; and creating the class from the selection of the plurality of time intervals associated with the class by the computing device.
In the same field of endeavor, McCune teaches wherein creating each class comprises: displaying some or all of the plurality of time intervals in a user interface by the computing device ([0062] the time period settings 304 enable selection of the time periods that are associated with or that define each corresponding season, such as via month icons 314; the interface 300 illustrated in FIG. 7 correspond to a “Spring” season, as indicated by the selection of the month icons 314 for March, April, and May; a user may select different months to adjust the time period of the season via tapping or otherwise actuating the corresponding month to select and unselect the months via the interface 300; although FIG. 7 illustrates that the seasons are defined by particular months, additional embodiments may include other time periods or designations, such as days, dates, years, and so forth, to define each season); receiving a selection of the plurality of time intervals associated with the class from among the displayed some or all of the plurality of time intervals through the user interface by the computing device ([0062] the time period settings 304 enable selection of the time periods that are associated with or that define each corresponding season, such as via month icons 314; the interface 300 illustrated in FIG. 7 correspond to a “Spring” season, as indicated by the selection of the month icons 314 for March, April, and May; a user may select different months to adjust the time period of the season via tapping or otherwise actuating the corresponding month to select and unselect the months via the interface 300; although FIG. 7 illustrates that the seasons are defined by particular months, additional embodiments may include other time periods or designations, such as days, dates, years, and so forth, to define each season (i.e., class)); and creating the class from the selection of the plurality of time intervals associated with the class by the computing device ([0062] the time period settings 304 enable selection of the time periods that are associated with or that define each corresponding season, such as via month icons 314; a user may select different months to adjust the time period of the season via tapping or otherwise actuating the corresponding month to select and unselect the months via the interface 300; although FIG. 7 illustrates that the seasons are defined by particular months, additional embodiments may include other time periods or designations, such as days, dates, years, and so forth, to define each season (i.e., creating class)).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated wherein creating each class comprises: displaying some or all of the plurality of time intervals in a user interface by the computing device; receiving a selection of the plurality of time intervals associated with the class from among the displayed some or all of the plurality of time intervals through the user interface by the computing device; and creating the class from the selection of the plurality of time intervals associated with the class by the computing device, as taught by McCune into Yocum. Doing so would be desirable because it would enable user customization and control by users (McCune [0062]), thereby enhancing user experience.
As to dependent Claim 2, Yocum and McCune teach all the limitations of Claim 1. Yocum further teaches wherein the predicted interval value is one of a communication volume, an average handling time, or a shrinkage ([0037] the process generates predictions used to generate estimated headcounts (i.e., forecast interval value) from models with multiple horizons and classes; [0017] the headcount identifies how many representatives are needed to be able to handle the expected amount of calls).
As to dependent Claim 4, Yocum and McCune teach all the limitations of Claim 1. Yocum further teaches wherein selecting the forecasting algorithm from the plurality of forecasting algorithms based on the subsequences of the time series data associated with the class comprises selecting the forecasting algorithm with a minimum associated forecast error when predicting the interval value for time intervals from the plurality of subsequences of the time series data associated with the class ([0045] the training data used to train the models may include historical time series data for at least two years; [0046] cross validation metrics are generated; different classes of models may use different data for testing and generating cross validation metrics; in-season models may be tested against in-season data, out-of-season model may be tested against out-of-season data, and all-season models may be tested against all-season data (i.e., models based on the subsequences of the time series data associated with the class/ days associated with the season); the cross validation metrics may include a mean absolute scaled error (MASE), a weighted mean absolute percentage error (wMAPE) for the models, mean absolute percentage error (MAPE), root mean square error (RMSE), explained variance (R2), and mean absolute error (MAE); [0047] models are selected based on cross validation metrics; the model discovery service may select the model with the lowest mean absolute scaled error or the lowest weighted mean absolute percentage error (i.e., minimum associated error)).
As to dependent Claim 5, Yocum and McCune teach all the limitations of Claim 1. Yocum further teaches wherein each class of the set of classes is one of a user class or a subsequence class ([0023] for a given schedule of individual days/ points in time for which predictions are to be made, the system may select the best model by forecast group, prediction horizon, and model class for a particular day; [0020] a model class identifies, for a model, the class of the model; the classes may relate to seasonality, which is how the model performs with respect to a particular period of time, such as, Summer, Fall, tax season, etc.; example classes may include “all-season”, “in-season”, and “out-of-season” (equivalent to user class/ subsequence class)).
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Yocum in view of McCune, further in view of Boughton et al. (US 2008/0037760 A1 hereinafter Boughton).
As to dependent Claim 3, Yocum and McCune teach all the limitations of Claim 1. However, Yocum and McCune fail to expressly teach wherein one or more of scheduling one or more workers to work during the future time interval based on the predicted interval value and generating a hiring plan for the future time interval.
In the same field of endeavor, Boughton teaches wherein one or more of scheduling one or more workers to work during the future time interval based on the predicted interval value and generating a hiring plan for the future time interval ([0025] the estimation tool 10 uses the historical call volume data 12 to predict future call volume; [0030] based on the historical call volume data as provided in the weekly and/or daily historical call volume charts, the schedule of the agents servicing the call center, and the average handling time of calls to the call center by the agents, the estimation tool generates a staffing impact chart for a future period of time; The staffing impact chart provides, for each time interval , a positive/negative number representative of the estimated staffing need during that time interval (i.e., predicted interval value); [0034] the estimation tool can be used to adjust the schedules of agent(s) for proper call coverage in the call center; agents can also be selectively added for additional call coverage).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated wherein one or more of scheduling one or more workers to work during the future time interval based on the predicted interval value and generating a hiring plan for the future time interval, as taught by Boughton into Yocum and McCune. Doing so would be desirable because it allows a user to make staffing decisions proactively based on both historical and real-time data (Boughton [0003]).
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Yocum in view of McCune, further in view of Saleh et al. (US 2023/0252267 A1 hereinafter Saleh).
As to dependent Claim 7, Yocum and McCune teach all the limitations of Claim 1. However, Yocum and McCune fail to expressly teach wherein receiving a set of external data values by the computing device, wherein each external data value in the set of external data values is associated with a time interval of the plurality of time intervals; and for at least one class in the set of classes, selecting the plurality of subsequences of the time series data for the at least one class based on the set of external data values.
In the same field of endeavor, Saleh teaches wherein receiving a set of external data values by the computing device, wherein each external data value in the set of external data values is associated with a time interval of the plurality of time intervals ([0013] different data records may also include different feature data at different points in time or timesteps, such as data occurring on a particular day of week, a selected time period or time in the past, in a particular month or other time; training data may be used for these features, as well as additional external features that may be specifically selected for a time series forecasting task; external features may include those associated with additional customer data, fraud data, transaction data, a macro-economical feature, a trend in an e-commerce industry, a pandemic effect feature, a total payment volume migration feature, or a combination thereof; these features may provide additional time-based data that may have a temporal nature and affect predictive forecasting at future timesteps/times - thus, receiving a set of external data value/ external feature data associated with time interval); and for at least one class in the set of classes, selecting the plurality of subsequences of the time series data for the at least one class based on the set of external data values ([0012] utilize a trained DNN model, such as one using an LSTM architecture, that includes an attention mechanism to focus on particular past timesteps and/or time-based/temporal data (i.e., time series data) when training the DNN to predict future timesteps and corresponding forecasts; [0013] different data records may also include different feature data at different points in time or timesteps, such as data occurring on a particular day of week, a selected time period or time in the past, in a particular month or other time (i.e., subsequences of the time series data for a class); training data may be used for these features, as well as additional external features that may be specifically selected for a time series forecasting task; external features may include those associated with additional customer data, fraud data, transaction data, a macro-economical feature, a trend in an e-commerce industry, a pandemic effect feature, a total payment volume migration feature, etc.; these features may provide additional time-based data that may have a temporal nature and affect predictive forecasting at future timesteps/times; [0021] when training the DNN model for predictive forecasts of a future traits, features, or variables, additional external features utilized to provide additional accuracy, temporal relevance, and/or feature dependence; external feature may therefore be used to provide additional context or factors that may assist in explaining past trends and predicting future forecasts for a particular variable; external features correspond to segments of temporal data that may affect the main input attribute or feature - thus, selecting the subsequences of the time series data/ temporal data for particular points in time/ class corresponding to the external data values/external feature data for training and predicting forecasts).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated wherein receiving a set of external data values by the computing device, wherein each external data value in the set of external data values is associated with a time interval of the plurality of time intervals; and for at least one class in the set of classes, selecting the plurality of subsequences of the time series data for the at least one class based on the set of external data values, as taught by Saleh into Yocum and McCune. Doing so would be desirable because using external features would provide additional context or factors that may assist in explaining past trends and predicting future forecasts for a particular variable and provide additional accuracy, temporal relevance, and/or feature dependence (Saleh [0021]).
Claims 8-9, 11, and 14-17 are rejected under 35 U.S.C. 103 as being unpatentable over Yocum in view of McCune, further in view of Mitra et al. (US 2023/0075453 A1 hereinafter Mitra).
Regarding Claim 8, Yocum teaches a method ([0023] for a given schedule of individual days/ points in time for which predictions are to be made, the system may select the best model by forecast group, prediction horizon, and model class for a particular day) comprising:
receiving time series data by a computing device, wherein the time series data comprises a plurality of time intervals and each time interval is associated with an interval value ([0013] models are trained to predict time series data, which may include the average handle time and call volume; [0015] the call volume identifies the number of phone calls that are expected to be received during a specified time interval; [0023] for a given schedule of individual days/ points in time for which predictions are to be made, the system may select the best model by forecast group, prediction horizon, and model class for a particular day; predictions are then made with a set of models to generate a piecewise forecast using the best model for a particular day; [0032] the repository (150) is a computing system that may include multiple computing devices in accordance with the computing system (500) and the nodes (522) and (524); the data in the repository (150) include multiple variables, parameters, historical time series data (i.e., time series data comprising plurality of time intervals), and forecasts that are generated (i.e., interval value) and used by the components of the system - thus, comprising plurality of time intervals and each time interval associated with forecasts/ interval value; [0075] the nodes may include functionality to receive requests from the client device and transmit responses to the client device; the client device may be a computing system; [0076] the computing system or group of computing systems include functionality to perform a variety of operations disclosed);
receiving a plurality of forecasting algorithms by the computing device ([0037] the process generates predictions used to generate estimated headcounts from models with multiple horizons and classes; in Step 202, models are obtained corresponding to model horizons; [0043] multiple models may be trained having multiple model horizons and classes; a model discovery service may have a model space of different models that are trained; the model space may include time series models that have different classes, different lags, and be of different types; the different classes of models may include in-season models, out-of-season models, and all-season models; [0076] the computing system or group of computing systems include functionality to perform a variety of operations disclosed);
creating a set of classes by the computing device, wherein each class in the set of classes is associated with a plurality of subsequences of the time series data, wherein each of the plurality of subsequences comprises a time interval of the plurality of time intervals ([0020] a model class identifies, for a model, the class of the model; the classes may relate to seasonality, which is how the model performs with respect to a particular period of time; example classes may include “all-season”, “in-season”, and “out-of-season” (i.e., set of classes); different classes may be trained and scored differently during training to identify the best performing model - thus, the set of classes trained/ created for the model; [0023] for a given schedule of individual days/ points in time for which predictions are to be made, the system may select the best model by forecast group, prediction horizon, and model class for a particular day; [0037] the models may include lags from target time series; a target time series is a time series for which a model is used to generate predictions; [0039] as a part of generating the values and for each day to be predicted, the model having the model horizon with a lowest value that is greater than or equal to an ordinal value of the day to be predicted may be selected for that day to generate a piecewise forecast for the time series variable being predicted - thus, each class/ particular period of time is associated with plurality of subsequences of the time series data/ individual days comprising plurality of time intervals; [0076] the computing system or group of computing systems include functionality to perform a variety of operations disclosed);
training each forecasting algorithm to predict the interval value using a portion of the time series data by the computing device ([0036] training models and predicting call center volume (i.e., interval value); [0038] the time series variable values are predictions generated with trained models; the values may correspond to the days to be predicted; [0045] the training data used to train the models may include historical time series data for at least two years; additional data beyond two years may be included and more or less data may be used; the initial training of the models may use a subset of data (i.e., portion of time series data), such as the trailing two years of data; then after the models are selected, the selected models may be trained with the additional training data that was not used prior to model selection); and
training a selection model by the computing device using the received time series data, the set of classes ([0025] the model discovery service (equivalent to selection model) is a set of programs executing on the server computing system as part of the server application; the model discovery service identifies and trains the models used to generate predictions of time series variable values that may be used as inputs to generate headcount predictions; the model discovery service may receive input data from the repository, train the models, score the models, select the best scoring models according to the future time range to forecast, and provide this selection of models to the demand prediction service; [0043] multiple models may be trained having multiple model horizons and classes; a model discovery service may have a model space of different models that are trained; the model space may include time series models that have different classes, different lags, and be of different types; the different classes of models may include in-season models, out-of-season models, and all-season models; [0032] the repository is a computing system that may include multiple computing devices in accordance with the computing system and the nodes; the data in the repository include multiple variables, parameters, historical time series data (i.e., time series data), and forecasts that are generated and used by the components of the system - thus, the model discovery service including model space of different models is trained using time series data and classes/ particular periods of time);
However, Yocum fails to expressly teach wherein creating each class comprises: displaying some or all of the plurality of time intervals in a user interface by the computing device; receiving a selection of the plurality of time intervals associated with the class from among the displayed some or all of the plurality of time intervals through the user interface by the computing device; and creating the class from the selection of the plurality of time intervals associated with the class by the computing device.
In the same field of endeavor, McCune teaches wherein creating each class comprises: displaying some or all of the plurality of time intervals in a user interface by the computing device ([0062] the time period settings 304 enable selection of the time periods that are associated with or that define each corresponding season, such as via month icons 314; the interface 300 illustrated in FIG. 7 correspond to a “Spring” season, as indicated by the selection of the month icons 314 for March, April, and May; a user may select different months to adjust the time period of the season via tapping or otherwise actuating the corresponding month to select and unselect the months via the interface 300; although FIG. 7 illustrates that the seasons are defined by particular months, additional embodiments may include other time periods or designations, such as days, dates, years, and so forth, to define each season); receiving a selection of the plurality of time intervals associated with the class from among the displayed some or all of the plurality of time intervals through the user interface by the computing device ([0062] the time period settings 304 enable selection of the time periods that are associated with or that define each corresponding season, such as via month icons 314; the interface 300 illustrated in FIG. 7 correspond to a “Spring” season, as indicated by the selection of the month icons 314 for March, April, and May; a user may select different months to adjust the time period of the season via tapping or otherwise actuating the corresponding month to select and unselect the months via the interface 300; although FIG. 7 illustrates that the seasons are defined by particular months, additional embodiments may include other time periods or designations, such as days, dates, years, and so forth, to define each season (i.e., class)); and creating the class from the selection of the plurality of time intervals associated with the class by the computing device ([0062] the time period settings 304 enable selection of the time periods that are associated with or that define each corresponding season, such as via month icons 314; a user may select different months to adjust the time period of the season via tapping or otherwise actuating the corresponding month to select and unselect the months via the interface 300; although FIG. 7 illustrates that the seasons are defined by particular months, additional embodiments may include other time periods or designations, such as days, dates, years, and so forth, to define each season (i.e., creating class)).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated wherein creating each class comprises: displaying some or all of the plurality of time intervals in a user interface by the computing device; receiving a selection of the plurality of time intervals associated with the class from among the displayed some or all of the plurality of time intervals through the user interface by the computing device; and creating the class from the selection of the plurality of time intervals associated with the class by the computing device, as taught by McCune into Yocum. Doing so would be desirable because it would enable user customization and control by users (McCune [0062]), thereby enhancing user experience.
However, Yocum and McCune fail to expressly teach wherein for each time interval of the plurality of time intervals of the time series data that is not in the portion: for each forecasting algorithm of the plurality of forecasting algorithms: predicting the interval value for the time interval using the forecasting algorithm by the computing device; and determining a difference between the interval value associated with the time interval in the time series data and the predicted interval value for the time interval by the computing device; and training a selection model by the computing device using the determined differences for each forecasting algorithm for each time interval of the plurality of time intervals of the time series data.
In the same field of endeavor, Mitra teaches wherein for each time interval of the plurality of time intervals of the time series data that is not in the portion: for each forecasting algorithm of the plurality of forecasting algorithms: predicting the interval value for the time interval using the forecasting algorithm by the computing device ([0008] the system performs forecasting using the machine learning based model by determining a training data set based on the time series data; the training dataset includes a training subset and a test subset; the system trains the machine learning based model using the training subset of the training dataset and evaluates the machine learning based model using the test subset of the training dataset; [0047] the training dataset D includes a training subset T1 and a test subset T2; the model generation module 130 trains 420 the machine learning based model using the training subset T1; the model generation module 130 evaluates 430 the machine learning based model using the test subset T2; [0057] the system trains all machine learning models on S11 and evaluates their performance on S1); and determining a difference between the interval value associated with the time interval in the time series data and the predicted interval value for the time interval by the computing device ([0036] a model metric represents a criterion for evaluating machine learning based models; a model metric represents a function or an expression used for determining a difference between data forecasted using a model with observed data (or labelled data)); and training a selection model by the computing device using the determined differences for each forecasting algorithm for each time interval of the plurality of time intervals of the time series data ([0057] the system trains all machine learning models on S11 and evaluates their performance on S12; based on the performance, the system selects top model from each pool; the top models from each pool (selected based on the evaluation in previous round) is trained on S21, and the system evaluates their performance on S22; based on the performance, the system selects the TOP model across all pools; [0058] the model generation module 130 (equivalent to selection model) trains 720 each machine learning based model from the pool P1 of machine learning based models using training subset S11; the model generation module 130 evaluates 730 each machine learning based model from the pool 610 of machine learning based models using the given model metric and using the test subset S12; [0060] the model generation module 130 evaluates 760 each machine learning based model from the pool P2 of machine learning based models using the given model metric and using training subset S22; [0061] the model generation module 130 selects the best model for forecasting time series data for the given application or based on a given model metric from the pool P2 of machine learning based models; [0036] a model metric represents a criterion for evaluating machine learning based models; a model metric represents a function or an expression used for determining a difference between data forecasted using a model with observed data (or labelled data) - thus, training using the determined differences for each model/ forecasting algorithm for each time interval of the plurality of time intervals of the time series data).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated wherein for each time interval of the plurality of time intervals of the time series data that is not in the portion: for each forecasting algorithm of the plurality of forecasting algorithms: predicting the interval value for the time interval using the forecasting algorithm by the computing device; and determining a difference between the interval value associated with the time interval in the time series data and the predicted interval value for the time interval by the computing device; and training a selection model by the computing device using the determined differences for each forecasting algorithm for each time interval of the plurality of time intervals of the time series data, as taught by Mitra into Yocum and McCune. Doing so would be desirable because it would enable the system to automatically select a model from different models based on previously known applications (Mitra [0070]).
As to dependent Claim 9, Yocum, McCune, and Mitra teach all the limitations of Claim 8. Yocum further teaches wherein receiving a request to forecast the interval value at a future time interval ([0025] select the best scoring models according to the future time range to forecast, and provide this selection of models to the demand prediction service; [0058] FIG. 4 illustrates how the piecewise model selection displayed on the graphical user interface (400); [0063] the row (410) displays the date for the predicted days; the dates range from January 5 to January 13 (i.e., future time interval); [0064] the row (412) displays a piecewise prediction forecast generated based on multiple models; [0037] the process generates predictions used to generate estimated headcounts (i.e., forecast interval value) from models with multiple horizons and classes); using the selection model to select a forecasting algorithm of the plurality of forecasting algorithms for the future time interval ([0023] for a given schedule of individual days/ points in time for which predictions are to be made, the system may select the best model by forecast group, prediction horizon, and model class for a particular day; [0025] the model discovery service may receive input data from the repository, train the models, score the models, select the best scoring models according to the future time range to forecast, and provide this selection of models to the demand prediction service; [0037] the process generates predictions used to generate estimated headcounts (i.e., forecast interval value) from models with multiple horizons and classes; [0058] FIG. 4 illustrates how the piecewise model selection displayed on the graphical user interface (400); [0064] the row (412) displays a piecewise prediction forecast generated based on multiple models; the model “T(1,A)” is used to generate the predicted value of “8” for the predicted day “0” that corresponds to the date of January 5 using a horizon of 1 and a class of A;[0042] the estimated headcount generated using the Erlang-C traffic modeling formula with the values from the piecewise forecast and displayed on a client computing device); using the selected forecasting algorithm to predict the interval value for the future time interval ([0023] for a given schedule of individual days/ points in time for which predictions are to be made, the system may select the best model by forecast group, prediction horizon, and model class for a particular day; [0025] select the best scoring models according to the future time range to forecast; [0037] the process generates predictions used to generate estimated headcounts (i.e., forecast interval value) from models with multiple horizons and classes; [0058] FIG. 4 illustrates how the piecewise model selection displayed on the graphical user interface (400); [0064] the row (412) displays a piecewise prediction forecast generated based on multiple models; the model “T(1,A)” is used to generate the predicted value of “8” for the predicted day “0” that corresponds to the date of January 5 using a horizon of 1 and a class of A;[0042] the estimated headcount generated using the Erlang-C traffic modeling formula with the values from the piecewise forecast and displayed on a client computing device); and providing the predicted interval value for the future time interval ([0037] the process generates predictions used to generate estimated headcounts (i.e., forecast interval value) from models with multiple horizons and classes; [0058] FIG. 4 illustrates how the piecewise model selection displayed on the graphical user interface (400); [0064] the row (412) displays a piecewise prediction forecast generated based on multiple models; the model “T(1,A)” is used to generate the predicted value of “8” for the predicted day “0” that corresponds to the date of January 5 using a horizon of 1 and a class of A; [0042] the estimated headcount generated using the Erlang-C traffic modeling formula with the values from the piecewise forecast and displayed on a client computing device).
As to dependent Claim 11, Yocum, McCune, and Mitra teach all the limitations of Claim 8. Yocum further teaches wherein the interval value is one of a communication volume, an average handling time, or a shrinkage ([0037] the process generates predictions used to generate estimated headcounts (i.e., forecast interval value) from models with multiple horizons and classes; [0017] the headcount identifies how many representatives are needed to be able to handle the expected amount of calls).
As to dependent Claim 14, Yocum, McCune, and Mitra teach all the limitations of Claim 8. Yocum further teaches wherein the selection model comprises a decision tree ([0025] the model discovery service (equivalent to selection model) is a set of programs executing on the server computing system as part of the server application; the model discovery service identifies and trains the models used to generate predictions of time series variable values that may be used as inputs to generate headcount predictions; the model discovery service may score the models, select the best scoring models according to the future time range to forecast - thus, comprising a decision tree).
As to dependent Claim 15, Yocum, McCune, and Mitra teach all the limitations of Claim 8. Yocum further teaches wherein for one or more classes of the set of classes, some or all of the subsequences of the plurality of subsequences of the time series data associated with the class are selected by an entity computing device ([0023] for a given schedule of individual days/ points in time for which predictions are to be made, the system may select the best model by forecast group, prediction horizon, and model class for a particular day; [0025]the model discovery service identifies and trains the models used to generate predictions of time series variable values that may be used as inputs to generate headcount predictions; the model discovery service may receive input data from the repository, train the models, score the models, select the best scoring models according to the future time range to forecast, and provide this selection of models to the demand prediction service; [0039] as a part of generating the values and for each day to be predicted, the model having the model horizon with a lowest value that is greater than or equal to an ordinal value of the day to be predicted may be selected for that day to generate a piecewise forecast for the time series variable being predicted (i.e., each class/ schedule is associated with plurality of subsequences of the time series data/ individual days comprising plurality of time intervals); [0043] multiple models may be trained having multiple model horizons and classes (i.e., set of classes); a model discovery service may have a model space of different models that are trained; the model space may include time series models that have different classes, different lags, and be of different types; the different classes of models may include in-season models, out-of-season models, and all-season models; [0037] the models may include lags from target time series; a target time series is a time series for which a model is used to generate predictions - thus, the subsequences of the time series data associated with the class of the model are selected by the system including model discovery service/ entity computing device).
As to dependent Claim 16, Yocum, McCune, and Mitra teach all the limitations of Claim 8. Yocum further teaches wherein the classes in the set of classes comprise one or more of user classes or subsequence classes ([0023] for a given schedule of individual days/ points in time for which predictions are to be made, the system may select the best model by forecast group, prediction horizon, and model class for a particular day; [0020] a model class identifies, for a model, the class of the model; the classes may relate to seasonality, which is how the model performs with respect to a particular period of time, such as, Summer, Fall, tax season, etc.; example classes may include “all-season”, “in-season”, and “out-of-season” (equivalent to user class/ subsequence class)).
As to dependent Claim 17, Yocum, McCune, and Mitra teach all the limitations of Claim 16. Yocum further teaches wherein some or all of the subsequence classes correspond to motifs ([0020] a model class identifies, for a model, the class of the model; the classes may relate to seasonality, which is how the model performs with respect to a particular period of time, such as, Summer, Fall, tax season, etc.; example classes may include “all-season”, “in-season”, and “out-of-season” - thus, corresponding to seasons/ motifs).
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Yocum in view McCune and Mitra, further in view of Boughton et al. (US 2008/0037760 A1 hereinafter Boughton).
As to dependent Claim 10, Yocum, McCune, and Mitra teach all the limitations of Claim 9. However, Yocum, McCune, and Mitra fail to expressly teach wherein one or more of scheduling one or more workers to work during the future time interval based on the predicted interval value and generating a hiring plan for the future time interval.
In the same field of endeavor, Boughton teaches wherein one or more of scheduling one or more workers to work during the future time interval based on the predicted interval value and generating a hiring plan for the future time interval ([0025] the estimation tool 10 uses the historical call volume data 12 to predict future call volume; [0030] based on the historical call volume data as provided in the weekly and/or daily historical call volume charts, the schedule of the agents servicing the call center, and the average handling time of calls to the call center by the agents, the estimation tool generates a staffing impact chart for a future period of time; The staffing impact chart provides, for each time interval , a positive/negative number representative of the estimated staffing need during that time interval (i.e., predicted interval value); [0034] the estimation tool can be used to adjust the schedules of agent(s) for proper call coverage in the call center; agents can also be selectively added for additional call coverage).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated wherein one or more of scheduling one or more workers to work during the future time interval based on the predicted interval value and generating a hiring plan for the future time interval, as taught by Boughton into Yocum, McCune, and Mitra. Doing so would be desirable because it allows a user to make staffing decisions proactively based on both historical and real-time data (Boughton [0003]).
Claims 12-13 and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Yocum in view of McCune and Mitra, further in view of Saleh et al. (US 2023/0252267 A1 hereinafter Saleh).
As to dependent Claim 12, Yocum, McCune, and Mitra teach all the limitations of Claim 8. However, Yocum, McCune, and Mitra fail to expressly teach wherein receiving external data by the computing device, wherein the external data comprises a set of external values and each external value of the set of external values is associated with a time interval of the plurality of time intervals.
In the same field of endeavor, Saleh teaches wherein receiving external data by the computing device, wherein the external data comprises a set of external values and each external value of the set of external values is associated with a time interval of the plurality of time intervals ([0013] different data records may also include different feature data at different points in time or timesteps, such as data occurring on a particular day of week, a selected time period or time in the past, in a particular month or other time; training data may be used for these features, as well as additional external features that may be specifically selected for a time series forecasting task; external features may include those associated with additional customer data, fraud data, transaction data, a macro-economical feature, a trend in an e-commerce industry, a pandemic effect feature, a total payment volume migration feature, or a combination thereof; these features may provide additional time-based data that may have a temporal nature and affect predictive forecasting at future timesteps/times - thus, receiving a set of external data value/ external feature data associated with time interval).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated wherein receiving external data by the computing device, wherein the external data comprises a set of external values and each external value of the set of external values is associated with a time interval of the plurality of time intervals, as taught by Saleh into Yocum, McCune, and Mitra. Doing so would be desirable because using external features would provide additional context or factors that may assist in explaining past trends and predicting future forecasts for a particular variable and provide additional accuracy, temporal relevance, and/or feature dependence (Saleh [0021]).
As to dependent Claim 13, Yocum, McCune, Mitra, and Saleh teach all the limitations of Claim 12. Saleh further teaches wherein training the selection model using the received time series data, the set of classes, the determined differences for each forecasting algorithm for each time interval of the plurality of time intervals of the time series data, and the external data ([0012] utilize a trained DNN model, such as one using an LSTM architecture, that includes an attention mechanism to focus on particular past timesteps and/or time-based/temporal data (i.e., time series data) when training the DNN to predict future timesteps and corresponding forecasts; [0013] different data records may also include different feature data at different points in time or timesteps, such as data occurring on a particular day of week, a selected time period or time in the past, in a particular month or other time (i.e., set of classes); training data may be used for these features, as well as additional external features that may be specifically selected for a time series forecasting task; external features may include those associated with additional customer data, fraud data, transaction data, a macro-economical feature, a trend in an e-commerce industry, a pandemic effect feature, a total payment volume migration feature, etc.; these features may provide additional time-based data that may have a temporal nature and affect predictive forecasting at future timesteps/times; [0021] when training the DNN model for predictive forecasts of a future traits, features, or variables, additional external features utilized to provide additional accuracy, temporal relevance, and/or feature dependence; external feature may therefore be used to provide additional context or factors that may assist in explaining past trends and predicting future forecasts for a particular variable; external features correspond to segments of temporal data that may affect the main input attribute or feature - thus, training the selection model using the received time series data, the set of classes and the external data). Mitra further teaches wherein training the selection model using the determined differences for each forecasting algorithm for each time interval of the plurality of time intervals of the time series data ([0057] the system trains all machine learning models on S11 and evaluates their performance on S12; based on the performance, the system selects top model from each pool; the top models from each pool (selected based on the evaluation in previous round) is trained on S21, and the system evaluates their performance on S22; based on the performance, the system selects the TOP model across all pools; [0058] the model generation module 130 (equivalent to selection model) trains 720 each machine learning based model from the pool P1 of machine learning based models using training subset S11; the model generation module 130 evaluates 730 each machine learning based model from the pool 610 of machine learning based models using the given model metric and using the test subset S12; [0060] the model generation module 130 evaluates 760 each machine learning based model from the pool P2 of machine learning based models using the given model metric and using training subset S22; [0061] the model generation module 130 selects the best model for forecasting time series data for the given application or based on a given model metric from the pool P2 of machine learning based models; [0036] a model metric represents a criterion for evaluating machine learning based models; a model metric represents a function or an expression used for determining a difference between data forecasted using a model with observed data (or labelled data) - thus, training using the determined differences for each model/ forecasting algorithm for each time interval of the plurality of time intervals of the time series data).
Regarding Claim 18, Yocum teaches a system ([0067] computing system) comprising:
one or more processors ([0067] computing system include one or more computer processors); and
a computer-readable medium storing computer-executable instructions that when executed by the one or more processors cause the system ([0067] computing system include one or more computer processors, non-persistent storage (e.g., volatile memory, such as random access memory (RAM), cache memory), persistent storage (e.g., a hard disk, an optical drive such as a compact disk (CD) drive or digital versatile disk (DVD) drive, a flash memory, etc.); [0071] software instructions in the form of computer readable program code to perform embodiments of the invention may be stored on a non-transitory computer readable medium; the software instructions may correspond to computer readable program code that, when executed by a processor(s), is configured to perform one or more embodiments of the invention) to:
receive time series data, wherein the time series data comprises a plurality of time intervals and each time interval is associated with an interval value ([0013] models are trained to predict time series data, which may include the average handle time and call volume; [0015] the call volume identifies the number of phone calls that are expected to be received during a specified time interval; [0023] for a given schedule of individual days/ points in time for which predictions are to be made, the system may select the best model by forecast group, prediction horizon, and model class for a particular day; predictions are then made with a set of models to generate a piecewise forecast using the best model for a particular day; [0032] the repository (150) is a computing system that may include multiple computing devices in accordance with the computing system (500) and the nodes (522) and (524); the data in the repository (150) include multiple variables, parameters, historical time series data (i.e., time series data comprising plurality of time intervals), and forecasts that are generated (i.e., interval value) and used by the components of the system - thus, comprising plurality of time intervals and each time interval associated with forecasts/ interval value);
receive a plurality of forecasting algorithms ([0037] the process generates predictions used to generate estimated headcounts from models with multiple horizons and classes; in Step 202, models are obtained corresponding to model horizons; [0043] multiple models may be trained having multiple model horizons and classes; a model discovery service may have a model space of different models that are trained; the model space may include time series models that have different classes, different lags, and be of different types; the different classes of models may include in-season models, out-of-season models, and all-season models;
create at least one class, wherein the at least one class is associated with a plurality of subsequences of the time series data, and wherein each of the plurality of subsequences comprises a time interval of the plurality of time intervals (([0020] a model class identifies, for a model, the class of the model; the classes may relate to seasonality, which is how the model performs with respect to a particular period of time; example classes may include “all-season”, “in-season”, and “out-of-season” (i.e., set of classes); different classes may be trained and scored differently during training to identify the best performing model - thus, the class is trained/ created for the model; [0023] for a given schedule of individual days/ points in time for which predictions are to be made, the system may select the best model by forecast group, prediction horizon, and model class for a particular day; [0037] the models may include lags from target time series; a target time series is a time series for which a model is used to generate predictions; [0039] as a part of generating the values and for each day to be predicted, the model having the model horizon with a lowest value that is greater than or equal to an ordinal value of the day to be predicted may be selected for that day to generate a piecewise forecast for the time series variable being predicted - thus, the class/ particular period of time is associated with plurality of subsequences of the time series data/ individual days comprising plurality of time intervals);
train each forecasting algorithm of the plurality of forecasting algorithms to predict the interval value using a portion of the time series data ([0036] training models and predicting call center volume (i.e., interval value); [0038] the time series variable values are predictions generated with trained models; the values may correspond to the days to be predicted; [0045] the training data used to train the models may include historical time series data for at least two years; additional data beyond two years may be included and more or less data may be used; the initial training of the models may use a subset of data (i.e., portion of time series data), such as the trailing two years of data; then after the models are selected, the selected models may be trained with the additional training data that was not used prior to model selection); and
train a selection model using the received time series data, the at least one class ([0025] the model discovery service (equivalent to selection model) is a set of programs executing on the server computing system as part of the server application; the model discovery service identifies and trains the models used to generate predictions of time series variable values that may be used as inputs to generate headcount predictions; the model discovery service may receive input data from the repository, train the models, score the models, select the best scoring models according to the future time range to forecast, and provide this selection of models to the demand prediction service; [0043] multiple models may be trained having multiple model horizons and classes; a model discovery service may have a model space of different models that are trained; the model space may include time series models that have different classes, different lags, and be of different types; the different classes of models may include in-season models, out-of-season models, and all-season models; [0032] the repository is a computing system that may include multiple computing devices in accordance with the computing system and the nodes; the data in the repository include multiple variables, parameters, historical time series data (i.e., time series data), and forecasts that are generated and used by the components of the system - thus, the model discovery service including model space of different models is trained using time series data and classes).
However, Yocum fails to expressly teach wherein creating each class comprises: displaying some or all of the plurality of time intervals in a user interface by the computing device; receiving a selection of the plurality of time intervals associated with the class from among the displayed some or all of the plurality of time intervals through the user interface by the computing device; and creating the class from the selection of the plurality of time intervals associated with the class by the computing device.
In the same field of endeavor, McCune teaches wherein creating each class comprises: displaying some or all of the plurality of time intervals in a user interface by the computing device ([0062] the time period settings 304 enable selection of the time periods that are associated with or that define each corresponding season, such as via month icons 314; the interface 300 illustrated in FIG. 7 correspond to a “Spring” season, as indicated by the selection of the month icons 314 for March, April, and May; a user may select different months to adjust the time period of the season via tapping or otherwise actuating the corresponding month to select and unselect the months via the interface 300; although FIG. 7 illustrates that the seasons are defined by particular months, additional embodiments may include other time periods or designations, such as days, dates, years, and so forth, to define each season); receiving a selection of the plurality of time intervals associated with the class from among the displayed some or all of the plurality of time intervals through the user interface by the computing device ([0062] the time period settings 304 enable selection of the time periods that are associated with or that define each corresponding season, such as via month icons 314; the interface 300 illustrated in FIG. 7 correspond to a “Spring” season, as indicated by the selection of the month icons 314 for March, April, and May; a user may select different months to adjust the time period of the season via tapping or otherwise actuating the corresponding month to select and unselect the months via the interface 300; although FIG. 7 illustrates that the seasons are defined by particular months, additional embodiments may include other time periods or designations, such as days, dates, years, and so forth, to define each season (i.e., class)); and creating the class from the selection of the plurality of time intervals associated with the class by the computing device ([0062] the time period settings 304 enable selection of the time periods that are associated with or that define each corresponding season, such as via month icons 314; a user may select different months to adjust the time period of the season via tapping or otherwise actuating the corresponding month to select and unselect the months via the interface 300; although FIG. 7 illustrates that the seasons are defined by particular months, additional embodiments may include other time periods or designations, such as days, dates, years, and so forth, to define each season (i.e., creating class)).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated wherein creating each class comprises: displaying some or all of the plurality of time intervals in a user interface by the computing device; receiving a selection of the plurality of time intervals associated with the class from among the displayed some or all of the plurality of time intervals through the user interface by the computing device; and creating the class from the selection of the plurality of time intervals associated with the class by the computing device, as taught by McCune into Yocum. Doing so would be desirable because it would enable user customization and control by users (McCune [0062]), thereby enhancing user experience.
However, Yocum and McCune fail to expressly teach wherein for each time interval of the plurality of time intervals of the time series data that is not in the portion: for each forecasting algorithm of the plurality of forecasting algorithms: predict the interval value for the time interval using the forecasting algorithm; and determine a difference between the interval value associated with the time interval in the time series data and the predicted interval value for the time interval; and training a selection model by the computing device using the determined differences for each forecasting algorithm for each time interval of the plurality of time intervals of the time series data.
In the same field of endeavor, Mitra teaches wherein for each time interval of the plurality of time intervals of the time series data that is not in the portion: for each forecasting algorithm of the plurality of forecasting algorithms: predicting the interval value for the time interval using the forecasting algorithm by the computing device ([0008] the system performs forecasting using the machine learning based model by determining a training data set based on the time series data; the training dataset includes a training subset and a test subset; the system trains the machine learning based model using the training subset of the training dataset and evaluates the machine learning based model using the test subset of the training dataset; [0047] the training dataset D includes a training subset T1 and a test subset T2; the model generation module 130 trains 420 the machine learning based model using the training subset T1; the model generation module 130 evaluates 430 the machine learning based model using the test subset T2; [0057] the system trains all machine learning models on S11 and evaluates their performance on S1); and determining a difference between the interval value associated with the time interval in the time series data and the predicted interval value for the time interval by the computing device ([0036] a model metric represents a criterion for evaluating machine learning based models; a model metric represents a function or an expression used for determining a difference between data forecasted using a model with observed data (or labelled data)); and training a selection model by the computing device using the determined differences for each forecasting algorithm for each time interval of the plurality of time intervals of the time series data ([0057] the system trains all machine learning models on S11 and evaluates their performance on S12; based on the performance, the system selects top model from each pool; the top models from each pool (selected based on the evaluation in previous round) is trained on S21, and the system evaluates their performance on S22; based on the performance, the system selects the TOP model across all pools; [0058] the model generation module 130 (equivalent to selection model) trains 720 each machine learning based model from the pool P1 of machine learning based models using training subset S11; the model generation module 130 evaluates 730 each machine learning based model from the pool 610 of machine learning based models using the given model metric and using the test subset S12; [0060] the model generation module 130 evaluates 760 each machine learning based model from the pool P2 of machine learning based models using the given model metric and using training subset S22; [0061] the model generation module 130 selects the best model for forecasting time series data for the given application or based on a given model metric from the pool P2 of machine learning based models; [0036] a model metric represents a criterion for evaluating machine learning based models; a model metric represents a function or an expression used for determining a difference between data forecasted using a model with observed data (or labelled data) - thus, training using the determined differences for each model/ forecasting algorithm for each time interval of the plurality of time intervals of the time series data).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated wherein for each time interval of the plurality of time intervals of the time series data that is not in the portion: for each forecasting algorithm of the plurality of forecasting algorithms: predicting the interval value for the time interval using the forecasting algorithm by the computing device; and determining a difference between the interval value associated with the time interval in the time series data and the predicted interval value for the time interval by the computing device; and training a selection model by the computing device using the determined differences for each forecasting algorithm for each time interval of the plurality of time intervals of the time series data, as taught by Mitra into Yocum and McCune. Doing so would be desirable because it would enable the system to automatically select a model from different models based on previously known applications (Mitra [0070]).
However, Yocum, McCune, and Mitra fail to expressly teach wherein receive a set of external data values, wherein each external value in the set of external values is associated with a time interval of the plurality of time intervals; create at least one class based on the set of external data values.
In the same field of endeavor, Saleh teaches wherein receive a set of external data values, wherein each external value in the set of external values is associated with a time interval of the plurality of time intervals ([0013] different data records may also include different feature data at different points in time or timesteps, such as data occurring on a particular day of week, a selected time period or time in the past, in a particular month or other time; training data may be used for these features, as well as additional external features that may be specifically selected for a time series forecasting task; external features may include those associated with additional customer data, fraud data, transaction data, a macro-economical feature, a trend in an e-commerce industry, a pandemic effect feature, a total payment volume migration feature, or a combination thereof; these features may provide additional time-based data that may have a temporal nature and affect predictive forecasting at future timesteps/times - thus, receiving a set of external data value/ external feature data associated with time interval); create at least one class based on the set of external data values ([0012] utilize a trained DNN model, such as one using an LSTM architecture, that includes an attention mechanism to focus on particular past timesteps and/or time-based/temporal data (i.e., time series data) when training the DNN to predict future timesteps and corresponding forecasts; [0013] different data records may also include different feature data at different points in time or timesteps, such as data occurring on a particular day of week, a selected time period or time in the past, in a particular month or other time (i.e., class); training data may be used for these features, as well as additional external features that may be specifically selected for a time series forecasting task; external features may include those associated with additional customer data, fraud data, transaction data, a macro-economical feature, a trend in an e-commerce industry, a pandemic effect feature, a total payment volume migration feature, etc.; these features may provide additional time-based data that may have a temporal nature and affect predictive forecasting at future timesteps/times; [0015] the DNN model trained for a predictive score, classification, or output variable associated with input features - thus, creating class/ classification in training the model based on the external data value/ external feature data associated with time interval ).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated wherein receive a set of external data values, wherein each external value in the set of external values is associated with a time interval of the plurality of time intervals; create at least one class based on the set of external data values, as taught by Saleh into Yocum, McCune, and Mitra. Doing so would be desirable because using external features would provide additional context or factors that may assist in explaining past trends and predicting future forecasts for a particular variable and provide additional accuracy, temporal relevance, and/or feature dependence (Saleh [0021]).
As to dependent Claim 19, Yocum, McCune, Mitra, and Saleh teach all the limitations of Claim 18. Yocum further teaches wherein receive a request to forecast the interval value at a future time interval ([0025] select the best scoring models according to the future time range to forecast, and provide this selection of models to the demand prediction service; [0058] FIG. 4 illustrates how the piecewise model selection displayed on the graphical user interface (400); [0063] the row (410) displays the date for the predicted days; the dates range from January 5 to January 13 (i.e., future time interval); [0064] the row (412) displays a piecewise prediction forecast generated based on multiple models; [0037] the process generates predictions used to generate estimated headcounts (i.e., forecast interval value) from models with multiple horizons and classes); use the selection model to select a forecasting algorithm of the plurality of forecasting algorithms for the future time interval ([0023] for a given schedule of individual days/ points in time for which predictions are to be made, the system may select the best model by forecast group, prediction horizon, and model class for a particular day; [0025] the model discovery service may receive input data from the repository, train the models, score the models, select the best scoring models according to the future time range to forecast, and provide this selection of models to the demand prediction service; [0037] the process generates predictions used to generate estimated headcounts (i.e., forecast interval value) from models with multiple horizons and classes; [0058] FIG. 4 illustrates how the piecewise model selection displayed on the graphical user interface (400); [0064] the row (412) displays a piecewise prediction forecast generated based on multiple models; the model “T(1,A)” is used to generate the predicted value of “8” for the predicted day “0” that corresponds to the date of January 5 using a horizon of 1 and a class of A;[0042] the estimated headcount generated using the Erlang-C traffic modeling formula with the values from the piecewise forecast and displayed on a client computing device); use the selected forecasting algorithm to predict the interval value for the future time interval ([0023] for a given schedule of individual days/ points in time for which predictions are to be made, the system may select the best model by forecast group, prediction horizon, and model class for a particular day; [0025] select the best scoring models according to the future time range to forecast; [0037] the process generates predictions used to generate estimated headcounts (i.e., forecast interval value) from models with multiple horizons and classes; [0058] FIG. 4 illustrates how the piecewise model selection displayed on the graphical user interface (400); [0064] the row (412) displays a piecewise prediction forecast generated based on multiple models; the model “T(1,A)” is used to generate the predicted value of “8” for the predicted day “0” that corresponds to the date of January 5 using a horizon of 1 and a class of A;[0042] the estimated headcount generated using the Erlang-C traffic modeling formula with the values from the piecewise forecast and displayed on a client computing device); and provide the predicted interval value for the future time interval ([0037] the process generates predictions used to generate estimated headcounts (i.e., forecast interval value) from models with multiple horizons and classes; [0058] FIG. 4 illustrates how the piecewise model selection displayed on the graphical user interface (400); [0064] the row (412) displays a piecewise prediction forecast generated based on multiple models; the model “T(1,A)” is used to generate the predicted value of “8” for the predicted day “0” that corresponds to the date of January 5 using a horizon of 1 and a class of A; [0042] the estimated headcount generated using the Erlang-C traffic modeling formula with the values from the piecewise forecast and displayed on a client computing device).
As to dependent Claim 20, Yocum, McCune, Mitra, and Saleh teach all the limitations of Claim 18. Yocum further teaches wherein each class in the set of classes comprise one or more of a user class or a subsequence class ([0023] for a given schedule of individual days/ points in time for which predictions are to be made, the system may select the best model by forecast group, prediction horizon, and model class for a particular day; [0020] a model class identifies, for a model, the class of the model; the classes may relate to seasonality, which is how the model performs with respect to a particular period of time, such as, Summer, Fall, tax season, etc.; example classes may include “all-season”, “in-season”, and “out-of-season” (equivalent to user class/ subsequence class)).
Response to Arguments
Claim Objection: Applicant’s amendments have overcome the claim objections previously set forth.
Double Patenting: Applicant’s Terminal Disclaimer filed on 10/06/2025 in association with Co-pending application 17/812,312 has been reviewed and is accepted and recorded; therefore, the provisional nonstatutory double patenting rejection is withdrawn.
35 U.S.C. §101: Applicant’s amendments and arguments with respect to 101 rejections have been fully considered and are persuasive. The 101 rejections are withdrawn.
35 U.S.C. §102/§103: In the remarks, applicant argues that the cited references do not teach the features recited in amended independent claims 1, 8, and 18.
Applicant's arguments with respect to the 103 rejections have been considered, but are moot in view of new ground of rejection made under 35 U.S.C. § 103.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Applicant is required under 37 CFR § 1.111(c) to consider these references fully when responding to this action.
Li et al. (US 2015/0302433 A1) teaches: classification module 211 can take time series information, hierarchical information, and configuration information as input at 402. At 404, classification module 211 may process the time series using a user-defined class-by-variable; classification module 211 can produce outputs for each group including, but not limited to, the classification results, demand specific statistics, and the derived information based on a user's selection; forecast information related to at least one time series of the multiple time series may be provided, for example, to a user (see [0049], [0103]).
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to REJI KARTHOLY whose telephone number is (571)272-3432. The examiner can normally be reached on Monday - Thursday 7:30 am - 3:30 pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Welch, can be reached at telephone number (571)272-7212. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center. Status information for published applications may be obtained from Patent Center. Status information for unpublished applications is available through Patent Center for authorized users only. Should you have questions about access to Patent Center, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/patents/uspto-automated- interview-request-air-form.
/REJI KARTHOLY/Primary Examiner, Art Unit 2143