DETAILED ACTION
This communication is a Final Office Action rejection on the merits. Claims 1, 3, 7-9, 11, 15-17, and 19-20 are currently pending and have been addressed below.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement (IDS)
The information disclosure statement(s) filed on 11/26/2024 comply with the provisions 37 CFR 1.97, 1.98, and MPEP 609 and is considered by the Examiner.
Response to Arguments
Applicant's arguments filed on 05/15/2026 (related to the 112 Rejection) have been fully considered and are persuasive. Applicant deleted the term “similar” from claims 6 and 14. Therefore, the 112 Rejection has been withdrawn.
Applicant's arguments filed on 05/15/2026 (related to the 101 Rejection) have been fully considered but they are not persuasive.
Applicant states, on pages 12-24, that amended claims 1, 9, and 17 integrates judicial exception into practical application with an improvement in the functioning of a computer, or an improvement to other technology or technical field, as discussed in MPEP §§ 2106.04(d)(l) and 2106.0S(a) i.e., processing historical retail data by applying cleaning, normalization, sampling, joining, filtering, and denoising techniques to produce a standardized, reliable pre-processed dataset for analysis. Although the claims include comparison operations, these comparisons are not performed as standalone mathematical calculations. Instead, comparison results are consumed by the system as execution control signals that determine whether subsequent pipeline stages are triggered, skipped, or terminated. The threshold evaluation governs control flow within the ML pipeline, including whether further data tuning or feature recalibration is required. Accordingly, the comparison step functions as a technical control mechanism rather than a mere abstract calculation.
Applicant asserts that amended claims 1, 9 and 17 integrates judicial exception into practical application with an improvement in the functioning of a computer, or an improvement to other technology or technical field, as discussed in MPEP §§ 2106.04(d)(l) and 2106.0S(a) i.e., applying regression-based univariate and multivariate data treatments on statistical features to handle missing values and outliers, resulting in transformed and analytically robust features.
Also, Applicant asserts that amended claims 1, 9 and 17 integrates judicial exception into practical application with an improvement in the functioning of a computer, or an improvement to other technology or technical field, as discussed in MPEP §§ 2106.04(d)(l) and 2106.0S(a) i.e., selects the retail algorithm with the highest accuracy by comparing accuracy metrics across available algorithms. It then checks whether this chosen algorithm produces an optimal result by comparing its output to a predefined threshold range. If the output falls within that range, the optimal result includes the selected algorithm along with its statistical features, parameters, and hyperparameters.
Further, the system and the method automatically finds a best fit algorithm at each stage of ML pipeline based on changing business requirement, thereby reducing manual interventions/supervisions required from business experts in deciding the algorithm which ultimately reduces the technical dependency and development time spent on defining ML pipeline during model creation. The system also stores statistical features and the set of parameters and the hyperparameters used in the best fit algorithm, thereby reducing the time spent on finding the set of parameters and the hyperparameters to be used in the best fit algorithm for a similar business problem faced by a new retailer.
Examiner respectfully disagrees with Applicant. Step 2A, Prong One: Amended claim 1 limitations are considered to be abstract ideas because they are directed to “mathematical concepts” which include “mathematical calculations.” In this case, determining an optimal result by comparing an output of the identified retail algorithm with a predefined output threshold is a mathematical calculation. If a claim limitation, under its broadest reasonable interpretation, covers mathematical concepts, then it falls within the “mathematical concepts” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
Step 2A, Prong Two: Claim 1 includes additional elements: one or more hardware processors; one or more processing techniques; a generative artificial intelligence (Gen AI); a Machine Learning (ML) pipeline; one or more retail algorithms; an algorithm container; and an algorithm calibrator. The processor is merely used to fetch and execute computer-readable instructions (Paragraph 0040). The processing technique is merely used to obtain pre-processed historical data (Paragraph 0045). The Gen AI is merely used to decide the predefined output threshold and the predefined range of the predefined output threshold. The Gen AI recommendations are received from a Gen AI based model which is trained with specific information related with specific business requirements of a specific enterprise. For an instance, the specific enterprise may be a pet retailer or an electronic retailer or a home improvement retailer and the like. The Gen AI work by learning the patterns in a dataset and then using that knowledge to create new content similar to the original data. The Gen AI models are ‘trained’ by feeding them the datasets to facilitate this learning (Paragraphs 0109-0110 & 0113-0114). The ML pipeline is merely used to evaluate different stages until the model starts giving optimal results. It should be noted that every stage in the ML pipeline is unique. Also, different parameter outcomes are expected based on retail business problems, such as price elasticity, price optimization, promotions, markdown optimization, space optimization, etc. (Paragraph 0104 & 0120). The retail algorithm is merely used to solve the retail problem based on the one or more statistical features (Paragraph 0006). The algorithm container is merely used to store one or more retail algorithms (Paragraph 0006). The algorithm calibrator is merely used to obtain an improved system upon determining that the optimal result is not obtained, wherein the improved system solves the retail business problem in an accurate manner (Paragraph 0006). These elements of “processor,” “processing techniques,” “Gen AI,” “ML pipeline,” “retail algorithm,” “algorithm container,” and “algorithm calibrator” are recited at a high level of generality such that it amounts no more than mere instructions to apply the exception using a generic computer element (MPEP 2106.05f). In this case, the ML pipeline is merely using different parameters and hyperparameters for solving the retail business problem (Paragraph 0104). Although the ML has different parameters and hyperparameters for each stage, the claim does not include any specific details about how the ML pipeline is selecting the different parameters and hyperparameters, which is merely claiming the idea of a solution or outcome (MPEP 2106.05(a)). The same rationale is applicable for the Gen AI, the claim does not include any specific details about how the Gen AI is generating the suggested optimal value (Paragraph 0113-0113, threshold or price elasticity), which is merely claiming the idea of a solution or outcome (MPEP 2106.05(a)). Also, “fine-tuning using an algorithm calibrator to obtain an improved system upon determining that the optimal result is not obtained” is merely describing how the machine learning is receiving continuous data to iteratively adjust the values/parameters to minimize a loss function (e.g., improve an accuracy score). Further, the “processor function of data fetch” is considered “field of use” since it’s just used to gather data for updating the machine learning, but the technology is not improved (MPEP 2106.05h). Lastly, “transforming data” is merely used to generate different type of data and standardize the data to a particular desired format (see Paragraphs 0046-0047). In this case, transforming data in a data gathering step or using a subset of data is considered extra-solution activity or a field-of-use (see MPEP 2106.05(c) & 2106.05(g))
Step 2B: As discussed above with respect to integration of the abstract idea into a practical application, the claim describes how to generally “apply” the concept of identifying a retail algorithm among the one or more retail algorithms that achieves highest accuracy. As discussed in Step 2A, Prong Two above, the recitation of the additional elements amount to no more than mere instructions to apply the exception using a generic computer component. Also, the step of “fine-tuning using an algorithm calibrator to obtain an improved system upon determining that the optimal result is not obtained” is considered a well-understood, routing, and conventional function since it’s just “receiving or transmitting data over a network” and “performing repetitive calculations” (MPEP 2106.05(d)).
The claim fails to recite any improvements to another technology or technical field, improvements to the functioning of the computer itself, use of a particular machine, effecting a transformation or reduction of a particular article to a different state or thing, adding unconventional steps that confine the claim to a particular useful application, and/or meaningful limitations beyond generally linking the use of an abstract idea to a particular environment. See 84 Fed. Reg. 55. Viewed individually or as a whole, these additional claim element(s) do not provide meaningful limitation(s) to transform the abstract idea into a patent eligible application of the abstract idea such that the claim(s) amounts to significantly more than the abstract idea itself. Thus, the claim is ineligible.
Independent claims 9 and 17 recite similar features and therefore are rejected for the same reasons as independent claim 1. Claims 3, 7-8, 11, 15-16, and 19-20 are rejected for having the same deficiencies as those set forth with respect to the claims that they depend from, independent claims 1,9, and 17.
Applicant's arguments filed on 05/15/2026 (related to the 103 Rejection) have been fully considered but are moot in view of new grounds of rejection. Applicant's amendments necessitated the new ground(s) of rejection presented in this Office action. Rejection based on a newly cited reference(s) follows.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 3, 7-9, 11, 15-17, and 19-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., an abstract idea) without reciting significantly more.
Independent Claim 1
Step One - First, pursuant to step 1 in the January 2019 Revised Patent Subject Matter Eligibility Guidance (“2019 PEG”) on 84 Fed. Reg. 53, the claim 1 is directed to a method which is a statutory category.
Step 2A, Prong One - Claim 1 recites: A method, comprising: receiving historical data associated with a retailer and a retail business problem, wherein the historical data comprises transaction data, and sale associated data of a predefined time period; processing the historical data based on the retail business problem to obtain a pre-processed historical data, wherein the one or more processing include a data cleaning, a data normalization, a data sampling, a data joining, a data filtering and denoising; dynamically generating one or more statistical features and recommendations based on the pre-processed historical data based on data feeds available to a pipeline; performing one or more data treatments on the one or more statistical features to obtain one or more transformed statistical features including missing value treatments, outlier treatments; solving the retail business problem based on the one or more statistical features using one or more retail algorithms, and wherein each retail algorithm uses a set of parameters and hyperparameters; identifying a retail algorithm among the one or more retail algorithms that achieves highest accuracy, wherein the retail algorithm is identified by comparing accuracy measures of the one or more retail algorithms; determining whether an optimal result is obtained using the identified retail algorithm by comparing an output of the identified retail algorithm with a predefined output threshold, wherein the optimal result is obtained when the output of the identified retail algorithm is in a predefined range of the predefined output threshold, wherein the optimal result comprises of the determined retail algorithm, the one or more statistical features, and the set of parameters and the hyperparameters used in the determined retail algorithm; saving the determined optimal result along with historical data and the retail business problem, upon determining that the optimal result is obtained, wherein helps in identifying the optimal result and the set of parameters and the hypermeters of the ML pipeline for future validation of associated business or user's problem for a new retailer, and automatically finds a fit algorithm at each stage of the pipeline based on the received business or user's problem statement and reducing a technical dependency and time spent on defining the pipeline; and fine-tuning a system to obtain an improved system upon determining that the optimal result is not obtained, wherein the improved system solves the retail business problem in an accurate manner, wherein fine-tuning the system comprises: fine-tuning the historical data used for solving the retail business problem to obtain a fine-tuned historical data; fine-tuning the one or more transformed statistical features that are used for solving the retail business problem to obtain a plurality of fine-tuned statistical features; and fine-tuning the one or more retail algorithms that are used for solving the retail business problem, wherein each retail algorithm of the one or more retail algorithms is fine-tuned using the plurality of fine-tuned statistical features and a plurality of combinations of the set of parameters, and the hyperparameters as an improvement process until the improved system is obtained, wherein fine-tune at each stage of the pipeline, including data fetch, data filter, outlier detection and removal, feature engineering, clusters, algorithms, post processing or results extraction with parameters and hyperparameters, as a continuous process until the pipeline provides the optimal result. These claim elements are considered to be abstract ideas because they are directed to “mathematical concepts” which include “mathematical calculations.” In this case, determining an optimal result by comparing an output of the identified retail algorithm with a predefined output threshold is a mathematical calculation. If a claim limitation, under its broadest reasonable interpretation, covers mathematical calculations, then it falls within the “mathematical concepts” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
Step 2A Prong 2 - The judicial exception is not integrated into a practical application. Claim 1 includes additional elements: one or more hardware processors; one or more processing techniques; a generative artificial intelligence (Gen AI); a Machine Learning (ML) pipeline; one or more retail algorithms; an algorithm container; and an algorithm calibrator.
The processor is merely used to o fetch and execute computer-readable instructions (Paragraph 0040). The processing technique is merely used to obtain pre-processed historical data (Paragraph 0045). The Gen AI is merely used to decide the predefined output threshold and the predefined range of the predefined output threshold. The Gen AI recommendations are received from a Gen AI based model which is trained with specific information related with specific business requirements of a specific enterprise. For an instance, the specific enterprise may be a pet retailer or an electronic retailer or a home improvement retailer and the like. The Gen AI work by learning the patterns in a dataset and then using that knowledge to create new content similar to the original data. The Gen AI models are ‘trained’ by feeding them the datasets to facilitate this learning (Paragraphs 0109-0110 & 0113-0114). The ML pipeline is merely used to evaluate different stages until the model starts giving optimal results. It should be noted that every stage in the ML pipeline is unique. Also, different parameter outcomes are expected based on retail business problems, such as price elasticity, price optimization, promotions, markdown optimization, space optimization, etc. (Paragraph 0104 & 0120). The retail algorithm is merely used to solve the retail problem based on the one or more statistical features (Paragraph 0006). The algorithm container is merely used to store one or more retail algorithms (Paragraph 0006). The algorithm calibrator is merely used to obtain an improved system upon determining that the optimal result is not obtained, wherein the improved system solves the retail business problem in an accurate manner (Paragraph 0006). Merely stating that the step is performed by a computer component results in “apply it” on a computer (MPEP 2106.05f). These elements of “processor,” “processing technique,” “Gen AI,” “ML pipeline,” “retail algorithm,” “algorithm container,” and “algorithm calibrator” are recited at a high level of generality such that it amounts no more than mere instructions to apply the exception using a generic computer element. The processor is considered “field of use” since it’s just used to receive/fetch historical data for an analysis, but the technology is not improved (MPEP 2106.05h). Accordingly, alone and in combination, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. Therefore, the claim is directed to an abstract idea.
Step 2B - The claim does not include additional elements that are sufficient to amount significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the claims describe how to generally “apply” the concept of identifying a retail algorithm among the one or more retail algorithms that achieves highest accuracy. The specification shows that the processor is merely used to o fetch and execute computer-readable instructions (Paragraph 0040). The processing technique is merely used to obtain pre-processed historical data (Paragraph 0045). The Gen AI is merely used to decide the predefined output threshold and the predefined range of the predefined output threshold. The Gen AI recommendations are received from a Gen AI based model which is trained with specific information related with specific business requirements of a specific enterprise. For an instance, the specific enterprise may be a pet retailer or an electronic retailer or a home improvement retailer and the like. The Gen AI work by learning the patterns in a dataset and then using that knowledge to create new content similar to the original data. The Gen AI models are ‘trained’ by feeding them the datasets to facilitate this learning (Paragraphs 0109-0110 & 0113-0114). The ML pipeline is merely used to evaluate different stages until the model starts giving optimal results. It should be noted that every stage in the ML pipeline is unique. Also, different parameter outcomes are expected based on retail business problems, such as price elasticity, price optimization, promotions, markdown optimization, space optimization, etc. (Paragraph 0104 & 0120). The retail algorithm is merely used to solve the retail problem based on the one or more statistical features (Paragraph 0006). The algorithm container is merely used to store one or more retail algorithms (Paragraph 0006). The algorithm calibrator is merely used to obtain an improved system upon determining that the optimal result is not obtained, wherein the improved system solves the retail business problem in an accurate manner (Paragraph 0006). In this case, the ML pipeline is merely using different parameters and hyperparameters for solving the retail business problem (Paragraph 0104). Although the ML has different parameters and hyperparameters for each stage, the claim does not include any specific details about how the ML pipeline is selecting the different parameters and hyperparameters, which is merely claiming the idea of a solution or outcome (MPEP 2106.05(a)). The same rationale is applicable for the Gen AI, the claim does not include any specific details about how the Gen AI is generating the suggested optimal value (Paragraph 0113-0113, threshold or price elasticity), which is merely claiming the idea of a solution or outcome (MPEP 2106.05(a)). Also, “fine-tuning using an algorithm calibrator to obtain an improved system upon determining that the optimal result is not obtained” is merely describing how the machine learning is receiving continuous data to iteratively adjust the values/parameters to minimize a loss function (e.g., improve an accuracy score). Further, the step of “fine-tuning to obtain an improved system upon determining that the optimal result is not obtained” is considered a well-understood, routine, and conventional function since it's just “receiving or transmitting data over a network” and “performing repetitive calculations” (MPEP 2106.05(d)). Thus, nothing in the claim adds significantly more to the abstract idea. The claim is ineligible.
Independent claim 9 is directed to a system at step 1, which is a statutory category. Claim 9 recites similar limitations as claim 1 and is rejected for the same reasons at step 2a, prong one; step 2a, prong 2; and step 2b. Claim 9 further recites “memory” and “communication interface” – which are treated as just an explicit “processor/computer” for storing and executing the operations and are treated under MPEP 2106.05f in the same manner as claim 1. Also, the communication interface is considered a well-understood, routing, and conventional function of “receiving or transmitting data over a network” (MPEP 2106.05(d)). Accordingly, these additional elements are viewed as “apply it on a computer” at step 2a, prong 2 and step 2b.
Independent claim 17 is directed to an article of manufacture at step 1, which is a statutory category. Claim 17 recites similar limitations as claim 1 and is rejected for the same reasons at step 2a, prong one; step 2a, prong 2; and step 2b. Claim 17 further recites “non-transitory machine-readable information storage medium” – which is treated as just an explicit “processor/computer” for storing and executing the operations and is treated under MPEP 2106.05f in the same manner as claim 1. Accordingly, this additional element is viewed as “apply it on a computer” at step 2a, prong 2 and step 2b.
Dependent claims 3 and 11 are not directed to any additional claim elements. Rather, these claims offer further descriptive limitations of elements found in the independent claims and addressed above - such as to: solve the retail business problem based on the one or more transformed statistical features using the one or more retail algorithms. In this case, the function of “transforming data” is merely used to remove outliers and/or fill missing values and/or transform data to a different format (Paragraph 0064). Transforming data in a data gathering step is not considered an eligible transformation (MPEP 2106.05(c)). Thus, nothing in the claim adds significantly more to the abstract idea. The claim is ineligible.
Dependent claims 7-8, 15-16, and 19-20 are not directed to any additional claim elements. Rather, these claims offer further descriptive limitations of elements found in the independent claims and addressed above - such as: wherein the algorithm calibrator repository includes a value suggested by at least one subject matter expert; and wherein the one or more retail algorithms comprise one or more of open-source retail algorithms, licensed retail algorithms, and customized retail algorithms. In this case, the algorithm calibrator is merely used to store the optimal results based on an optimal value suggested by at least one subject matter expert. However, using a database is considered “field of use” MPEP 2106.05h at Step 2A, Prong 2, since the database is not improved, and that data is just placed there. At Step 2B, this is conventional still, storing information in a memory (see MPEP 2106.05d). Thus, nothing in the claim adds significantly more to the abstract idea. The claim is ineligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 3, 7-9, 11, 15-17, and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Khanafer et al. (US 2023/0401592 A1), in view of Joseph et al. (US 2020/0184494 A1), in further view of Athreya et al. (US 12,118,494 B1).
Regarding claim 1 (Currently Amended), Khanafer et al. discloses a processor implemented method, comprising (Abstract, Systems and methods for dynamic demand sensing in a supply chain in which constantly-updated data is used to select a machine learning model or retrain a pre-selected machine learning model, for forecasting sales of a product at a specific location. The updated data includes product information and geographic information. Also disclosed are systems and methods relating to demand forecasting and readjusting forecasts based on forecast error; Figure 2 and related text in Paragraph 0062, item 204, Processor):
receiving, via one or more hardware processors, historical data associated with a retailer and a retail business problem, wherein the historical data comprises transaction data, and sale associated data of a predefined time period (Figure 2 and related text in Paragraph 0062, item 204, Processor; Paragraph 0049, Historical data may be collected from a variety of sources. For example, data may be collected from a client/user that includes historical plus forwarding looking data such as campaigns. In some embodiments, historical client data can include point-of-sales data that provides information on the amount of product sold at a particular day at a particular location; and inventory of a particular product at a particular location. Other types of data can be mined from the web and social media, such as weather data, financial markets, and the like. Calendar data that includes local holidays, along with local event data may also be collected. Promotion campaign details for a particular product at a particular location can also be included, and other relevant events. In summary, any information that relates to, or impacts upon, the sales of a particular product at a particular location, can be used as part of the input dataset; Paragraph 0150, At block 1402, Method 1400 includes collecting historical data during a first time interval. For example, the retailer transmits sales data of the first product on a daily basis corresponding to each store location from Client server 1512 to Data store 1510. The sales data is collected and stored daily in Data store 1510 over a first time interval for future use. In this example, the first time interval includes 1 year. In practice, however, sales data may be collected over any time interval, e.g., weeks, months, years, etc. In this example, daily weather data corresponding to each day of that 1 year period is also collected; As stated in Paragraph 0036 of Applicant’s specification, sale associated data may include promotion details and weather);
processing, via the one or more hardware processors, the historical data based on the retail business problem using one or more processing techniques to obtain a pre-processed historical data, wherein the one or more processing techniques include a data cleaning, a data normalization, a data sampling, a data joining, a data filtering and denoising (Figure 2 and related text in Paragraph 0062, item 204, Processor; Paragraph 0050, The raw data is first processed, before being used by machine learning models. In some embodiments, that can entail features generation (which is fed into the various models of the Machine Learning Module); Paragraph 0053, n some embodiments, that time frame is a 13-week horizon. The “live” input data used for forecasting can include sales data from a previous time period (e.g. sales from 1 month, or 2 months, or 3 months ago, or more); promotion campaigns, weather data for the location and in the vicinity of the location, market indexes for the location and in the vicinity of the location; and events at or in the vicinity of the location; Paragraph 0082, External data module 110 fetches data (at block 602) from external data source(s) 108 which can include raw data about weather, market indices, trends, etc. The external data source(s) 108 provide data that complements client data source 102 (of FIG. 1). The raw data is cleaned (or validated) to remove outliers, and transformed (at block 604) for storage, at block 606, in the ML storage 106; Paragraph 0083, Pre-processing may include transformation, validation, remediation, or any combination thereof, of the data; Examiner notes that “generating features” includes the steps of data normalization, data sampling, and data joining (e.g., selecting data for a specific location and period of time). Also, “removing outliers” is part of the data cleaning, a data normalization, data filtering, and denoising).
dynamically generating, via the one or more hardware processors, one or more statistical features … based on the pre-processed historical data using a feature generation technique and an algorithm container framework based on the data feeds available to a Machine Learning (ML pipeline), … (Figure 2 and related text in Paragraph 0062, item 204, Processor; Paragraph 0048, Once a model is chosen (for best performance for an item and store combination), the contribution of each feature (on the demand predictions) may be evaluated through model interpretation techniques (e.g. SHapley Additive exPlantions); Paragraph 0050, The raw data is first processed, before being used by machine learning models. In some embodiments, that can entail features generation (which is fed into the various models of the Machine Learning Module); Paragraph 0053, n some embodiments, that time frame is a 13-week horizon. The “live” input data used for forecasting can include sales data from a previous time period (e.g. sales from 1 month, or 2 months, or 3 months ago, or more); promotion campaigns, weather data for the location and in the vicinity of the location, market indexes for the location and in the vicinity of the location; and events at or in the vicinity of the location);
performing, via the one or more hardware processors, using regression-based univariate and multivariate technique one or more data treatments on the one or more statistical features to obtain one or more transformed statistical features including missing value treatments, outlier treatments (Paragraph 0071, Validation of the data simple means to determine whether there are potential errors in the incoming data. For example, validation can include identification of missing data, null data, differences in row counts and data mismatches. In some embodiments, data validation module may use a machine learning algorithm in conjunction with a z-score threshold value to identify anomalous data values; Paragraph 0082, External data module 110 fetches data (at block 602) from external data source(s) 108 which can include raw data about weather, market indices, trends, etc. The external data source(s) 108 provide data that complements client data source 102 (of FIG. 1). The raw data is cleaned (or validated) to remove outliers, and transformed (at block 604) for storage, at block 606, in the ML storage 106; Paragraph 0083, Pre-processing may include transformation, validation, remediation, or any combination thereof, of the data);
solving, via the one or more hardware processors, the retail business problem based on the one or more statistical features using one or more retail algorithms, wherein the one or more retail algorithms are accessed from an algorithm container, and wherein each retail algorithm uses a set of parameters and hyperparameters (Figure 2 and related text in Paragraph 0062, item 204, Processor; Paragraph 0047, The demand sensing method can provide predicted daily sales for a single products (for example, according to their stock keeping unit (SKU) identification codes) for single locations (e.g. retail stores) over some horizon (e.g. 13 weeks ahead) for a variety of purposes, including: allowance by the user to use the predictions to drive replenishment orders at the defined locations; and gaining an analytical understanding of the factors driving the predicted sales in order to plan for the future; Paragraph 0048, The data processing services are composed of various components of a machine learning pipeline. Per user request, features may be generated from the raw user-specific and public datasets. Then one or more quantile regression models can be trained with these features. Selection of features and hyperparameters can be achieved through the evaluation of each model on the same validation set. The evaluation comprises managing a simulated inventory for the period of time equivalent to the validation set, where orders are given based on simple heuristics and key performance metrics are measured, such as excessive inventory over a period of time and number of stock out days. Once a model is chosen (for best performance for an item and store combination), the contribution of each feature (on the demand predictions) may be evaluated through model interpretation techniques (e.g. SHapley Additive exPlantions). In a last step, data related to predictions, prediction quality, and prediction contributions may be gathered and illustrated to the user by a number of interactive visualizations that are found in user-application interfaces mentioned above; Paragraph 0103, If this is not the first time a forecasting request for this particular product and location is made, then monitor module 112 checks the ML storage 106 to see if any new class of relevant signal data has been added since the last forecast request for the particular product and location, at block 1008. If the answer is yes, then monitor module 112 flags the request to undergo a full model selection process at block 1006, which is subsequently sent to forecasting module 114 (see FIG. 9); Paragraph 0110, If the time threshold is not surpassed, monitor module 112 proceeds to instruct forecasting module 114 to forecast using the current model at block 1018, without any retraining; Examiner interprets the ML storage as the algorithm container since the ML storage stores one or more retail algorithms for a specific product and location);
identifying, via the one or more hardware processors, a retail algorithm among the one or more retail algorithms that achieves highest accuracy, wherein the retail algorithm is identified by comparing accuracy measures of the one or more retail algorithms (Figure 2 and related text in Paragraph 0062, item 204, Processor; Paragraph 0107, If the answer at block 1010 is no, monitor module 112 proceeds to block 1012 to evaluate the performance of the machine learning model used in the previous forecast. With reference to FIG. 9, once the forecasting module 114 provides a forecast, the forecast is stored in the ML storage 106. Monitor module 112 evaluates the forecast on an ongoing basis by comparing the forecasted values with the actual values as the latter are uploaded to ML storage 106 on an ongoing basis. Evaluation methods known in the art may be used to evaluate the accuracy of the forecasted values, and a criterion may be selected to determine whether or not the forecast remains viable. In some embodiments, the evaluation method can be selected from mean absolute percentage error (MAPE); mean absolute scaled error (MASE), mean absolute error (MAE), and Weighted Mean Absolute Percentage Error (WMAPE). If the forecast is not deemed viable, then monitor module 112 flags the request to undergo a full model selection process at block 1006, which is subsequently sent to forecasting module 114 (see FIG. 9); Paragraph 0137, A number of ML models, such as gradient-boosted trees, ensemble of trees and support vector regression, were used during the initial training set. A gradient-boosted tree model, Light GBM, was selected during validation, and retrained on the dataset from September 2016 to Jan. 15, 2018. In this example, all the data, except for the last 20%, was used for training the selected model. In some embodiments, the testing dataset may be the smaller of the dataset of the period of the last 10-20 weeks and the last 20% of the entire dataset. In some embodiments, where the historical data set spans 1 year (52 weeks), the training/validation period can be 40-42 weeks, with remaining 10-12 weeks used for testing the selected model. In some embodiments, a nested validation scheme can be used. The best ML model may be selected according to a configuration set by the user, or any standard criteria such as MASE, MAE, WMAPE (Weighted Mean Absolute Percentage Error), etc.);
determining, via the one or more hardware processors, whether an optimal result is obtained using the identified retail algorithm by comparing an output of the identified retail algorithm with a predefined output threshold, wherein the optimal result is obtained when the output of the identified retail algorithm is in a predefined range of the predefined output threshold (Figure 2 and related text in Paragraph 0062, item 204, Processor; Paragraph 0107, If the answer at block 1010 is no, monitor module 112 proceeds to block 1012 to evaluate the performance of the machine learning model used in the previous forecast. With reference to FIG. 9, once the forecasting module 114 provides a forecast, the forecast is stored in the ML storage 106. Monitor module 112 evaluates the forecast on an ongoing basis by comparing the forecasted values with the actual values as the latter are uploaded to ML storage 106 on an ongoing basis. Evaluation methods known in the art may be used to evaluate the accuracy of the forecasted values, and a criterion may be selected to determine whether or not the forecast remains viable. In some embodiments, the evaluation method can be selected from mean absolute percentage error (MAPE); mean absolute scaled error (MASE), mean absolute error (MAE), and Weighted Mean Absolute Percentage Error (WMAPE). If the forecast is not deemed viable, then monitor module 112 flags the request to undergo a full model selection process at block 1006, which is subsequently sent to forecasting module 114 (see FIG. 9); Paragraph 0137, The best ML model may be selected according to a configuration set by the user, or any standard criteria such as MASE, MAE, WMAPE (Weighted Mean Absolute Percentage Error), etc.; Examiner interprets the criterion selected to determine whether or not the forecast remains viable as the predefined output threshold), wherein the optimal result comprises of the determined retail algorithm, the one or more statistical features, and the set of parameters and the hyperparameters used in the determined retail algorithm (Paragraph 0048, The data processing services are composed of various components of a machine learning pipeline. Per user request, features may be generated from the raw user-specific and public datasets. Then one or more quantile regression models can be trained with these features. Selection of features and hyperparameters can be achieved through the evaluation of each model on the same validation set. The evaluation comprises managing a simulated inventory for the period of time equivalent to the validation set, where orders are given based on simple heuristics and key performance metrics are measured, such as excessive inventory over a period of time and number of stock out days. Once a model is chosen (for best performance for an item and store combination), the contribution of each feature (on the demand predictions) may be evaluated through model interpretation techniques (e.g. SHapley Additive exPlantions). In a last step, data related to predictions, prediction quality, and prediction contributions may be gathered and illustrated to the user by a number of interactive visualizations that are found in user-application interfaces mentioned above);
saving, via the one or more hardware processors, the determined optimal result in an algorithm calibrator repository along with historical data and the retail business problem, upon determining that the optimal result is obtained, wherein the algorithm calibrator repository helps in identifying the optimal result and the set of parameters and the hypermeters of the ML pipeline for future validation of associated business or user's problem for a new retailer, and automatically finds a fit algorithm at each stage of the ML pipeline based on the received business or user's problem statement and reducing a technical dependency and time spent on defining the ML pipeline (Figure 2 and related text in Paragraph 0062, item 204, Processor; Paragraph 0047, The demand sensing method can provide predicted daily sales for a single products (for example, according to their stock keeping unit (SKU) identification codes) for single locations (e.g. retail stores) over some horizon (e.g. 13 weeks ahead) for a variety of purposes, including: allowance by the user to use the predictions to drive replenishment orders at the defined locations; and gaining an analytical understanding of the factors driving the predicted sales in order to plan for the future; Paragraph 0048, The data processing services are composed of various components of a machine learning pipeline. Per user request, features may be generated from the raw user-specific and public datasets. Then one or more quantile regression models can be trained with these features. Selection of features and hyperparameters can be achieved through the evaluation of each model on the same validation set. The evaluation comprises managing a simulated inventory for the period of time equivalent to the validation set, where orders are given based on simple heuristics and key performance metrics are measured, such as excessive inventory over a period of time and number of stock out days. Once a model is chosen (for best performance for an item and store combination), the contribution of each feature (on the demand predictions) may be evaluated through model interpretation techniques (e.g. SHapley Additive exPlantions). In a last step, data related to predictions, prediction quality, and prediction contributions may be gathered and illustrated to the user by a number of interactive visualizations that are found in user-application interfaces mentioned above; Paragraph 0103, If this is not the first time a forecasting request for this particular product and location is made, then monitor module 112 checks the ML storage 106 to see if any new class of relevant signal data has been added since the last forecast request for the particular product and location, at block 1008. If the answer is yes, then monitor module 112 flags the request to undergo a full model selection process at block 1006, which is subsequently sent to forecasting module 114 (see FIG. 9); Paragraph 0110, If the time threshold is not surpassed, monitor module 112 proceeds to instruct forecasting module 114 to forecast using the current model at block 1018, without any retraining; Examiner notes that the ML with the best performance is stored in the ML storage. Also, it can be noted that the claim language is written in alternative form. The limitation taught by Khanafer et al. is based on an optimal result for future validation of associated business." Examiner interprets “specific product and location” as the associated business);
and fine-tuning, via the one or more hardware processors, a system using an algorithm calibrator to obtain an improved system upon determining [an expanded data set], wherein the improved system solves the retail business problem in an accurate manner, wherein fine-tuning the system using the algorithm calibrator comprises: fine-tuning, via the one or more hardware processors, the historical data used for solving the retail business problem by the algorithm calibrator to obtain a fine-tuned historical data; fine-tuning, via the one or more hardware processors, the one or more transformed statistical features that are used for solving the retail business problem by the algorithm calibrator to obtain a plurality of fine-tuned statistical features; and fine-tuning, via the one or more hardware processors, the one or more retail algorithms that are used for solving the retail business problem by the algorithm calibrator, wherein each retail algorithm of the one or more retail algorithms is fine-tuned using the plurality of fine-tuned statistical features and a plurality of combinations of the set of parameters, and the hyperparameters as an improvement process until the improved system is obtained, wherein the algorithm calibrator fine-tune at each stage of the ML pipeline (Paragraph 0047, The data processing services are composed of various components of a machine learning pipeline. Per user request, features may be generated from the raw user-specific and public datasets. Then one or more quantile regression models can be trained with these features. Selection of features and hyperparameters can be achieved through the evaluation of each model on the same validation set. The evaluation comprises managing a simulated inventory for the period of time equivalent to the validation set, where orders are given based on simple heuristics and key performance metrics are measured, such as excessive inventory over a period of time and number of stock out days. Once a model is chosen (for best performance for an item and store combination), the contribution of each feature (on the demand predictions) may be evaluated through model interpretation techniques (e.g. SHapley Additive exPlantions); Figure 2 and related text in Paragraph 0062, item 204, Processor; Paragraph 0107, If the answer at block 1010 is no, monitor module 112 proceeds to block 1012 to evaluate the performance of the machine learning model used in the previous forecast. With reference to FIG. 9, once the forecasting module 114 provides a forecast, the forecast is stored in the ML storage 106. Monitor module 112 evaluates the forecast on an ongoing basis by comparing the forecasted values with the actual values as the latter are uploaded to ML storage 106 on an ongoing basis. Evaluation methods known in the art may be used to evaluate the accuracy of the forecasted values, and a criterion may be selected to determine whether or not the forecast remains viable. In some embodiments, the evaluation method can be selected from mean absolute percentage error (MAPE); mean absolute scaled error (MASE), mean absolute error (MAE), and Weighted Mean Absolute Percentage Error (WMAPE). If the forecast is not deemed viable, then monitor module 112 flags the request to undergo a full model selection process at block 1006, which is subsequently sent to forecasting module 114 (see FIG. 9); Paragraph 0121, Forecasting module 114 receives instructions from monitor module 112, as shown in FIG. 9, to either select a model (block 902), train/retrain (block 904), or forecast (block 906). In FIG. 12, block series 1222 describes a flowchart of the model selection process 1202 in an embodiment; block series 1224 describes a flowchart of the training process 1212 in an embodiment, and block 1220 refers to the forecasting of the trained ML model.; Paragraph 0126, Retraining of a selected ML model is described in block series 1224, in accordance with one embodiment. A selected ML model is first retrained on an expanded dataset at block 1214; it then makes a forecast corresponding to the period of a testing portion at block 1216, and its accuracy is evaluated, based on its performance in the testing portion, at block 1218. Details of the training/retraining vary slightly, depending on where in the overall process of FIG. 10, the selected model is being trained—within a model selection process (i.e. in block 1006, block 1006, ML storage 106 or 618); or within a retraining process alone (i.e. Block 1006)), including data fetch, data filter, outlier detection and removal, feature engineering, clusters, algorithms, post processing or results extraction with parameters and hyperparameters, as a continuous process until the ML pipeline provides the optimal result (Paragraph 0047, The data processing services are composed of various components of a machine learning pipeline. Per user request, features may be generated from the raw user-specific and public datasets. Then one or more quantile regression models can be trained with these features. Selection of features and hyperparameters can be achieved through the evaluation of each model on the same validation set; Paragraph 0082, External data module 110 fetches data (at block 602) from external data source(s) 108 which can include raw data about weather, market indices, trends, etc. The external data source(s) 108 provide data that complements client data source 102 (of FIG. 1). The raw data is cleaned (or validated) to remove outliers, and transformed (at block 604) for storage, at block 606, in the ML storage 106; Paragraph 0082, External data module 110 fetches data (at block 602) from external data source(s) 108 which can include raw data about weather, market indices, trends, etc. The external data source(s) 108 provide data that complements client data source 102 (of FIG. 1). The raw data is cleaned (or validated) to remove outliers, and transformed (at block 604) for storage, at block 606, in the ML storage 106; Paragraph 0083, Pre-processing may include transformation, validation, remediation, or any combination thereof, of the data; Paragraph 0107, Monitor module 112 evaluates the forecast on an ongoing basis by comparing the forecasted values with the actual values as the latter are uploaded to ML storage 106 on an ongoing basis. Evaluation methods known in the art may be used to evaluate the accuracy of the forecasted values, and a criterion may be selected to determine whether or not the forecast remains viable. In some embodiments, the evaluation method can be selected from mean absolute percentage error (MAPE); mean absolute scaled error (MASE), mean absolute error (MAE), and Weighted Mean Absolute Percentage Error (WMAPE). If the forecast is not deemed viable, then monitor module 112 flags the request to undergo a full model selection process at block 1006, which is subsequently sent to forecasting module 114 (see FIG. 9)).
Although Khanafer et al. discloses evaluating accuracy of the model and fine-tuning/retraining the model upon determining that there’s new data available (e.g., retrain when there’s an expanded data set), Khanafer et al. does not specifically disclose fine-tuning/retraining the model upon determining that the optimal result is not obtained (e.g., accuracy below a threshold).
However, Joseph et al. discloses and fine-tuning, via the one or more hardware processors, a system using an algorithm calibrator to obtain an improved system upon determining that the optimal result is not obtained, wherein the improved system solves the retail business problem in an accurate manner (Paragraph 0043, The observer 325 can compare the demand forecasts output by the model 330 against the actual historical data once it is received to determine how accurate the demand forecast results were, and to adjust the machine learning model/algorithm in response thereto as appropriate. The observer unit 325 communicates with the model (re-)training component 111 via connection 328 to train/retrain the selected model 330 whenever the accuracy of the demand forecasts output by the model 330 falls to a specified level or threshold (or when the demand forecast satisfies some other user-configurable criteria). The level at which the tripping point is triggered can be configurable by users).
It would have been obvious to one ordinary skill in the art before the effective filing date to modify the method for fine-tuning/retraining and identifying a retail algorithm among the one or more retail algorithms that achieves highest accuracy (e.g., fine-tuning/retraining is in response to detecting that there’s new data) of the invention of Khanafer et al. to further incorporate wherein the fine-tuning/retraining is upon determining that the optimal result is not obtained (e.g., accuracy below a threshold) of the invention of Joseph et al. because doing so would allow the method to retrain the selected model whenever the accuracy of the demand forecasts output by the model falls to a specified level or threshold (see Joseph et al., Paragraph 0043). Further, the claimed invention is merely a combination of old elements, and in combination each element would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Although Khanafer et al. discloses wherein the predefined output threshold and the predefined range of the predefined output threshold are decided based on a value suggested by at least one subject matter expert (Paragraph 0137, The best ML model may be selected according to a configuration set by the user, or any standard criteria such as MASE, MAE, WMAPE (Weighted Mean Absolute Percentage Error), etc.), Khanafer et al. does not specifically disclose wherein the predefined output threshold and the predefined range of the predefined output threshold are decided based on an optimal value suggested by a generative artificial intelligence (AI) based model present for the retail business problem.
However, Athreya et al. discloses dynamically generating, via the one or more hardware processors, one or more statistical features and generative artificial intelligence (Gen Al) recommendations based on the pre-processed historical data using a feature generation technique and an algorithm container framework based on data feeds available to a Machine Learning (ML) pipeline, wherein dynamically generating the Gen Al recommendations comprises training a Gen Al based model using datasets specific to business requirements of a particular enterprise, wherein the training comprising learning patterns within the datasets and generating a new content based on the learned patterns depending on the type of dataset being used and the desired outcome (Column 9, lines 5-18, The forecast adjustments or insights may be displayed to the demand planner on a display of the user-device 104 using Natural Language Generation (NLG) technique. In an embodiment, one or more natural language processing models may be used to recommend the forecast adjustment. The one or more natural language processing models may comprise, but are not limited to, Bidirectional Encoder Representations from Transformers (BERT), Robustly Optimized BERT-Pretraining Approach (ROBERTa), Generative Pre-Trained Transformer (GPT), Amazon Comprehend (AWS Comprehend), Google Cloud Natural API, an Information Retrieval model, a Semantic Textual Similarity model, and a Paraphrase Identification; Column 9, lines 19-51, Below are a few examples of insights generated (maybe referred to as forecast adjustment recommendations or generating guidance) by the system, for demand planner, for mitigating the risk associated with the machine-generated forecast for the SKU: a) Consider increasing the value of forecasted sales because machine-generated (or model) forecasts for the SKU have been significantly lower than the actual sales for the past several periods. b) Maintain the forecasted sale within a range of 100 to 180 units because the model has 95% confidence that the actual sale for the SKU would fall within this range. c) Consider reducing the forecast for the upcoming sale event because model has shown tendency to over-forecast sales for the SKU during large sale events. d) Validate or adjust the forecasted value in the context of the significant price increase planned for a given SKU in the next month. The model has never seen this kind of price increase in history. So the impact of price may not be correctly captured by the model. e) Adjust the forecast of the given SKU to account for the impact of 2 degree Celsius higher temperature projected for the next three months in the region than the same months last year. Model doesn't use temperature as an input driver and, therefore cannot account for the impact of temperature change in the demand. f) There has been a consistent decline in month-on-month volume for the category of given SKU over the last six months. The model is forecasting demand for the SKU, assuming the declining trend will continue over the coming months. We recommend the demand planner validate the category trend and accordingly make adjustments to the forecast of the SKU; Column 9, lines 60-67, Based on the historical sales data for the laptop SKU, the forecasted demand consistently deviates from the actual sales figures. To address this issue, make forecast adjustments based on customer buying patterns, seasonal variations, and market trends specific to laptops. Also, the demand planner must consider external factors like competitor pricing, technological advancements, and customer reviews can help refine the forecast accuracy; Examiner notes that ChatGPT is a part of GenAI. Examiner interprets the “recommendations for a specific SKU” as the “desired outcome.” For example, the desired outcome may include suggesting to maintain the forecasted sale within a range of 100 to 180 units or suggesting to adjust the forecast based on seasonal variations. Examiner recommends to further specify how the recommendations are generated, if supported by the specification).
It would have been obvious to one ordinary skill in the art before the effective filing date to modify the method for generating one or more statistical features based on the pre-processed historical data using a feature generation technique (e.g., according to a configuration set by the user) of the invention of Khanafer et al. to further incorporate a Gen AI for generating a new content based on the learned patterns depending on the type of dataset being used and the desired outcome (e.g., provide recommendations for a particular SKU) of the invention of Athreya et al. because doing so would allow the method to generate insights for demand planner, for mitigating the risk associated with the machine-generated forecast for the SKU (see Athreya et al., Column 9, lines 19-51). Further, the claimed invention is merely a combination of old elements, and in combination each element would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Regarding claim 9 (Currently Amended), Khanafer et al. discloses a system, comprising: a memory storing instructions; one or more communication interfaces; and one or more hardware processors coupled to the memory via the one or more communication interfaces, wherein the one or more hardware processors are configured by the instructions to (Abstract, Systems and methods for dynamic demand sensing in a supply chain in which constantly-updated data is used to select a machine learning model or retrain a pre-selected machine learning model, for forecasting sales of a product at a specific location. The updated data includes product information and geographic information. Also disclosed are systems and methods relating to demand forecasting and readjusting forecasts based on forecast error; Figure 2 and related text in Paragraph 0062, System 200 includes a system server 202, ML storage 106, client data source 102 and external data source(s) 108. System server 202 can include a memory 206, a disk 208, a processor 204 and a dynamic demand sensing module 120. While one processor 204 is shown, the system server 202 can comprise one or more processors. In some embodiments, memory 206 can be volatile memory, compared with disk 208 which can be non-volatile memory. In some embodiments, system server 202 can communicate with ML storage 106, external data source(s) 108 and client data source 102 via network 210):
receive historical data associated with a retailer and a retail business problem, wherein the historical data comprises transaction data, and sale associated data of a predefined time period (Paragraph 0049, Historical data may be collected from a variety of sources. For example, data may be collected from a client/user that includes historical plus forwarding looking data such as campaigns. In some embodiments, historical client data can include point-of-sales data that provides information on the amount of product sold at a particular day at a particular location; and inventory of a particular product at a particular location. Other types of data can be mined from the web and social media, such as weather data, financial markets, and the like. Calendar data that includes local holidays, along with local event data may also be collected. Promotion campaign details for a particular product at a particular location can also be included, and other relevant events. In summary, any information that relates to, or impacts upon, the sales of a particular product at a particular location, can be used as part of the input dataset; Paragraph 0150, At block 1402, Method 1400 includes collecting historical data during a first time interval. For example, the retailer transmits sales data of the first product on a daily basis corresponding to each store location from Client server 1512 to Data store 1510. The sales data is collected and stored daily in Data store 1510 over a first time interval for future use. In this example, the first time interval includes 1 year. In practice, however, sales data may be collected over any time interval, e.g., weeks, months, years, etc. In this example, daily weather data corresponding to each day of that 1 year period is also collected; As stated in Paragraph 0036 of Applicant’s specification, sale associated data may include promotion details and weather);
process the historical data based on the retail business problem using one or more processing techniques to obtain a pre-processed historical data, wherein the one or more processing techniques include a data cleaning, a data normalization, a data sampling, a data joining, a data filtering and denoising (Figure 2 and related text in Paragraph 0062, item 204, Processor; Paragraph 0050, The raw data is first processed, before being used by machine learning models. In some embodiments, that can entail features generation (which is fed into the various models of the Machine Learning Module); Paragraph 0053, n some embodiments, that time frame is a 13-week horizon. The “live” input data used for forecasting can include sales data from a previous time period (e.g. sales from 1 month, or 2 months, or 3 months ago, or more); promotion campaigns, weather data for the location and in the vicinity of the location, market indexes for the location and in the vicinity of the location; and events at or in the vicinity of the location; Paragraph 0082, External data module 110 fetches data (at block 602) from external data source(s) 108 which can include raw data about weather, market indices, trends, etc. The external data source(s) 108 provide data that complements client data source 102 (of FIG. 1). The raw data is cleaned (or validated) to remove outliers, and transformed (at block 604) for storage, at block 606, in the ML storage 106; Paragraph 0083, Pre-processing may include transformation, validation, remediation, or any combination thereof, of the data; Examiner notes that “generating features” includes the steps of data normalization, data sampling, and data joining (e.g., selecting data for a specific location and period of time). Also, “removing outliers” is part of the data cleaning, a data normalization, data filtering, and denoising).
dynamically generate one or more statistical features … based on the pre-processed historical data using a feature generation technique and an algorithm container framework based on the data feeds available to a Machine Learning (ML pipeline), … (Figure 2 and related text in Paragraph 0062, item 204, Processor; Paragraph 0048, Once a model is chosen (for best performance for an item and store combination), the contribution of each feature (on the demand predictions) may be evaluated through model interpretation techniques (e.g. SHapley Additive exPlantions); Paragraph 0050, The raw data is first processed, before being used by machine learning models. In some embodiments, that can entail features generation (which is fed into the various models of the Machine Learning Module); Paragraph 0053, n some embodiments, that time frame is a 13-week horizon. The “live” input data used for forecasting can include sales data from a previous time period (e.g. sales from 1 month, or 2 months, or 3 months ago, or more); promotion campaigns, weather data for the location and in the vicinity of the location, market indexes for the location and in the vicinity of the location; and events at or in the vicinity of the location);
perform, using regression-based univariate and multivariate technique one or more data treatments on the one or more statistical features to obtain one or more transformed statistical features including missing value treatments, outlier treatments (Paragraph 0071, Validation of the data simple means to determine whether there are potential errors in the incoming data. For example, validation can include identification of missing data, null data, differences in row counts and data mismatches. In some embodiments, data validation module may use a machine learning algorithm in conjunction with a z-score threshold value to identify anomalous data values; Paragraph 0082, External data module 110 fetches data (at block 602) from external data source(s) 108 which can include raw data about weather, market indices, trends, etc. The external data source(s) 108 provide data that complements client data source 102 (of FIG. 1). The raw data is cleaned (or validated) to remove outliers, and transformed (at block 604) for storage, at block 606, in the ML storage 106; Paragraph 0083, Pre-processing may include transformation, validation, remediation, or any combination thereof, of the data);
solve the retail business problem based on the one or more statistical features using one or more retail algorithms, wherein the one or more retail algorithms are accessed from an algorithm container, and wherein each retail algorithm uses a set of parameters and hyperparameters (Figure 2 and related text in Paragraph 0062, item 204, Processor; Paragraph 0047, The demand sensing method can provide predicted daily sales for a single products (for example, according to their stock keeping unit (SKU) identification codes) for single locations (e.g. retail stores) over some horizon (e.g. 13 weeks ahead) for a variety of purposes, including: allowance by the user to use the predictions to drive replenishment orders at the defined locations; and gaining an analytical understanding of the factors driving the predicted sales in order to plan for the future; Paragraph 0048, The data processing services are composed of various components of a machine learning pipeline. Per user request, features may be generated from the raw user-specific and public datasets. Then one or more quantile regression models can be trained with these features. Selection of features and hyperparameters can be achieved through the evaluation of each model on the same validation set. The evaluation comprises managing a simulated inventory for the period of time equivalent to the validation set, where orders are given based on simple heuristics and key performance metrics are measured, such as excessive inventory over a period of time and number of stock out days. Once a model is chosen (for best performance for an item and store combination), the contribution of each feature (on the demand predictions) may be evaluated through model interpretation techniques (e.g. SHapley Additive exPlantions). In a last step, data related to predictions, prediction quality, and prediction contributions may be gathered and illustrated to the user by a number of interactive visualizations that are found in user-application interfaces mentioned above; Paragraph 0103, If this is not the first time a forecasting request for this particular product and location is made, then monitor module 112 checks the ML storage 106 to see if any new class of relevant signal data has been added since the last forecast request for the particular product and location, at block 1008. If the answer is yes, then monitor module 112 flags the request to undergo a full model selection process at block 1006, which is subsequently sent to forecasting module 114 (see FIG. 9); Paragraph 0110, If the time threshold is not surpassed, monitor module 112 proceeds to instruct forecasting module 114 to forecast using the current model at block 1018, without any retraining; Examiner interprets the ML storage as the algorithm container since the ML storage stores one or more retail algorithms for a specific product and location);
identify a retail algorithm among the one or more retail algorithms that achieves highest accuracy, wherein the retail algorithm is identified by comparing accuracy measures of the one or more retail algorithms (Figure 2 and related text in Paragraph 0062, item 204, Processor; Paragraph 0107, If the answer at block 1010 is no, monitor module 112 proceeds to block 1012 to evaluate the performance of the machine learning model used in the previous forecast. With reference to FIG. 9, once the forecasting module 114 provides a forecast, the forecast is stored in the ML storage 106. Monitor module 112 evaluates the forecast on an ongoing basis by comparing the forecasted values with the actual values as the latter are uploaded to ML storage 106 on an ongoing basis. Evaluation methods known in the art may be used to evaluate the accuracy of the forecasted values, and a criterion may be selected to determine whether or not the forecast remains viable. In some embodiments, the evaluation method can be selected from mean absolute percentage error (MAPE); mean absolute scaled error (MASE), mean absolute error (MAE), and Weighted Mean Absolute Percentage Error (WMAPE). If the forecast is not deemed viable, then monitor module 112 flags the request to undergo a full model selection process at block 1006, which is subsequently sent to forecasting module 114 (see FIG. 9); Paragraph 0137, A number of ML models, such as gradient-boosted trees, ensemble of trees and support vector regression, were used during the initial training set. A gradient-boosted tree model, Light GBM, was selected during validation, and retrained on the dataset from September 2016 to Jan. 15, 2018. In this example, all the data, except for the last 20%, was used for training the selected model. In some embodiments, the testing dataset may be the smaller of the dataset of the period of the last 10-20 weeks and the last 20% of the entire dataset. In some embodiments, where the historical data set spans 1 year (52 weeks), the training/validation period can be 40-42 weeks, with remaining 10-12 weeks used for testing the selected model. In some embodiments, a nested validation scheme can be used. The best ML model may be selected according to a configuration set by the user, or any standard criteria such as MASE, MAE, WMAPE (Weighted Mean Absolute Percentage Error), etc.);
determine whether an optimal result is obtained using the identified retail algorithm by comparing an output of the identified retail algorithm with a predefined output threshold, wherein the optimal result is obtained when the output of the identified retail algorithm is in a predefined range of the predefined output threshold (Figure 2 and related text in Paragraph 0062, item 204, Processor; Paragraph 0107, If the answer at block 1010 is no, monitor module 112 proceeds to block 1012 to evaluate the performance of the machine learning model used in the previous forecast. With reference to FIG. 9, once the forecasting module 114 provides a forecast, the forecast is stored in the ML storage 106. Monitor module 112 evaluates the forecast on an ongoing basis by comparing the forecasted values with the actual values as the latter are uploaded to ML storage 106 on an ongoing basis. Evaluation methods known in the art may be used to evaluate the accuracy of the forecasted values, and a criterion may be selected to determine whether or not the forecast remains viable. In some embodiments, the evaluation method can be selected from mean absolute percentage error (MAPE); mean absolute scaled error (MASE), mean absolute error (MAE), and Weighted Mean Absolute Percentage Error (WMAPE). If the forecast is not deemed viable, then monitor module 112 flags the request to undergo a full model selection process at block 1006, which is subsequently sent to forecasting module 114 (see FIG. 9); Paragraph 0137, The best ML model may be selected according to a configuration set by the user, or any standard criteria such as MASE, MAE, WMAPE (Weighted Mean Absolute Percentage Error), etc.; Examiner interprets the criterion selected to determine whether or not the forecast remains viable as the predefined output threshold), wherein the optimal result comprises of the determined retail algorithm, the one or more statistical features, and the set of parameters and the hyperparameters used in the determined retail algorithm (Paragraph 0048, The data processing services are composed of various components of a machine learning pipeline. Per user request, features may be generated from the raw user-specific and public datasets. Then one or more quantile regression models can be trained with these features. Selection of features and hyperparameters can be achieved through the evaluation of each model on the same validation set. The evaluation comprises managing a simulated inventory for the period of time equivalent to the validation set, where orders are given based on simple heuristics and key performance metrics are measured, such as excessive inventory over a period of time and number of stock out days. Once a model is chosen (for best performance for an item and store combination), the contribution of each feature (on the demand predictions) may be evaluated through model interpretation techniques (e.g. SHapley Additive exPlantions). In a last step, data related to predictions, prediction quality, and prediction contributions may be gathered and illustrated to the user by a number of interactive visualizations that are found in user-application interfaces mentioned above);
saving the determined optimal result in an algorithm calibrator repository along with historical data and the retail business problem, upon determining that the optimal result is obtained, wherein the algorithm calibrator repository helps in identifying the optimal result and the set of parameters and the hypermeters of the ML pipeline for future validation of associated business or user's problem for a new retailer, and automatically finds a fit algorithm at each stage of the ML pipeline based on the received business or user's problem statement and reducing a technical dependency and time spent on defining the ML pipeline (Figure 2 and related text in Paragraph 0062, item 204, Processor; Paragraph 0047, The demand sensing method can provide predicted daily sales for a single products (for example, according to their stock keeping unit (SKU) identification codes) for single locations (e.g. retail stores) over some horizon (e.g. 13 weeks ahead) for a variety of purposes, including: allowance by the user to use the predictions to drive replenishment orders at the defined locations; and gaining an analytical understanding of the factors driving the predicted sales in order to plan for the future; Paragraph 0048, The data processing services are composed of various components of a machine learning pipeline. Per user request, features may be generated from the raw user-specific and public datasets. Then one or more quantile regression models can be trained with these features. Selection of features and hyperparameters can be achieved through the evaluation of each model on the same validation set. The evaluation comprises managing a simulated inventory for the period of time equivalent to the validation set, where orders are given based on simple heuristics and key performance metrics are measured, such as excessive inventory over a period of time and number of stock out days. Once a model is chosen (for best performance for an item and store combination), the contribution of each feature (on the demand predictions) may be evaluated through model interpretation techniques (e.g. SHapley Additive exPlantions). In a last step, data related to predictions, prediction quality, and prediction contributions may be gathered and illustrated to the user by a number of interactive visualizations that are found in user-application interfaces mentioned above; Paragraph 0103, If this is not the first time a forecasting request for this particular product and location is made, then monitor module 112 checks the ML storage 106 to see if any new class of relevant signal data has been added since the last forecast request for the particular product and location, at block 1008. If the answer is yes, then monitor module 112 flags the request to undergo a full model selection process at block 1006, which is subsequently sent to forecasting module 114 (see FIG. 9); Paragraph 0110, If the time threshold is not surpassed, monitor module 112 proceeds to instruct forecasting module 114 to forecast using the current model at block 1018, without any retraining; Examiner notes that the ML with the best performance is stored in the ML storage. Also, it can be noted that the claim language is written in alternative form. The limitation taught by Khanafer et al. is based on an optimal result for future validation of associated business." Examiner interprets “specific product and location” as the associated business);
and fine-tune a system using an algorithm calibrator to obtain an improved system upon determining [an expanded data set], wherein the improved system solves the retail business problem in an accurate manner, wherein fine-tuning the system using the algorithm calibrator comprises: fine-tuning, via the one or more hardware processors, the historical data used for solving the retail business problem by the algorithm calibrator to obtain a fine-tuned historical data; fine-tuning, via the one or more hardware processors, the one or more transformed statistical features that are used for solving the retail business problem by the algorithm calibrator to obtain a plurality of fine-tuned statistical features; and fine-tuning, via the one or more hardware processors, the one or more retail algorithms that are used for solving the retail business problem by the algorithm calibrator, wherein each retail algorithm of the one or more retail algorithms is fine-tuned using the plurality of fine-tuned statistical features and a plurality of combinations of the set of parameters, and the hyperparameters as an improvement process until the improved system is obtained, wherein the algorithm calibrator fine-tune at each stage of the ML pipeline (Paragraph 0047, The data processing services are composed of various components of a machine learning pipeline. Per user request, features may be generated from the raw user-specific and public datasets. Then one or more quantile regression models can be trained with these features. Selection of features and hyperparameters can be achieved through the evaluation of each model on the same validation set. The evaluation comprises managing a simulated inventory for the period of time equivalent to the validation set, where orders are given based on simple heuristics and key performance metrics are measured, such as excessive inventory over a period of time and number of stock out days. Once a model is chosen (for best performance for an item and store combination), the contribution of each feature (on the demand predictions) may be evaluated through model interpretation techniques (e.g. SHapley Additive exPlantions); Figure 2 and related text in Paragraph 0062, item 204, Processor; Paragraph 0107, If the answer at block 1010 is no, monitor module 112 proceeds to block 1012 to evaluate the performance of the machine learning model used in the previous forecast. With reference to FIG. 9, once the forecasting module 114 provides a forecast, the forecast is stored in the ML storage 106. Monitor module 112 evaluates the forecast on an ongoing basis by comparing the forecasted values with the actual values as the latter are uploaded to ML storage 106 on an ongoing basis. Evaluation methods known in the art may be used to evaluate the accuracy of the forecasted values, and a criterion may be selected to determine whether or not the forecast remains viable. In some embodiments, the evaluation method can be selected from mean absolute percentage error (MAPE); mean absolute scaled error (MASE), mean absolute error (MAE), and Weighted Mean Absolute Percentage Error (WMAPE). If the forecast is not deemed viable, then monitor module 112 flags the request to undergo a full model selection process at block 1006, which is subsequently sent to forecasting module 114 (see FIG. 9); Paragraph 0121, Forecasting module 114 receives instructions from monitor module 112, as shown in FIG. 9, to either select a model (block 902), train/retrain (block 904), or forecast (block 906). In FIG. 12, block series 1222 describes a flowchart of the model selection process 1202 in an embodiment; block series 1224 describes a flowchart of the training process 1212 in an embodiment, and block 1220 refers to the forecasting of the trained ML model.; Paragraph 0126, Retraining of a selected ML model is described in block series 1224, in accordance with one embodiment. A selected ML model is first retrained on an expanded dataset at block 1214; it then makes a forecast corresponding to the period of a testing portion at block 1216, and its accuracy is evaluated, based on its performance in the testing portion, at block 1218. Details of the training/retraining vary slightly, depending on where in the overall process of FIG. 10, the selected model is being trained—within a model selection process (i.e. in block 1006, block 1006, ML storage 106 or 618); or within a retraining process alone (i.e. Block 1006)), including data fetch, data filter, outlier detection and removal, feature engineering, clusters, algorithms, post processing or results extraction with parameters and hyperparameters, as a continuous process until the ML pipeline provides the optimal result (Paragraph 0047, The data processing services are composed of various components of a machine learning pipeline. Per user request, features may be generated from the raw user-specific and public datasets. Then one or more quantile regression models can be trained with these features. Selection of features and hyperparameters can be achieved through the evaluation of each model on the same validation set; Paragraph 0082, External data module 110 fetches data (at block 602) from external data source(s) 108 which can include raw data about weather, market indices, trends, etc. The external data source(s) 108 provide data that complements client data source 102 (of FIG. 1). The raw data is cleaned (or validated) to remove outliers, and transformed (at block 604) for storage, at block 606, in the ML storage 106; Paragraph 0082, External data module 110 fetches data (at block 602) from external data source(s) 108 which can include raw data about weather, market indices, trends, etc. The external data source(s) 108 provide data that complements client data source 102 (of FIG. 1). The raw data is cleaned (or validated) to remove outliers, and transformed (at block 604) for storage, at block 606, in the ML storage 106; Paragraph 0083, Pre-processing may include transformation, validation, remediation, or any combination thereof, of the data; Paragraph 0107, Monitor module 112 evaluates the forecast on an ongoing basis by comparing the forecasted values with the actual values as the latter are uploaded to ML storage 106 on an ongoing basis. Evaluation methods known in the art may be used to evaluate the accuracy of the forecasted values, and a criterion may be selected to determine whether or not the forecast remains viable. In some embodiments, the evaluation method can be selected from mean absolute percentage error (MAPE); mean absolute scaled error (MASE), mean absolute error (MAE), and Weighted Mean Absolute Percentage Error (WMAPE). If the forecast is not deemed viable, then monitor module 112 flags the request to undergo a full model selection process at block 1006, which is subsequently sent to forecasting module 114 (see FIG. 9)).
Although Khanafer et al. discloses to evaluate accuracy of the model and to fine-tune/retrain the model upon determining that there’s new data available (e.g., retrain when there’s an expanded data set), Khanafer et al. does not specifically disclose to fine-tune/retrain the model upon determining that the optimal result is not obtained (e.g., accuracy below a threshold).
However, Joseph et al. discloses and fine-tune a system using an algorithm calibrator to obtain an improved system upon determining that the optimal result is not obtained, wherein the improved system solves the retail business problem in an accurate manner (Paragraph 0043, The observer 325 can compare the demand forecasts output by the model 330 against the actual historical data once it is received to determine how accurate the demand forecast results were, and to adjust the machine learning model/algorithm in response thereto as appropriate. The observer unit 325 communicates with the model (re-)training component 111 via connection 328 to train/retrain the selected model 330 whenever the accuracy of the demand forecasts output by the model 330 falls to a specified level or threshold (or when the demand forecast satisfies some other user-configurable criteria). The level at which the tripping point is triggered can be configurable by users).
It would have been obvious to one ordinary skill in the art before the effective filing date to modify the method for fine-tuning/retraining and identifying a retail algorithm among the one or more retail algorithms that achieves highest accuracy (e.g., fine-tuning/retraining is in response to detecting that there’s new data) of the invention of Khanafer et al. to further incorporate wherein the fine-tuning/retraining is upon determining that the optimal result is not obtained (e.g., accuracy below a threshold) of the invention of Joseph et al. because doing so would allow the method to retrain the selected model whenever the accuracy of the demand forecasts output by the model falls to a specified level or threshold (see Joseph et al., Paragraph 0043). Further, the claimed invention is merely a combination of old elements, and in combination each element would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Although Khanafer et al. discloses wherein the predefined output threshold and the predefined range of the predefined output threshold are decided based on a value suggested by at least one subject matter expert (Paragraph 0137, The best ML model may be selected according to a configuration set by the user, or any standard criteria such as MASE, MAE, WMAPE (Weighted Mean Absolute Percentage Error), etc.), Khanafer et al. does not specifically disclose wherein the predefined output threshold and the predefined range of the predefined output threshold are decided based on an optimal value suggested by a generative artificial intelligence (AI) based model present for the retail business problem.
However, Athreya et al. discloses dynamically generate one or more statistical features and generative artificial intelligence (Gen Al) recommendations based on the pre-processed historical data using a feature generation technique and an algorithm container framework based on data feeds available to a Machine Learning (ML) pipeline, wherein dynamically generating the Gen Al recommendations comprises training a Gen Al based model using datasets specific to business requirements of a particular enterprise, wherein the training comprising learning patterns within the datasets and generating a new content based on the learned patterns depending on the type of dataset being used and the desired outcome (Column 9, lines 5-18, The forecast adjustments or insights may be displayed to the demand planner on a display of the user-device 104 using Natural Language Generation (NLG) technique. In an embodiment, one or more natural language processing models may be used to recommend the forecast adjustment. The one or more natural language processing models may comprise, but are not limited to, Bidirectional Encoder Representations from Transformers (BERT), Robustly Optimized BERT-Pretraining Approach (ROBERTa), Generative Pre-Trained Transformer (GPT), Amazon Comprehend (AWS Comprehend), Google Cloud Natural API, an Information Retrieval model, a Semantic Textual Similarity model, and a Paraphrase Identification; Column 9, lines 19-51, Below are a few examples of insights generated (maybe referred to as forecast adjustment recommendations or generating guidance) by the system, for demand planner, for mitigating the risk associated with the machine-generated forecast for the SKU: a) Consider increasing the value of forecasted sales because machine-generated (or model) forecasts for the SKU have been significantly lower than the actual sales for the past several periods. b) Maintain the forecasted sale within a range of 100 to 180 units because the model has 95% confidence that the actual sale for the SKU would fall within this range. c) Consider reducing the forecast for the upcoming sale event because model has shown tendency to over-forecast sales for the SKU during large sale events. d) Validate or adjust the forecasted value in the context of the significant price increase planned for a given SKU in the next month. The model has never seen this kind of price increase in history. So the impact of price may not be correctly captured by the model. e) Adjust the forecast of the given SKU to account for the impact of 2 degree Celsius higher temperature projected for the next three months in the region than the same months last year. Model doesn't use temperature as an input driver and, therefore cannot account for the impact of temperature change in the demand. f) There has been a consistent decline in month-on-month volume for the category of given SKU over the last six months. The model is forecasting demand for the SKU, assuming the declining trend will continue over the coming months. We recommend the demand planner validate the category trend and accordingly make adjustments to the forecast of the SKU; Column 9, lines 60-67, Based on the historical sales data for the laptop SKU, the forecasted demand consistently deviates from the actual sales figures. To address this issue, make forecast adjustments based on customer buying patterns, seasonal variations, and market trends specific to laptops. Also, the demand planner must consider external factors like competitor pricing, technological advancements, and customer reviews can help refine the forecast accuracy; Examiner notes that ChatGPT is a part of GenAI. Examiner interprets the “recommendations for a specific SKU” as the “desired outcome.” For example, the desired outcome may include suggesting to maintain the forecasted sale within a range of 100 to 180 units or suggesting to adjust the forecast based on seasonal variations. Examiner recommends to further specify how the recommendations are generated, if supported by the specification).
It would have been obvious to one ordinary skill in the art before the effective filing date to modify the method for generating one or more statistical features based on the pre-processed historical data using a feature generation technique (e.g., according to a configuration set by the user) of the invention of Khanafer et al. to further incorporate a Gen AI for generating a new content based on the learned patterns depending on the type of dataset being used and the desired outcome (e.g., provide recommendations for a particular SKU) of the invention of Athreya et al. because doing so would allow the method to generate insights for demand planner, for mitigating the risk associated with the machine-generated forecast for the SKU (see Athreya et al., Column 9, lines 19-51). Further, the claimed invention is merely a combination of old elements, and in combination each element would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Regarding claim 17 (Currently Amended), Khanafer et al. discloses one or more non-transitory machine-readable information storage mediums comprising one or more instructions which when executed by one or more hardware processors cause (Paragraph 0062, System 200 includes a system server 202, ML storage 106, client data source 102 and external data source(s) 108. System server 202 can include a memory 206, a disk 208, a processor 204 and a dynamic demand sensing module 120. While one processor 204 is shown, the system server 202 can comprise one or more processors. In some embodiments, memory 206 can be volatile memory, compared with disk 208 which can be non-volatile memory; Paragraph 0063, System 200 can also include additional features and/or functionality. For example, system 200 can also include additional storage (removable and/or non-removable) including, but not limited to, magnetic or optical disks or tape. Such additional storage is illustrated in FIG. 2 by memory 206 and disk 208. Storage media can include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data. Memory 206 and disk 208 are examples of non-transitory computer-readable storage media):
receiving historical data associated with a retailer and a retail business problem, wherein the historical data comprises transaction data, and sale associated data of a predefined time period (Paragraph 0049, Historical data may be collected from a variety of sources. For example, data may be collected from a client/user that includes historical plus forwarding looking data such as campaigns. In some embodiments, historical client data can include point-of-sales data that provides information on the amount of product sold at a particular day at a particular location; and inventory of a particular product at a particular location. Other types of data can be mined from the web and social media, such as weather data, financial markets, and the like. Calendar data that includes local holidays, along with local event data may also be collected. Promotion campaign details for a particular product at a particular location can also be included, and other relevant events. In summary, any information that relates to, or impacts upon, the sales of a particular product at a particular location, can be used as part of the input dataset; Paragraph 0150, At block 1402, Method 1400 includes collecting historical data during a first time interval. For example, the retailer transmits sales data of the first product on a daily basis corresponding to each store location from Client server 1512 to Data store 1510. The sales data is collected and stored daily in Data store 1510 over a first time interval for future use. In this example, the first time interval includes 1 year. In practice, however, sales data may be collected over any time interval, e.g., weeks, months, years, etc. In this example, daily weather data corresponding to each day of that 1 year period is also collected; As stated in Paragraph 0036 of Applicant’s specification, sale associated data may include promotion details and weather);
processing the historical data based on the retail business problem using one or more processing techniques to obtain a pre-processed historical data, wherein the one or more processing techniques include a data cleaning, a data normalization, a data sampling, a data joining, a data filtering and denoising (Figure 2 and related text in Paragraph 0062, item 204, Processor; Paragraph 0050, The raw data is first processed, before being used by machine learning models. In some embodiments, that can entail features generation (which is fed into the various models of the Machine Learning Module); Paragraph 0053, n some embodiments, that time frame is a 13-week horizon. The “live” input data used for forecasting can include sales data from a previous time period (e.g. sales from 1 month, or 2 months, or 3 months ago, or more); promotion campaigns, weather data for the location and in the vicinity of the location, market indexes for the location and in the vicinity of the location; and events at or in the vicinity of the location; Paragraph 0082, External data module 110 fetches data (at block 602) from external data source(s) 108 which can include raw data about weather, market indices, trends, etc. The external data source(s) 108 provide data that complements client data source 102 (of FIG. 1). The raw data is cleaned (or validated) to remove outliers, and transformed (at block 604) for storage, at block 606, in the ML storage 106; Paragraph 0083, Pre-processing may include transformation, validation, remediation, or any combination thereof, of the data; Examiner notes that “generating features” includes the steps of data normalization, data sampling, and data joining (e.g., selecting data for a specific location and period of time). Also, “removing outliers” is part of the data cleaning, a data normalization, data filtering, and denoising).
dynamically generating one or more statistical features … based on the pre-processed historical data using a feature generation technique and an algorithm container framework based on the data feeds available to a Machine Learning (ML pipeline), … (Figure 2 and related text in Paragraph 0062, item 204, Processor; Paragraph 0048, Once a model is chosen (for best performance for an item and store combination), the contribution of each feature (on the demand predictions) may be evaluated through model interpretation techniques (e.g. SHapley Additive exPlantions); Paragraph 0050, The raw data is first processed, before being used by machine learning models. In some embodiments, that can entail features generation (which is fed into the various models of the Machine Learning Module); Paragraph 0053, n some embodiments, that time frame is a 13-week horizon. The “live” input data used for forecasting can include sales data from a previous time period (e.g. sales from 1 month, or 2 months, or 3 months ago, or more); promotion campaigns, weather data for the location and in the vicinity of the location, market indexes for the location and in the vicinity of the location; and events at or in the vicinity of the location);
performing, via the one or more hardware processors, using regression-based univariate and multivariate technique one or more data treatments on the one or more statistical features to obtain one or more transformed statistical features including missing value treatments, outlier treatments (Paragraph 0071, Validation of the data simple means to determine whether there are potential errors in the incoming data. For example, validation can include identification of missing data, null data, differences in row counts and data mismatches. In some embodiments, data validation module may use a machine learning algorithm in conjunction with a z-score threshold value to identify anomalous data values; Paragraph 0082, External data module 110 fetches data (at block 602) from external data source(s) 108 which can include raw data about weather, market indices, trends, etc. The external data source(s) 108 provide data that complements client data source 102 (of FIG. 1). The raw data is cleaned (or validated) to remove outliers, and transformed (at block 604) for storage, at block 606, in the ML storage 106; Paragraph 0083, Pre-processing may include transformation, validation, remediation, or any combination thereof, of the data);
solving the retail business problem based on the one or more statistical features using one or more retail algorithms, wherein the one or more retail algorithms are accessed from an algorithm container, and wherein each retail algorithm uses a set of parameters and hyperparameters (Paragraph 0047, The demand sensing method can provide predicted daily sales for a single products (for example, according to their stock keeping unit (SKU) identification codes) for single locations (e.g. retail stores) over some horizon (e.g. 13 weeks ahead) for a variety of purposes, including: allowance by the user to use the predictions to drive replenishment orders at the defined locations; and gaining an analytical understanding of the factors driving the predicted sales in order to plan for the future; Paragraph 0048, The data processing services are composed of various components of a machine learning pipeline. Per user request, features may be generated from the raw user-specific and public datasets. Then one or more quantile regression models can be trained with these features. Selection of features and hyperparameters can be achieved through the evaluation of each model on the same validation set. The evaluation comprises managing a simulated inventory for the period of time equivalent to the validation set, where orders are given based on simple heuristics and key performance metrics are measured, such as excessive inventory over a period of time and number of stock out days. Once a model is chosen (for best performance for an item and store combination), the contribution of each feature (on the demand predictions) may be evaluated through model interpretation techniques (e.g. SHapley Additive exPlantions). In a last step, data related to predictions, prediction quality, and prediction contributions may be gathered and illustrated to the user by a number of interactive visualizations that are found in user-application interfaces mentioned above; Paragraph 0103, If this is not the first time a forecasting request for this particular product and location is made, then monitor module 112 checks the ML storage 106 to see if any new class of relevant signal data has been added since the last forecast request for the particular product and location, at block 1008. If the answer is yes, then monitor module 112 flags the request to undergo a full model selection process at block 1006, which is subsequently sent to forecasting module 114 (see FIG. 9); Paragraph 0110, If the time threshold is not surpassed, monitor module 112 proceeds to instruct forecasting module 114 to forecast using the current model at block 1018, without any retraining; Examiner interprets the ML storage as the algorithm container since the ML storage stores one or more retail algorithms for a specific product and location);
identifying a retail algorithm among the one or more retail algorithms that achieves highest accuracy, wherein the retail algorithm is identified by comparing accuracy measures of the one or more retail algorithms (Paragraph 0107, If the answer at block 1010 is no, monitor module 112 proceeds to block 1012 to evaluate the performance of the machine learning model used in the previous forecast. With reference to FIG. 9, once the forecasting module 114 provides a forecast, the forecast is stored in the ML storage 106. Monitor module 112 evaluates the forecast on an ongoing basis by comparing the forecasted values with the actual values as the latter are uploaded to ML storage 106 on an ongoing basis. Evaluation methods known in the art may be used to evaluate the accuracy of the forecasted values, and a criterion may be selected to determine whether or not the forecast remains viable. In some embodiments, the evaluation method can be selected from mean absolute percentage error (MAPE); mean absolute scaled error (MASE), mean absolute error (MAE), and Weighted Mean Absolute Percentage Error (WMAPE). If the forecast is not deemed viable, then monitor module 112 flags the request to undergo a full model selection process at block 1006, which is subsequently sent to forecasting module 114 (see FIG. 9); Paragraph 0137, A number of ML models, such as gradient-boosted trees, ensemble of trees and support vector regression, were used during the initial training set. A gradient-boosted tree model, Light GBM, was selected during validation, and retrained on the dataset from September 2016 to Jan. 15, 2018. In this example, all the data, except for the last 20%, was used for training the selected model. In some embodiments, the testing dataset may be the smaller of the dataset of the period of the last 10-20 weeks and the last 20% of the entire dataset. In some embodiments, where the historical data set spans 1 year (52 weeks), the training/validation period can be 40-42 weeks, with remaining 10-12 weeks used for testing the selected model. In some embodiments, a nested validation scheme can be used. The best ML model may be selected according to a configuration set by the user, or any standard criteria such as MASE, MAE, WMAPE (Weighted Mean Absolute Percentage Error), etc.);
determining whether an optimal result is obtained using the identified retail algorithm by comparing an output of the identified retail algorithm with a predefined output threshold, wherein the optimal result is obtained when the output of the identified retail algorithm is in a predefined range of the predefined output threshold (Paragraph 0107, If the answer at block 1010 is no, monitor module 112 proceeds to block 1012 to evaluate the performance of the machine learning model used in the previous forecast. With reference to FIG. 9, once the forecasting module 114 provides a forecast, the forecast is stored in the ML storage 106. Monitor module 112 evaluates the forecast on an ongoing basis by comparing the forecasted values with the actual values as the latter are uploaded to ML storage 106 on an ongoing basis. Evaluation methods known in the art may be used to evaluate the accuracy of the forecasted values, and a criterion may be selected to determine whether or not the forecast remains viable. In some embodiments, the evaluation method can be selected from mean absolute percentage error (MAPE); mean absolute scaled error (MASE), mean absolute error (MAE), and Weighted Mean Absolute Percentage Error (WMAPE). If the forecast is not deemed viable, then monitor module 112 flags the request to undergo a full model selection process at block 1006, which is subsequently sent to forecasting module 114 (see FIG. 9); Paragraph 0137, The best ML model may be selected according to a configuration set by the user, or any standard criteria such as MASE, MAE, WMAPE (Weighted Mean Absolute Percentage Error), etc.; Examiner interprets the criterion selected to determine whether or not the forecast remains viable as the predefined output threshold), wherein the optimal result comprises of the determined retail algorithm, the one or more statistical features, and the set of parameters and the hyperparameters used in the determined retail algorithm (Paragraph 0048, The data processing services are composed of various components of a machine learning pipeline. Per user request, features may be generated from the raw user-specific and public datasets. Then one or more quantile regression models can be trained with these features. Selection of features and hyperparameters can be achieved through the evaluation of each model on the same validation set. The evaluation comprises managing a simulated inventory for the period of time equivalent to the validation set, where orders are given based on simple heuristics and key performance metrics are measured, such as excessive inventory over a period of time and number of stock out days. Once a model is chosen (for best performance for an item and store combination), the contribution of each feature (on the demand predictions) may be evaluated through model interpretation techniques (e.g. SHapley Additive exPlantions). In a last step, data related to predictions, prediction quality, and prediction contributions may be gathered and illustrated to the user by a number of interactive visualizations that are found in user-application interfaces mentioned above);
saving, via the one or more hardware processors, the determined optimal result in an algorithm calibrator repository along with historical data and the retail business problem, upon determining that the optimal result is obtained, wherein the algorithm calibrator repository helps in identifying the optimal result and the set of parameters and the hypermeters of the ML pipeline for future validation of associated business or user's problem for a new retailer, and automatically finds a fit algorithm at each stage of the ML pipeline based on the received business or user's problem statement and reducing a technical dependency and time spent on defining the ML pipeline (Figure 2 and related text in Paragraph 0062, item 204, Processor; Paragraph 0047, The demand sensing method can provide predicted daily sales for a single products (for example, according to their stock keeping unit (SKU) identification codes) for single locations (e.g. retail stores) over some horizon (e.g. 13 weeks ahead) for a variety of purposes, including: allowance by the user to use the predictions to drive replenishment orders at the defined locations; and gaining an analytical understanding of the factors driving the predicted sales in order to plan for the future; Paragraph 0048, The data processing services are composed of various components of a machine learning pipeline. Per user request, features may be generated from the raw user-specific and public datasets. Then one or more quantile regression models can be trained with these features. Selection of features and hyperparameters can be achieved through the evaluation of each model on the same validation set. The evaluation comprises managing a simulated inventory for the period of time equivalent to the validation set, where orders are given based on simple heuristics and key performance metrics are measured, such as excessive inventory over a period of time and number of stock out days. Once a model is chosen (for best performance for an item and store combination), the contribution of each feature (on the demand predictions) may be evaluated through model interpretation techniques (e.g. SHapley Additive exPlantions). In a last step, data related to predictions, prediction quality, and prediction contributions may be gathered and illustrated to the user by a number of interactive visualizations that are found in user-application interfaces mentioned above; Paragraph 0103, If this is not the first time a forecasting request for this particular product and location is made, then monitor module 112 checks the ML storage 106 to see if any new class of relevant signal data has been added since the last forecast request for the particular product and location, at block 1008. If the answer is yes, then monitor module 112 flags the request to undergo a full model selection process at block 1006, which is subsequently sent to forecasting module 114 (see FIG. 9); Paragraph 0110, If the time threshold is not surpassed, monitor module 112 proceeds to instruct forecasting module 114 to forecast using the current model at block 1018, without any retraining; Examiner notes that the ML with the best performance is stored in the ML storage. Also, it can be noted that the claim language is written in alternative form. The limitation taught by Khanafer et al. is based on an optimal result for future validation of associated business." Examiner interprets “specific product and location” as the associated business);
and fine-tuning a system using an algorithm calibrator to obtain an improved system upon determining [an expanded data set], wherein the improved system solves the retail business problem in an accurate manner, wherein fine-tuning the system using the algorithm calibrator comprises: fine-tuning, via the one or more hardware processors, the historical data used for solving the retail business problem by the algorithm calibrator to obtain a fine-tuned historical data; fine-tuning, via the one or more hardware processors, the one or more transformed statistical features that are used for solving the retail business problem by the algorithm calibrator to obtain a plurality of fine-tuned statistical features; and fine-tuning, via the one or more hardware processors, the one or more retail algorithms that are used for solving the retail business problem by the algorithm calibrator, wherein each retail algorithm of the one or more retail algorithms is fine-tuned using the plurality of fine-tuned statistical features and a plurality of combinations of the set of parameters, and the hyperparameters as an improvement process until the improved system is obtained, wherein the algorithm calibrator fine-tune at each stage of the ML pipeline (Paragraph 0047, The data processing services are composed of various components of a machine learning pipeline. Per user request, features may be generated from the raw user-specific and public datasets. Then one or more quantile regression models can be trained with these features. Selection of features and hyperparameters can be achieved through the evaluation of each model on the same validation set. The evaluation comprises managing a simulated inventory for the period of time equivalent to the validation set, where orders are given based on simple heuristics and key performance metrics are measured, such as excessive inventory over a period of time and number of stock out days. Once a model is chosen (for best performance for an item and store combination), the contribution of each feature (on the demand predictions) may be evaluated through model interpretation techniques (e.g. SHapley Additive exPlantions); Figure 2 and related text in Paragraph 0062, item 204, Processor; Paragraph 0107, If the answer at block 1010 is no, monitor module 112 proceeds to block 1012 to evaluate the performance of the machine learning model used in the previous forecast. With reference to FIG. 9, once the forecasting module 114 provides a forecast, the forecast is stored in the ML storage 106. Monitor module 112 evaluates the forecast on an ongoing basis by comparing the forecasted values with the actual values as the latter are uploaded to ML storage 106 on an ongoing basis. Evaluation methods known in the art may be used to evaluate the accuracy of the forecasted values, and a criterion may be selected to determine whether or not the forecast remains viable. In some embodiments, the evaluation method can be selected from mean absolute percentage error (MAPE); mean absolute scaled error (MASE), mean absolute error (MAE), and Weighted Mean Absolute Percentage Error (WMAPE). If the forecast is not deemed viable, then monitor module 112 flags the request to undergo a full model selection process at block 1006, which is subsequently sent to forecasting module 114 (see FIG. 9); Paragraph 0121, Forecasting module 114 receives instructions from monitor module 112, as shown in FIG. 9, to either select a model (block 902), train/retrain (block 904), or forecast (block 906). In FIG. 12, block series 1222 describes a flowchart of the model selection process 1202 in an embodiment; block series 1224 describes a flowchart of the training process 1212 in an embodiment, and block 1220 refers to the forecasting of the trained ML model.; Paragraph 0126, Retraining of a selected ML model is described in block series 1224, in accordance with one embodiment. A selected ML model is first retrained on an expanded dataset at block 1214; it then makes a forecast corresponding to the period of a testing portion at block 1216, and its accuracy is evaluated, based on its performance in the testing portion, at block 1218. Details of the training/retraining vary slightly, depending on where in the overall process of FIG. 10, the selected model is being trained—within a model selection process (i.e. in block 1006, block 1006, ML storage 106 or 618); or within a retraining process alone (i.e. Block 1006)), including data fetch, data filter, outlier detection and removal, feature engineering, clusters, algorithms, post processing or results extraction with parameters and hyperparameters, as a continuous process until the ML pipeline provides the optimal result (Paragraph 0047, The data processing services are composed of various components of a machine learning pipeline. Per user request, features may be generated from the raw user-specific and public datasets. Then one or more quantile regression models can be trained with these features. Selection of features and hyperparameters can be achieved through the evaluation of each model on the same validation set; Paragraph 0082, External data module 110 fetches data (at block 602) from external data source(s) 108 which can include raw data about weather, market indices, trends, etc. The external data source(s) 108 provide data that complements client data source 102 (of FIG. 1). The raw data is cleaned (or validated) to remove outliers, and transformed (at block 604) for storage, at block 606, in the ML storage 106; Paragraph 0082, External data module 110 fetches data (at block 602) from external data source(s) 108 which can include raw data about weather, market indices, trends, etc. The external data source(s) 108 provide data that complements client data source 102 (of FIG. 1). The raw data is cleaned (or validated) to remove outliers, and transformed (at block 604) for storage, at block 606, in the ML storage 106; Paragraph 0083, Pre-processing may include transformation, validation, remediation, or any combination thereof, of the data; Paragraph 0107, Monitor module 112 evaluates the forecast on an ongoing basis by comparing the forecasted values with the actual values as the latter are uploaded to ML storage 106 on an ongoing basis. Evaluation methods known in the art may be used to evaluate the accuracy of the forecasted values, and a criterion may be selected to determine whether or not the forecast remains viable. In some embodiments, the evaluation method can be selected from mean absolute percentage error (MAPE); mean absolute scaled error (MASE), mean absolute error (MAE), and Weighted Mean Absolute Percentage Error (WMAPE). If the forecast is not deemed viable, then monitor module 112 flags the request to undergo a full model selection process at block 1006, which is subsequently sent to forecasting module 114 (see FIG. 9)).
Although Khanafer et al. discloses evaluating accuracy of the model and fine-tuning/retraining the model upon determining that there’s new data available (e.g., retrain when there’s an expanded data set), Khanafer et al. does not specifically disclose fine-tuning/retraining the model upon determining that the optimal result is not obtained (e.g., accuracy below a threshold).
However, Joseph et al. discloses and fine-tuning a system using an algorithm calibrator to obtain an improved system upon determining that the optimal result is not obtained, wherein the improved system solves the retail business problem in an accurate manner (Paragraph 0043, The observer 325 can compare the demand forecasts output by the model 330 against the actual historical data once it is received to determine how accurate the demand forecast results were, and to adjust the machine learning model/algorithm in response thereto as appropriate. The observer unit 325 communicates with the model (re-)training component 111 via connection 328 to train/retrain the selected model 330 whenever the accuracy of the demand forecasts output by the model 330 falls to a specified level or threshold (or when the demand forecast satisfies some other user-configurable criteria). The level at which the tripping point is triggered can be configurable by users).
It would have been obvious to one ordinary skill in the art before the effective filing date to modify the method for fine-tuning/retraining and identifying a retail algorithm among the one or more retail algorithms that achieves highest accuracy (e.g., fine-tuning/retraining is in response to detecting that there’s new data) of the invention of Khanafer et al. to further incorporate wherein the fine-tuning/retraining is upon determining that the optimal result is not obtained (e.g., accuracy below a threshold) of the invention of Joseph et al. because doing so would allow the method to retrain the selected model whenever the accuracy of the demand forecasts output by the model falls to a specified level or threshold (see Joseph et al., Paragraph 0043). Further, the claimed invention is merely a combination of old elements, and in combination each element would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Although Khanafer et al. discloses wherein the predefined output threshold and the predefined range of the predefined output threshold are decided based on a value suggested by at least one subject matter expert (Paragraph 0137, The best ML model may be selected according to a configuration set by the user, or any standard criteria such as MASE, MAE, WMAPE (Weighted Mean Absolute Percentage Error), etc.), Khanafer et al. does not specifically disclose wherein the predefined output threshold and the predefined range of the predefined output threshold are decided based on an optimal value suggested by a generative artificial intelligence (AI) based model present for the retail business problem.
However, Athreya et al. discloses dynamically generating one or more statistical features and generative artificial intelligence (Gen Al) recommendations based on the pre-processed historical data using a feature generation technique and an algorithm container framework based on data feeds available to a Machine Learning (ML) pipeline, wherein dynamically generating the Gen Al recommendations comprises training a Gen Al based model using datasets specific to business requirements of a particular enterprise, wherein the training comprising learning patterns within the datasets and generating a new content based on the learned patterns depending on the type of dataset being used and the desired outcome (Column 9, lines 5-18, The forecast adjustments or insights may be displayed to the demand planner on a display of the user-device 104 using Natural Language Generation (NLG) technique. In an embodiment, one or more natural language processing models may be used to recommend the forecast adjustment. The one or more natural language processing models may comprise, but are not limited to, Bidirectional Encoder Representations from Transformers (BERT), Robustly Optimized BERT-Pretraining Approach (ROBERTa), Generative Pre-Trained Transformer (GPT), Amazon Comprehend (AWS Comprehend), Google Cloud Natural API, an Information Retrieval model, a Semantic Textual Similarity model, and a Paraphrase Identification; Column 9, lines 19-51, Below are a few examples of insights generated (maybe referred to as forecast adjustment recommendations or generating guidance) by the system, for demand planner, for mitigating the risk associated with the machine-generated forecast for the SKU: a) Consider increasing the value of forecasted sales because machine-generated (or model) forecasts for the SKU have been significantly lower than the actual sales for the past several periods. b) Maintain the forecasted sale within a range of 100 to 180 units because the model has 95% confidence that the actual sale for the SKU would fall within this range. c) Consider reducing the forecast for the upcoming sale event because model has shown tendency to over-forecast sales for the SKU during large sale events. d) Validate or adjust the forecasted value in the context of the significant price increase planned for a given SKU in the next month. The model has never seen this kind of price increase in history. So the impact of price may not be correctly captured by the model. e) Adjust the forecast of the given SKU to account for the impact of 2 degree Celsius higher temperature projected for the next three months in the region than the same months last year. Model doesn't use temperature as an input driver and, therefore cannot account for the impact of temperature change in the demand. f) There has been a consistent decline in month-on-month volume for the category of given SKU over the last six months. The model is forecasting demand for the SKU, assuming the declining trend will continue over the coming months. We recommend the demand planner validate the category trend and accordingly make adjustments to the forecast of the SKU; Column 9, lines 60-67, Based on the historical sales data for the laptop SKU, the forecasted demand consistently deviates from the actual sales figures. To address this issue, make forecast adjustments based on customer buying patterns, seasonal variations, and market trends specific to laptops. Also, the demand planner must consider external factors like competitor pricing, technological advancements, and customer reviews can help refine the forecast accuracy; Examiner notes that ChatGPT is a part of GenAI. Examiner interprets the “recommendations for a specific SKU” as the “desired outcome.” For example, the desired outcome may include suggesting to maintain the forecasted sale within a range of 100 to 180 units or suggesting to adjust the forecast based on seasonal variations. Examiner recommends to further specify how the recommendations are generated, if supported by the specification).
It would have been obvious to one ordinary skill in the art before the effective filing date to modify the method for generating one or more statistical features based on the pre-processed historical data using a feature generation technique (e.g., according to a configuration set by the user) of the invention of Khanafer et al. to further incorporate a Gen AI for generating a new content based on the learned patterns depending on the type of dataset being used and the desired outcome (e.g., provide recommendations for a particular SKU) of the invention of Athreya et al. because doing so would allow the method to generate insights for demand planner, for mitigating the risk associated with the machine-generated forecast for the SKU (see Athreya et al., Column 9, lines 19-51). Further, the claimed invention is merely a combination of old elements, and in combination each element would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Regarding claims 3 and 11 (Currently Amended), which are dependent of claims 1 and 9, the combination of Khanafer et al., Joseph et al., and Athreya et al. discloses all the limitations in claims 1 and 9. Khanafer et al. further discloses wherein solving the retail business problem based on the one or more statistical features using the one or more retail algorithms comprises: solving, via the one or more hardware processors, the retail business problem based on the one or more transformed statistical features using the one or more retail algorithms (Figure 2 and related text in Paragraph 0062, item 204, Processor; Paragraph 0075, FIG. 4 illustrates a transformation examples 400 in accordance with one embodiment. Examples of features 402 can include data related to: point of sales, weather, events/holidays, market index, web traffic and promotions. Features 402 may include additional categories of data, fewer, or different categories than those shown in FIG. 4.; Paragraph 0076, Example 1 404, shows how data related to a rare event, which is in binary form, is transformed to a form that includes integers, by specifying the number of days to the event. For example, the rare event can have the value ‘0’ to indicate the day a store is open (e.g. Mon-Sat) and ‘1’ to indicate the day a store is closed (e.g. Sunday). The series of ‘0’s and ‘1’s is transformed, instead, to a series of integers that indicate how many days away that a given day is to the rare event; Paragraph 0077, Example 2 406 shows an example of transforming consecutive dates to a tabular form that lists year (in one row); month (in a second row) and date (in the third row); Paragraph 0078, Example 3 408 shows an example of transforming temperature values on certain dates, to temperature values in relation to the lowest temperature reading (6° C.). The original 6° C. reading is transformed to ‘0’; 7° C. to ‘1’; 8° C. to ‘2’, and so forth. Graphical representations of transformations are discussed below; Paragraph 0104, As an example, in the intervening period between the first request and the subsequent request, ML storage 106 may have received weather data that includes a humidity index relevant to the location of the request, which was not present in the data used for the initial forecast. The humidity index is a new class of signal data that can be used in the machine learning forecasting of the particular product at the particular location. Note that if new humidity data has been received during the intervening period, but the new humidity data has no impact on the location of interest, then it is not considered as being relevant. For example, if ML storage 106 receives the humidity index for Washington, D.C., but not for Kanata ON (where the forecast is requested), then this is not considered as a relevant new class of signal data).
Regarding claims 7, 15, and 19 (7 & 15 Original, 19 Currently Amended), which are dependent of claims 1, 9, and 17 the combination of Khanafer et al., Joseph et al., and Athreya et al. discloses all the limitations in claims 1, 9, and 17. Khanafer et al. further discloses wherein the predefined output threshold and the predefined range of the predefined output threshold are decided based on one of: an optimal value suggested by a generative artificial intelligence (AI) based model present for the retail business problem, an algorithm calibrator repository maintained for the retail business problem, and a value suggested by at least one subject matter expert (SME) (Figure 2 and related text in Paragraph 0062, item 204, Processor; Paragraph 0048, The evaluation comprises managing a simulated inventory for the period of time equivalent to the validation set, where orders are given based on simple heuristics and key performance metrics are measured, such as excessive inventory over a period of time and number of stock out days. Once a model is chosen (for best performance for an item and store combination), the contribution of each feature (on the demand predictions) may be evaluated through model interpretation techniques (e.g. SHapley Additive exPlantions); Paragraph 0107, If the answer at block 1010 is no, monitor module 112 proceeds to block 1012 to evaluate the performance of the machine learning model used in the previous forecast. With reference to FIG. 9, once the forecasting module 114 provides a forecast, the forecast is stored in the ML storage 106. Monitor module 112 evaluates the forecast on an ongoing basis by comparing the forecasted values with the actual values as the latter are uploaded to ML storage 106 on an ongoing basis. Evaluation methods known in the art may be used to evaluate the accuracy of the forecasted values, and a criterion may be selected to determine whether or not the forecast remains viable. In some embodiments, the evaluation method can be selected from mean absolute percentage error (MAPE); mean absolute scaled error (MASE), mean absolute error (MAE), and Weighted Mean Absolute Percentage Error (WMAPE). If the forecast is not deemed viable, then monitor module 112 flags the request to undergo a full model selection process at block 1006, which is subsequently sent to forecasting module 114 (see FIG. 9); Paragraph 0137, The best ML model may be selected according to a configuration set by the user, or any standard criteria such as MASE, MAE, WMAPE (Weighted Mean Absolute Percentage Error), etc.; It can be noted that the claim language is written in alternative form. The limitation taught by Khanafer et al. is based on a value suggested by at least one subject matter expert (SME)." Examiner interprets the standard criteria set by the user as the predefined output threshold suggested by at least one SME).
Regarding claims 8, 16, and 20 (Original), which are dependent of claims 1, 9, and 17, the combination of Khanafer et al., Joseph et al., and Athreya et al. discloses all the limitations in claims 1, 9, and 17. Khanafer et al. further discloses wherein the one or more retail algorithms comprises one or more of open-source retail algorithms, licensed retail algorithms, and customized retail algorithms (Paragraph 0047, The demand sensing method can provide predicted daily sales for a single products (for example, according to their stock keeping unit (SKU) identification codes) for single locations (e.g. retail stores) over some horizon (e.g. 13 weeks ahead) for a variety of purposes, including: allowance by the user to use the predictions to drive replenishment orders at the defined locations; and gaining an analytical understanding of the factors driving the predicted sales in order to plan for the future; It can be noted that the claim language is written in alternative form. The limitation taught by Khanafer et al. is based on customized retail algorithms." Examiner interprets the retail algorithms customized for a specific product and location as the customized retail algorithms).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure.
Alagappan et al. (US 2024/0152775 A1) – discloses determining the accuracy of each model (i.e., validating each model) to determine which model is the optimal prediction mode 122. The optimal prediction model 122 is the model that most accurately predicts the testing data 113B. As previously mentioned, the modeling engine 120 sends the plurality of prediction models 121 to the validator 125, and the validator 125 receives the plurality of prediction models 121 from the modeling engine 120. The validator 125 charts the prediction of each prediction model to testing data 113B to determine which model is most accurate (e.g., the optimal prediction model 122) (see at least Paragraphs 0063-0064).
Chen (CN 117035841 A) – discloses a sales prediction model is constructed by different machine learning and statistical models, and the constructed sales prediction model is compared. At the same time, deep learning models such as RecurrentNeuralNetworks (RNN) and LongShortTermMemory (LSTM) can also be integrated. By attempting different models and evaluating their performance, the most suitable model or model combination can be found as a sales prediction model. The integrated learning method, such as random forest, gradient lifting tree or stacking method, is combined with the prediction result of multiple models to further improve the prediction accuracy and stability of the sales prediction model (see at least Pages 6-7).
Kirk (US 2025/0124024 A1) – discloses the query 215 from “user A” may be a natural language query 215 that states, “what are my sales numbers for the last quarter relative to my yearly targets,” then the interface may add instructions that are ingestible by the LLM 220 to cause the LLM 220 to generate data queries 240 (e.g., structured query language (SQL) queries, NoSQL queries) that are configured for corresponding data sets 245, such as a data set that includes sales data (see at least Paragraph 0038).
Steenbergen (Van Steenbergen, R.M. and Mes, M.R., 2020. Forecasting demand profiles of new products. Decision support systems, 139, p.113401) – discloses a novel approach called DemandForest that provides pre-launch forecasts for the demand of new products during their introduction period. Our approach, relying on Random Forest algorithms, utilizes product characteristics of new and existing products to predict a profile and the total demand of these new products during the introduction period. Our approach also provides prediction intervals and quantiles, which can be used to support decisions in inventory management. The forecasts are based on the historical demand of existing comparable products to overcome the challenge of new product fore casting: the lack of historical data. In this way, DemandForest is an automated data-driven approach that is able to provide estimations reducing the need of human judgment, extensive analysis or years of experience. This method is especially valuable for companies with a large number of new product introductions each year. By using the forecasts, prediction intervals, quantiles and the most important pro duct characteristics resulting from DemandForest, companies can support, enhance and also automate decision making in inventory management of new products (see at least Conclusion).
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MARJORIE PUJOLS-CRUZ whose telephone number is (571)272-4668. The examiner can normally be reached Mon-Thru 7:30 AM - 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Patricia H Munson can be reached at (571)270-5396. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/M.P./Examiner, Art Unit 3624 /HAMZEH OBAID/Primary Examiner, Art Unit 3624
Prosecution Insights