Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
The office action is in response to the application filed on October 11, 2023.
Claims 1-20 are pending and have been examined. Claims 1-20 are rejected.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter.
According to the USPTO guidelines, a claim is directed to non-statutory subject matter if:
Step 1: The claim does not fall within one of the four statutory categories of invention (process, machine, manufacture, or composition of matter), or,
Step 2: The claim recites a judicial exception, e.g. an abstract idea, without reciting additional elements that amount to significantly more than the judicial exception, as determined using the following analysis:
Step 2A, Prong 1: Does the claim recite an abstract idea, law of nature, or natural phenomenon?
Step 2A, Prong 2: Does the claim recite additional elements that integrate the judicial exception into a practical application?
Step 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception?
MPEP 2106.04(a)(2)(I) states: "The mathematical concepts grouping is defined as
mathematical relationships, mathematical formulas or equations, and mathematical
calculations."
MPEP 2106.04(a)(2)(III) states: "Accordingly, the "mental processes" abstract idea grouping is defined as concepts performed in the human mind, and examples of mental
processes include observations, evaluations, judgements, and opinions.
Further, the MPEP states: "The courts do not distinguish between mental processes that
are performed entirely in the human mind and mental processes that require a human to use a
physical aid (e.g. pen and paper or a slide run) to perform the claim limitation.
Using the two-step inquiry, it is clear that Claims 1-20 are each directed to non-statutory
subject matter as shown below:
Please note the following:
The following groups of claims are expressed in different statutory categories:
Claims 1-7 are directed to a method designed to forecast storage system performance by training, evaluating, and dynamically deploying the most accurate forecasting models to individual storage environments, where they are continuously updated to adapt to real-world inference results.
Claims 8-14 are directed to a non-transitory computer-readable medium storing a plurality of instructions which, when executed by a processor, cause the processor to carry out a process.
Claims 15-20 are directed to a system comprising of a memory, cloud computing system, and a processor configured to carry out a process designed to forecast storage system performance by training, evaluating, and dynamically deploying the most accurate forecasting models to individual storage environments.
With respect to Claims 1, 8, and 15, which are independent claims with identical claim limitations:
Step 1: Claim 1 is directed to a method, also known as a process, which is one of the four statutory categories of patentable subject matter. Claim 8 is directed to a non-transitory computer-readable medium on which computer-executable instructions are stored, corresponding to an article of manufacture, which is one of the four statutory categories of patentable subject matter. Claim 15 is directed to a system for a cloud-assisted machine learning lifecycle for storage forecasting, corresponding to an article of manufacture, which is one of the four statutory categories of patentable subject matter.
Step 2A, Prong 1: A judicial exception is recited in the claims as they recite mental processes, which are abstract ideas:
“selecting a trained machine learning model from the plurality of trained machine learning models to deploy to a target storage system;” ; Selecting a trained machine learning model from a plurality of trained machine learning models is an abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion) -See MPEP § 2106.04(a)(2)(III).
Step 2A, Prong 2: The claims do not recite additional elements that integrate the judicial exception into a practical application:
“generating a plurality of trained machine learning models using a cloud computing system by training a plurality of machine learning models to forecast storage performance for one or more storage objects of a storage system, wherein the cloud computing system is separate from any storage system;” ; Generating a plurality of trained machine learning models using a cloud computing system to train said models to forecast the storage performance of storage objects only amounts to "apply it" and the mere instructions to implement an abstract idea on a computer - see MPEP 2106.05(f)(1) in addition to generally links the use of the abstract idea to a particular technological environment or field of use - See MPEP § 2106.05(h).
“and deploying the trained machine learning model on the target storage system.” ; Deploying the selected trained machine learning model onto a target storage system only amounts to "apply it" and the mere instructions to apply the abstract idea using a generic computer component - see MPEP 2106.05(f)(2) in addition to generally links the use of the abstract idea to a particular technological environment or field of use - See MPEP § 2106.05(h).
Step 2B: The claims do not recite additional elements that amount to significantly more than the judicial exception. Generating a plurality of trained machine learning models using a cloud computing system to train said models to forecast the storage performance of storage objects amounts to "apply it" and mere instructions to implement an abstract idea on a computer. The claim fails to recite details of how a solution or outcome to a problem is accomplished because it is unclear how the "AI system" or "machine learning" is used nor does the specification make it clear how these actions are performed - see MPEP 2106.05(f)(1)). Deploying the selected trained machine learning model onto a target storage system amounts to "apply it" (or an equivalent) and mere instructions to implement an abstract idea on a computer using a generic computer component or merely uses a computer in its ordinary capacity as a tool to perform an existing process. -See MPEP 2106.05(f)(2). The usage of a machine learning model, cloud computing system, and storage system is generally linked to a particular technological environment or field of use (AI/ML/IT/IaaS/SaaS/Data Infrast.) - see MPEP 2106.05(h).
Therefore, Claims 1, 8, and 15 are directed to non-statutory subject matter and rejected.
With respect to Claims 2, 9, and 16 which have identical claim limitations and are dependent on Claims 1, 8, and 15 respectively:
Step 2A, Prong 1: A judicial exception is not recited in the claims as they do not recite an abstract idea (mathematical concepts, certain methods of organizing human activity, or mental processes), law of nature, or natural phenomenon.
Step 2A, Prong 2: The claims do not recite additional elements that integrate the judicial exception into a practical application:
“wherein selecting the trained machine learning model from the plurality of trained machine learning models includes performing a champion-challenger process with the plurality of trained machine learning models.” ; Using a selected machine learning model to perform a champion-challenger process with a plurality of models only amounts to "apply it" and the mere instructions to implement an abstract idea on a computer - see MPEP 2106.05(f)(1) in addition to generally links the use of the abstract idea to a particular technological environment or field of use - See MPEP § 2106.05(h).
Step 2B: Using a selected machine learning model to perform a champion-challenger process with a plurality of models amounts to "apply it" and mere instructions to implement an abstract idea on a computer. The claim fails to recite details of how a solution or outcome to a problem is accomplished because it is unclear how the "AI system" or "machine learning" is used nor does the specification make it clear how these actions are performed - see MPEP 2106.05(f)(1)). The usage of a machine learning model is generally linked to a particular technological environment or field of use (AI/ML) - see MPEP 2106.05(h).
Therefore, Claims 2, 9, and 16 are directed to non-statutory subject matter and rejected.
With respect to Claims 3, 10, and 17 which have identical claim limitations and are dependent on Claims 2, 9, and 16 respectively:
Step 2A, Prong 1: A judicial exception is recited in the claims as they recite mental processes, which are abstract ideas:
“determining a weighted symmetric mean absolute percentage (SMAPE) for the short-term forecast of each trained machine learning model;” ; Determining a weighted symmetric mean absolute percentage for a short-term forecast of each trained machine learning model is an abstract idea of a mathematical calculation, as directed to “a claim that recites a mathematical calculation, when the claim is given its broadest reasonable interpretation in light of the specification, will be considered as falling within the "mathematical concepts" grouping. A mathematical calculation is a mathematical operation (such as multiplication) or an act of calculating using mathematical methods to determine a variable or number”. See MPEP § 2106.04(a)(2)(I)(C).
“determining a weighted SMAPE for the long-term forecast of each trained machine learning model;” ; Determining a weighted symmetric mean absolute percentage for a long-term forecast of each trained machine learning model is an abstract idea of a mathematical calculation, as directed to “a claim that recites a mathematical calculation, when the claim is given its broadest reasonable interpretation in light of the specification, will be considered as falling within the "mathematical concepts" grouping. A mathematical calculation is a mathematical operation (such as multiplication) or an act of calculating using mathematical methods to determine a variable or number”. See MPEP § 2106.04(a)(2)(I)(C).
“and generating a weighted total SMAPE for each trained machine learning model.” ; Generating a weighted total SMAPE for each trained machine learning model is an abstract idea of a mathematical calculation, as directed to “a claim that recites a mathematical calculation, when the claim is given its broadest reasonable interpretation in light of the specification, will be considered as falling within the "mathematical concepts" grouping. A mathematical calculation is a mathematical operation (such as multiplication) or an act of calculating using mathematical methods to determine a variable or number”. See MPEP § 2106.04(a)(2)(I)(C).
Step 2A, Prong 2: The claims do not recite additional elements that integrate the judicial exception into a practical application:
“wherein the champion-challenger process includes: generating a short-term forecast with each trained machine learning model;” ; Generating a short-term forecast with a trained machine learning model only amounts to "apply it" and the mere instructions to implement an abstract idea on a computer - see MPEP 2106.05(f)(1) in addition to generally links the use of the abstract idea to a particular technological environment or field of use - See MPEP § 2106.05(h).
“generating a long-term forecast with each trained machine learning model;” ; Generating a long-term forecast with a trained machine learning model only amounts to "apply it" and the mere instructions to implement an abstract idea on a computer - see MPEP 2106.05(f)(1) in addition to generally links the use of the abstract idea to a particular technological environment or field of use - See MPEP § 2106.05(h).
Step 2B: The claims do not recite additional elements that amount to significantly more than the judicial exception. Generating a short-term forecast and a long-term forecast with a trained machine learning model amounts to "apply it" and mere instructions to implement an abstract idea on a computer. The claim fails to recite details of how a solution or outcome to a problem is accomplished because it is unclear how the "AI system" or "machine learning" is used nor does the specification make it clear how these actions are performed - see MPEP 2106.05(f)(1)). The usage of a machine learning model is generally linked to a particular technological environment or field of use (AI/ML) - see MPEP 2106.05(h).
Therefore, Claims 3, 10, and 17 are directed to non-statutory subject matter and rejected.
With respect to Claims 4, 11, and 18 which have identical claim limitations and are dependent on Claims 1, 8, and 15 respectively:
Step 2A, Prong 1: A judicial exception is recited in the claims as they recite mental processes, which are abstract ideas:
“wherein selecting the trained machine learning model from the plurality of trained machine learning models includes selecting the trained machine learning model using a static ruleset that describes one or more model inference performance constraints associated with the target storage system.” ; Selecting a trained machine learning model from a plurality of trained machine learning models using a static ruleset that describes model inference performance constraints is an abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion) -See MPEP § 2106.04(a)(2)(III).
Step 2A, Prong 2: The claims do not recite additional elements that integrate the judicial exception into a practical application:
Step 2B: The claims do not recite additional elements that amount to significantly more than the judicial exception.
Therefore, Claims 4, 11, and 18 are directed to non-statutory subject matter and rejected.
With respect to Claims 5, 12, and 19 which have identical claim limitations and are dependent on Claims 1, 8, and 15 respectively:
Step 2A, Prong 1: A judicial exception is not recited in the claims as they do not recite an abstract idea (mathematical concepts, certain methods of organizing human activity, or mental processes), law of nature, or natural phenomenon.
Step 2A, Prong 2: The claims do not recite additional elements that integrate the judicial exception into a practical application:
“receiving a model inference result associated with the deployment of the trained machine learning model on the target storage system.” ; Receiving a model inference result after the trained machine learning model has been deployed to the target storage system is considered insignificant extra-solution activity (post-solution activity) - see MPEP 2106.05(g).
Step 2B: Receiving a model inference result after the trained machine learning model has been deployed to the target storage system constitutes as an insignificant extra-solution activity, specifically a post-solution activity. - see MPEP 2106.05(g).
Therefore, Claims 5, 12, and 19 are directed to non-statutory subject matter and rejected.
With respect to Claims 6, 13, and 20 which have identical claim limitations and are dependent on Claims 5, 12, and 19 respectively:
Step 2A, Prong 1: A judicial exception is recited in the claims as they recite mental processes, which are abstract ideas:
“updating the trained machine learning model using the cloud computing system based upon, at least in part, the model inference result.” ; Updating the trained machine learning model based on the model inference result is an abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion) -See MPEP § 2106.04(a)(2)(III).
Step 2A, Prong 2: The claims do not recite additional elements that integrate the judicial exception into a practical application:
Step 2B: The claims do not recite additional elements that amount to significantly more than the judicial exception.
Therefore, Claims 6, 13, and 20 are directed to non-statutory subject matter and rejected.
With respect to Claims 7 and 14 which have identical claim limitations and are dependent on Claims 6 and 13 respectively:
Step 2A, Prong 1: A judicial exception is recited in the claims as they recite mental processes, which are abstract ideas:
“wherein updating the trained machine learning model includes detecting a drift in performance associated with the trained machine learning model.” ; Detecting a drift in performance upon updating the trained machine learning model is an abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion) -See MPEP § 2106.04(a)(2)(III).
Step 2A, Prong 2: The claims do not recite additional elements that integrate the judicial exception into a practical application:
Step 2B: The claims do not recite additional elements that amount to significantly more than the judicial exception.
Therefore, Claims 7 and 14 are directed to non-statutory subject matter and rejected.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or non-obviousness.
Claims 1, 8, and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Desai et. Al, (U.S PGPUB No. US 20200311573 A1, filed on August 26, 2019, hereinafter "Desai"), in view of Cady, (U.S PGPUB No. US 20220245485 A1, filed on February 04, 2021). Both disclosure dates are before the effective filing date of the instant application, i.e., October 11, 2023. Therefore, Desai and Cady constitute prior art under 35 U.S.C. 103.
With respect to Independent Claims 1, 8, and 15:
Desai teaches:
“generating a plurality of trained machine learning models using a cloud computing system…” (Paragraph [0030] discloses generating a plurality of trained machine learning models whilst collaborating with a cloud computing system, “For example, a server device may generate a model based on having trained the model in a manner similar to that described above and may provide the model to the cloud resource prediction platform…For example, the cloud resource prediction platform may perform a lookup of a model based on one or more characteristics of a customer and/or a task. In other words, the cloud resource prediction platform may utilize various models to make different predictions related to resource usage, thereby increasing an accuracy of making resource usage-related predictions.”)
“by training a plurality of machine learning models to forecast storage performance for one or more storage objects of a storage system,----------------------;” (Paragraph [0016] teaches training a plurality of machine learning models, “…a forecasting algorithm may train multiple time series forecasting models for each of the individual distributed storage systems.” Paragraph [0024] teaches training a machine learning model to forecast/determine storage performance (projected resource usage data), “The trained machine learning model may be utilized by the cloud resource prediction platform to determine projected resource usage data (e.g., related to resource usage by a new customer or new resource usage by an existing customer) based on a request for a new resource from a cloud computing environment (e.g., received from a client device associated with the new customer or the existing customer).” Paragraph [0073] further discloses forecasting storage performance for storage objects/storage system(s), “may determine, based on the historical cloud data and the historical customer data, usage deviation data indicating deviations between actual resource usage and planned resource usage of the cloud computing environment,”)
Desai does not appear to explicitly disclose:
“---------------- wherein the cloud computing system is separate from any storage system;”
“selecting a trained machine learning model from the plurality of trained machine learning models to deploy to a target storage system;”
“and deploying the trained machine learning model on the target storage system.”
However, Cady teaches:
“---------------- wherein the cloud computing system is separate from any storage system;” (Paragraph [0035] teaches forecasting storage performance for storage objects of a storage system (distributed storage devices), “In one embodiment, a forecasting approach implemented by the forecasting engine 124 represents a generalized solution to block capacity threshold forecasting for a distributed storage system and utilizes a best-model-candidate approach to account for and forecast various linear and nonlinear trends in block capacity usage for individual distributed storage devices in a field of distributed storage devices.” Paragraph [0049] further teaches the separation of the cloud computing system from the storage system, “According to one embodiment, in order to facilitate remote monitoring of multiple managed distributed storage systems, a cloud-based monitoring system (e.g., monitoring system 122) periodically pulls the time series telemetry data aggregated by the collector to provide a centralized data store from which the multiple managed distributed storage systems may be monitored remotely, individually or in various combinations.”) Examiner’s Note: See Fig. 1 for visual depiction of cloud computing system (120) and storage system(s) (135a-n) separation.
“selecting a trained machine learning model from the plurality of trained machine learning models to deploy to a target storage system;” (Paragraph [0017] teaches selecting a trained machine learning model to be deployed to a specified storage system, “The best performing time series forecasting model may be then independently selected for each of the multiple distributed storage systems based on performance metrics associated with the time series forecasting models. For example, as described further below, one of the trained machine-learning models may be selected based on their respective performance metrics.” )
“and deploying the trained machine learning model on the target storage system.” (Paragraph [0048] denotes a collaborative relationship between the machine learning model and the target storage system, “may vary from distributed storage system to distributed storage system and from model to model of the time series forecasting models (e.g., ML models 125a - y)…sufficient to produce desired forecasting accuracy for the worst-case combination of a particular distributed storage system and a particular time series forecasting model.” Paragraph [0080] further discloses that deployment of the model onto the target storage system (specified distributed storage system) has already taken place, “At block 570, a date may be forecasted at which a consumed block capacity threshold will be reached by the distributed storage system at issue based on the selected ML model”)
Desai and Cady are analogous art and in the same field of invention because both references pertain to predictive modeling approaches designed for capacity planning and resource management in tech environments. While Desai teaches projecting customer resource usage in a cloud-computing environment to process incoming allocation requests, Cady teaches predicting physical storage capacity limits and "when" a distributed storage system may run out of space. It would have been obvious to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to implement the base reference of Desai (optimizing resource allocation, scaling infrastructure dynamically without overpaying for idle capacity) with the teachings of Cady (forecasting data points to determine future timeframes when "fullness thresholds" will be crossed) in order to prevent system downtime and data loss by providing advance warnings to provision solutions. One of ordinary skill in the art would be motivated to do so because by integrating Cady's framework into the methods of Desai one would be able to note that a system implementation as such, "reduces or eliminates over usage of the resources, thereby conserving the resources. In addition, this reduces or eliminates over allocation of resources, thereby reducing instances of idle or unused resources and improving a utilization efficiency of the resources. Further, improving management of resources through improved anomaly detection facilitates improved cost management with regard to the resources and improved planning with regard to future resource needs, {[0013] of Desai}.”
Therefore, Claims 1, 8, and 15 are rejected.
Claims 2, 9, and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Desai, in view of Cady, further in view of Majumder, (U.S PGPUB No. US 20170132548 A1, filed on November 09, 2015). All disclosure dates are before the effective filing date of the instant application, i.e., October 11, 2023. Therefore, Desai, Cady and Majumder constitute prior art under 35 U.S.C. 103.
With respect to Claims 2, 9, and 16:
The combination of Desai and Cady does not appear to explicitly disclose:
“wherein selecting the trained machine learning model from the plurality of trained machine learning models includes performing a champion-challenger process with the plurality of trained machine learning models.”
However, Majumder teaches:
“wherein selecting the trained machine learning model from the plurality of trained machine learning models includes performing a champion-challenger process with the plurality of trained machine learning models.” (Paragraph [0018] teaches selecting a machine learning model dependent on the findings of a champion-challenger process taking place, “Depending on the events that activated trigger 208, model challenger can select models from the same family as the current model 202, such as from model family 212. In Such circumstances, model challenger may train and test each of the models, such as models 214, 216, 218, and 220 to select a model that most accurately aligns with the data input.” Paragraph [0020] further discloses selection based on a champion-challenger process, “Once model challenger 210 selects a new model, the newly selected model can replace the current model 202 and the new model can be used to make an updated prediction 206.”)
Desai, Cady, and Majumder are analogous art and in the same field of invention because all three references pertain to forecasting resource consumption, system capacity, and performance trends in complex computing environments. While Desai teaches using historical cloud data and customer resource requests to build a usage growth profile and identify deviations between planned and actual usage, Cady teaches cross-validating to find the most accurate model to predict future block storage capacities for specific distributed systems. Similarly, Majumder ensures that once a trigger (like a sudden deviation or shift in data) is detected, the system dynamically trains alternative models from multiple families on the new input data to improve and adapt the forecasting accuracy continuously. It would have been obvious to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to implement the base reference of Desai (proactive customer & cloud resource scaling) with the teachings of Cady (multi-model block capacity forecasting) and the teachings of Majumder (champion/challenger dynamic model switching) in order to predict future infrastructure demands against storage performance fluctuations based on volatile workloads to ensure forecasting models adapt on-the-fly to changing data characteristics to meet stakeholder needs. One of ordinary skill in the art would be motivated to do so because by integrating Cady and Majumder's frameworks into the methods of Desai one would be able to recognize that a system as such, "can be useful for ensuring appropriate amounts of parts and/or products have been ordered or are on hand, making decisions on price reductions, discounts, bundles, and the like, making changes to product pricing, or similar decisions to ensure customers’ orders can be fulfilled in a timely manner. Additionally, accurate sales forecasts can be useful for management when giving corporate guidance, setting company goals, or determining if new sales or marketing strategies are working (comparing forecast sales without the new strategy with actual sales with the new strategy), {[0031] of Majumder}."
Therefore, Claims 2, 9, and 16 are rejected.
Claims 3, 10, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Desai, in view of Cady, further in view of Majumder, further in view of Dar et. Al, (U.S PGPUB No. US 20230342280 A1, filed on April 22, 2022, hereinafter "Dar"), further in view of Rahimi et. Al, (U.S PGPUB No. US 20240311650 A1, filed on March 13, 2023, hereinafter "Rahimi"). All disclosure dates are before the effective filing date of the instant application, i.e., October 11, 2023. Therefore, Desai, Cady, Majumder, Dar, and Rahimi constitute prior art under 35 U.S.C. 103.
With respect to Claims 3, 10, and 17:
The combination of Desai-Cady-Majumder does not appear to explicitly disclose the extent of this claim set alone:
However, Dar teaches:
“wherein the champion-challenger process includes: generating a short-term forecast with each trained machine learning model;” (Paragraph [0079] teaches generating a short-term forecast with each trained machine learning model, “for the one or more storage objects using the at least one trained IO modeling system from the subset of the plurality of IO modeling systems…For example, performance modeling process 10 may forecast both the short-term and the long-term analysis of the IO performance data where the short-term analysis may provide the forecast for e.g., the next seven days while the long-term analysis may provide forecast results for e.g., the next year.” Paragraph [0082] further discloses generating a short-term forecast, “Referring again to FIG. 12, the storage system (e.g., storage system 12) is forecasted by IO modeling system 416 to cross the headroom value (i.e., the generally straight line on both long-term IO performance data 1116 and short-term IO performance data 1118) periodically, indicating that there might be a need for remedial action (e.g., storage administrator intervention, notifications provided, storage system properties automatically adjusted, etc.) to avoid low system performance or costly downtime.”)
“generating a long-term forecast with each trained machine learning model;” (Paragraph [0079] teaches generating a long-term forecast with each trained machine learning model, “for the one or more storage objects using the at least one trained IO modeling system from the subset of the plurality of IO modeling systems…For example, performance modeling process 10 may forecast both the short-term and the long-term analysis of the IO performance data where the short-term analysis may provide the forecast for e.g., the next seven days while the long-term analysis may provide forecast results for e.g., the next year.”)
“determining a weighted symmetric mean absolute percentage (SMAPE) for the short-term forecast of each trained machine learning model;” (Paragraph [0071] teaches determining a SMAPE for the short-term forecast of each trained ML model, “The modeling performance information may include a sMAPE value measuring the accuracy of the IO modeling system as additional portions of the historical IO performance data are used for training and testing the IO modeling system. Accordingly, performance modeling process 10 may determine 306 a forecast score for each IO modeling system by determining 324 a median value of the sMAPE across the historical IO performance data (i.e., the median value of the sMAPE values determined across the historical IO performance data).” Paragraph [0072] further discloses that the sMAPE is weighted, “For example, optional weights applied to the modeling performance information may be referred to as "momentum" weighting. Momentum weighting allows the IO modeling system to consider more heavily forecasting errors encountered when evaluating the IO modeling system's performance on the last test.” Paragraph [0074] further denotes that the generation of this weighted sMAPE can be for short-term and long-term forecasts (particular period of time in the future), “In some implementations, performance modeling process 10 may determine 326 a sMAPE standard deviation for each IO modeling system. For example, as performance modeling process 10 trains 318 and tests 320 an IO modeling system over the historical IO performance data, the accuracy may change over time. Referring again to FIG. 5, suppose that performance modeling process 10 trains 318 IO modeling system 416 with historical IO performance data portions 402 and 404. In this example, IO modeling system 416 may be tested 320 by forecasting the IO performance for a particular period of time in the future.”) Examiner’s Note: Under BRI, “particular period of time in the future/different periods of time” pertains to a short-term time interval and a long-term time interval for forecasting [0070].
“determining a weighted SMAPE for the long-term forecast of each trained machine learning model;” (Examiner’s Note: See citing for previous limitation. Also applies to long-term forecasting)
Dar does not appear to explicitly disclose:
“and generating a weighted total SMAPE for each trained machine learning model.”
However, Rahimi teaches:
“and generating a weighted total SMAPE for each trained machine learning model.” (Paragraph [0063] teaches that weighted mean absolute percentage error is generated for a trained ML model, “Each test may generate a weighted mean absolute percentage error (WMAPE) score for each time series for each machine learning model, where each WMAPE score may be normalized according to the highest WMAPE score for that specific time series input.” Paragraph [0063] further states that additional testing alternatives and combinations such as a (weighted total SMAPE) can also be generated for a trained ML model, “It should be appreciated that alternative or additional techniques may be used to assess the performance of the machine learning models included in the model search space 216. In particular, although the use of WMAPE is described, it should be appreciated that additional or alternative metrics or techniques for testing the time series data are envisioned (e.g., mean absolute error (MAE), root mean squared error (RMSE), symmetric mean absolute percentage error (SMAPE), and/or others).”)
Desai-Cady-Sarkar-Dar-Rahimi are analogous art and in the same field of invention because all five references pertain to intelligently managing infrastructure performance, preventing bottlenecks, and anticipating future capacity needs without human intervention. While Desai teaches predicting future customer-driven resource needs in a cloud environment, Cady teaches training multiple ML models to predict future storage capacity and prevent system-wide outages. Similarly, Sarkar teaches performing parallel storage operations while simultaneously triggering a model server to act upon it, while Dar teaches diagnosing systemic slowdowns. Additionally, Rahimi teaches utilizing a classifier model to assess and score which available ML models is best suited to analyze the data and generate a time-series forecast. It would have been obvious to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to implement the base reference of Desai (cloud resource projections) with the teachings of Cady (distributed storage telemetry analysis) with the teachings of Sarkar (real-time parallel AI evaluation) further with the teachings of Dar (historical I/O performance processing) and further with the teachings of Rahimi (time series forecast classification) in order to monitor, predict, and optimize storage performance, data placement, and resource allocation in complex computing systems. One of ordinary skill in the art would be motivated to do so because by integrating Cady, Sarkar, Dar and Rahimi’s frameworks into the methods of Desai one would be able to recognize that a system as such can, "employ various techniques to improve forecasting accuracy, such as spike adjustment to improve forecasts for outliers, {[0078] of Rahimi}."
Therefore, Claims 3, 10, and 17 are rejected.
Claims 4, 5, 6, 11, 12, 13, 18, 19, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Desai, in view of Cady, further in view of Sarkar, (U.S PGPUB No. US 20210224684 A1, filed on January 17, 2020). All disclosure dates are before the effective filing date of the instant application, i.e., October 11, 2023. Therefore, Desai, Cady, and Sarkar constitute prior art under 35 U.S.C. 103.
With respect to Claims 4, 11, and 18:
The combination of Desai and Cady does not appear to explicitly disclose:
“wherein selecting the trained machine learning model from the plurality of trained machine learning models includes selecting the trained machine learning model using a static ruleset that describes one or more model inference performance constraints associated with the target storage system.”
However, Sarkar teaches:
“wherein selecting the trained machine learning model from the plurality of trained machine learning models includes selecting the trained machine learning model using a static ruleset that describes one or more model inference performance constraints associated with the target storage system.” (Paragraph [0055] details that inference-specific training requests on selected models can be influenced/determined/validated by specified rulesets/constraints (inference scores) before deployment to the target storage system (primary storage system), “validator 250 may include functions, parameters, and/or data structures for applying validation data elements from validation data 226 to the newly generated model and comparing a resulting reliability value, such as an inference score, against a model validity threshold value. Measuring the reliability value based on a known validation dataset and comparing it to the model validity threshold may enable validator 250 to determine whether the threshold is met and the model instance appears valid or the threshold is not met and retraining should be initiated using different parameters for AI framework 242 and/or a different training data set. In some embodiments, validation data 226 for use by validator 250 may be curated by a user and/or generated from predetermined data sets with validated inference values and stored to primary storage system 220 from any other source…” Paragraph [0059] further teaches selecting a model based on static rulesets (functions/parameters), “For example, inference engine 264 may include functions, parameters, and/or data structures for using the model instance selected by model selector 262 to process a target data element and generate one or more inference values, including an inference score that may be used to evaluate the quality or validity of inference values generated by inference engine 264.” Paragraph [0140] further teaches that the model inference performance constraint (model validity threshold) determines the selection of the ML model that will be best suited for that instance of the target storage system (primary storage system), “…the inference score may be evaluated against a model validity threshold. For example, a deployment manager may receive the inference score and compare the inference score to a model validity threshold.”)
Desai, Cady, and Sarkar are analogous art and in the same field of invention because all three references pertain to utilizing historical and real-time data to analyze, optimize, or predict storage performance and capacity, ultimately preventing system bottlenecks. While Desai teaches predicting future capacity strain, Cady teaches processing massive time-series telemetry data with timestamps to train multiple machine learning models. Similarly, Sarkar ensures bridging the gap between raw data storage and computational analysis by removing lag and storage overheads during active model training. It would have been obvious to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to implement the base reference of Desai (cloud resource & deviation forecasting) with the teachings of Cady (storage telemetry & capacity forecasting) and the teachings of Sarkar (AI/ML workload & parallel storage management) in order to integrate ML models directly into the storage environment using it to forecast block capacity, project resource growth, and evaluate live data parallel to primary storage operations. One of ordinary skill in the art would be motivated to do so because by integrating Cady and Sarkar's frameworks into the methods of Desai one would be able to recognize that a system as such can, "improve resource efficiency, flexibility, and scalability, {[0024] of Sarkar}."
Therefore, Claims 4, 11, and 18 are rejected.
With respect to Claims 5, 12, and 19:
The combination of Desai and Cady does not appear to explicitly disclose:
“receiving a model inference result associated with the deployment of the trained machine learning model on the target storage system.”
However, Sarkar teaches:
“receiving a model inference result associated with the deployment of the trained machine learning model on the target storage system.” (Paragraph [0128] discloses a model inference result (acceptable inference score) being utilized to determine post-deployment operations within the target storage system (primary storage system), “For example, a model server or deployment manager may determine that inferences generated by a model instance are no longer generating an acceptable inference score (e.g., application to new data elements is generating inference scores below the model validity threshold) and send a retraining request to the model trainer and/or data preparer. The model trainer may directly or indirectly (e.g., from the data preparer or a system administrator) receive the retraining request.” Paragraph [0041] also depicts the acquirement of a model inference result associated with the deployment of the trained model, “In some embodiments, server systems 150 may host model server 130, such as a model serving application used by system 100 to serve inferences to clients 102 in the form of inference data, such as inference scores and inference metadata associated with specific data elements and/or aggregations thereof. For example, model server 130 may access trained models stored in primary storage pool 160 to deploy those models for production use in generating real time insights based on new data elements received by system 100.”) Examiner’s Note: [0047] denotes deploying trained models onto the target storage system.
Therefore, Claims 5, 12, and 19 are rejected.
With respect to Claims 6, 13, and 20:
The combination of Desai and Cady do not appear to explicitly disclose:
“updating the trained machine learning model using the cloud computing system based upon, at least in part, the model inference result.”
However, Sarkar teaches:
“updating the trained machine learning model using the cloud computing system based upon, at least in part, the model inference result.” (Paragraph [0067] teaches updating/retraining the trained ML model depending on the inference result, “to determine whether a valid inference has been generated by the application of a model instance from trained models 338 to new data element 302. For example, if the highest inference score for a new data element or data type is less than the inference validity threshold (e.g. , 80 %), system 300 may identify the inference (or inferences) as invalid and reconsider the validity of the model instance for that type of data. If the inference score does not meet the inference validity threshold, then a retraining request 356 may be generated by inference evaluator 346 and sent to data preparer 328 and/or model trainer 332 to initiate retraining to generate a new model instance.” Paragraph [0043] further teaches that the updating of the trained ML model can be performed using the cloud computing system and resulted inferences, “in some embodiments, some or all components of system 100 may be configured as a cloud computing system to provide Al or deep learning inferences as a service. In some embodiments, the cloud configuration may extend from ingest 10 at data ingester 122 to the inferences 60 generated by model server 130. Clients 102 may be end users of the inferences generated by the system or may themselves be part of the cloud computing configuration to provide one or more end user applications that utilize the inferences.”) Examiner’s Note: distributed computing system 300 may be configured similarly to system 200 [0061] which may also be configured similarly to system 100 [0044] cited.
Therefore, Claims 6, 13, and 20 are rejected.
Claims 7 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Desai, in view of Cady, further in view of Sarkar, further in view of Dar et. Al, (U.S PGPUB No. US20230342280A1, filed on April 22, 2022, hereinafter "Dar"). All disclosure dates are before the effective filing date of the instant application, i.e., October 11, 2023. Therefore, Desai, Cady, Sarkar and Dar constitute prior art under 35 U.S.C. 103.
With respect to Claims 7 and 14:
The inherited combination of Desai-Cady-Sarkar does not appear to explicitly disclose:
“wherein updating the trained machine learning model includes detecting a drift in performance associated with the trained machine learning model.”
However, Dar teaches:
“wherein updating the trained machine learning model includes detecting a drift in performance associated with the trained machine learning model.” (Paragraph [0086] teaches updating/retraining the ML model based on a performance drift, “In some implementations, performance modeling process 10 may determine 332 whether to retrain the at least one trained IO modeling system by comparing the data drift ratio and/or the performance degradation value to one or more predefined thresholds. For example, performance modeling process 10 may utilize one or more predefined thresholds to determine whether or not to retain the at least one trained IO modeling system and/or to determine the degree of retraining required... Similarly, performance modeling process 10 may determine 332 that the at least one IO modeling system needs to be completely retrained when a profound data drift ratio is determined, or a major performance degradation is identified.”)
Desai-Cady-Sarkar-Dar are analogous art and in the same field of invention because all four references pertain to transitioning storage infrastructure from a reactive state to a predictive and automated one by leveraging AI. While Desai teaches forecasting future block capacity and resource needs, ensuring clusters don't run out of disk space, Cady teaches infrastructure monitoring by ensuring accuracy across different hardware nodes. Similarly, Sarkar ensures bridging the gap between raw data storage and computational analysis by removing lag and storage overheads during active model training, while Dar teaches isolating past latency issues to optimize overall storage performance by processing historical input/output data. It would have been obvious to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to implement the base reference of Desai (cloud resource projections) with the teachings of Cady (distributed block capacity) with the teachings of Sarkar (real-time parallel AI evaluation) further with the teachings of Dar (historical I/O performance) in order to improve efficiency, proactively manage resource allocation, and accelerate intelligent operations in storage and cloud environments. One of ordinary skill in the art would be motivated to do so because by integrating Cady, Sarkar, and Dar’s frameworks into the methods of Desai one would be able to recognize that a system as such can, "improve the process of forecasting IO performance data for new storage systems and/or storage objects in terms of processing speed and time, {[0087] of Dar}."
Therefore, Claims 7 and 14 are rejected.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NOOR F CHEEMA whose telephone number is (571)272-9642. The examiner can normally be reached Monday-Friday 7:30am-5:00pm alternative Fridays off.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Mariela Reyes can be reached at (571) 270-1006. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/N.F.C./ Examiner, Art Unit 2142 /Mariela Reyes/Supervisory Patent Examiner, Art Unit 2142
Prosecution Insights