DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is responsive to the application filed 2/22/2023.
Claims 1-20 are presented for examination.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 2, 12, and 17 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding dependent claims 2, 12, and 17 (line 12 in each claim), reciting in part “selecting the hyperparameter configuration”, it is unclear what the scope of “the hyperparameter configuration” is as it is unclear the selecting is to be performed on the set of hyperparameter configurations or not, or something else. Thus, claims 2, 12, and 17 are indefinite. For the purposes of examination, the said unclear limitations are interpreted as “selecting a hyperparameter configuration”.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 16-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory matter.
Regarding claims 16-20 are directed to “tangible machine-readable storage medium”. However, the disclosure within the Applicant’s specification does not provide a specific definition for the recited “tangible machine-readable storage medium” only that it may encompass “e.g., a non-transitory storage medium” within paragraph [0164]. Based on the broadest reasonable interpretation, the recited “tangible machine-readable storage medium” does not need to be restricted to only non-transitory storage medium and consequently also encompasses non-statutory subject matter, e.g. propagating signals. Per MPEP 2106.03(II) “A claim whose BRI covers both statutory and non-statutory embodiments embraces subject matter that is not eligible for patent protection and therefore is directed to non-statutory subject matter”. Additionally, per MPEP 2106.03(I) “Even when a product has a physical or tangible form, it may not fall within a statutory category. For instance, a transitory signal, while physical and real, does not possess concrete structure that would qualify as a device or part under the definition of a machine, is not a tangible article or commodity under the definition of a manufacture (even though it is man-made and physical in that it exists in the real world and has tangible causes and effects), and is not composed of matter such that it would qualify as a composition of matter”. Consequently, “tangible machine-readable storage medium” recited by claims 16-20 do not fall within at least one of the four categories of patent eligible subject matter and are patent ineligible.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The analysis of the claims will follow the 2019 Revised Patent Subject Matter Eligibility Guidance, 84 Fed. Reg. 50 (“2019 PEG”).
Claim 1
Step 1: The claim recites “A computer-implemented method comprising:”; therefore, it is directed to the statutory category of a process.
Step 2A Prong 1: The claim recites, inter alia:
selecting a machine learning (ML) model for predicting future values of a time series for a metric: These limitations recite a mentally performable process of using judgement to select a machine learning model observed for predicting future values of a time series for a metric.
forecasting values of the metric for a forecast period: These limitations recite a mentally performable process of using judgement to forecast values of the metric for an observed forecast period.
comparing the actual values to the forecasted values; determining an anomaly in a behavior of the metric based on the comparison: These limitations recite a mentally performable process of using judgement to compare the observed actual values to the observed forecasted values and using judgement to determine an anomaly in an observed behavior of the metric based on the comparison.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The additional elements of the claim are as follows:
A computer-implemented method comprising…using the ML model: These additional elements are recited at a high level of generality and merely represent generic computer machinery performing in their ordinary capacity to implement the underlying judicial exception. See MPEP 2106.05(f).
collecting actual values of the metric during the forecast period;…and causing presentation in a computer user interface (UI) of the anomaly: These additional limitations represent insignificant extra solution activity of data gathering and output by collecting actual values of the metric during the forecast period and causing presentation in a computer user interface (UI) of the anomaly. See MPEP 2106.05(g).
Thus, the way in which the additional elements use or interact with the judicial exception do not integrate the judicial exception into a practical application.
Step 2B: The additional elements from Step 2A Prong 2 include invoking computers or other machinery to apply the underlying judicial exception and insignificant extra-solution activity of data gathering and output recited by “collecting actual values of the metric during the forecast period;…and causing presentation in a computer user interface (UI) of the anomaly” which are considered well-understood, routine, and conventional activities similar to presenting offers and gathering statistics see MPEP 2106.05(d) (II). Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself. See MPEP 2106.05.
Claim 2
Step 1: a process, as in claim 1.
Step 2A Prong 1: The claim recites, inter alia:
wherein selecting the ML model further comprises: testing a first model with several hyperparameter configurations, the testing of the first model with one of the hyperparameter configurations comprising: selecting values for one or more hyperparameters of the first model: These limitations further the mentally performable process with aid of pen and paper of selecting the ML model to further comprise using judgement to test a design of a first model with one of the hyperparameter configurations comprising selecting values for one or more hyperparameters of the first model.
and calculating an accuracy of the first model, using validation data, for the selected values for the one or more hyperparameters: These limitations recite a mathematical calculation of an accuracy of the first model, using validation data, for the selected values for the one or more hyperparameters.
and selecting the hyperparameter configuration with a highest accuracy: These limitations recite a mentally performable process of using judgement to select the hyperparameter configuration deemed to have a highest accuracy.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The additional elements of the claim are as follows:
training the first model with the selected values: These additional elements are mere instructions to implement the judicial exception because the additional elements only recite the idea of training the first model with the selected values but fail to recite any details of how training with the selected values is accomplished, e.g. is this supervised/unsupervised/hybrid training with portions of the selected data as validation, etc. See MPEP 2106.05(f).
Thus, the way in which the additional elements use or interact with the judicial exception do not integrate the judicial exception into a practical application.
Step 2B: The additional elements from Step 2A Prong 2 include adding the words “apply it” (or equivalent) with the judicial exception. Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself. See MPEP 2106.05.
Claim 3
Step 1: a process, as in claim 1.
Step 2A Prong 1: The claim recites, inter alia:
and inserting the obtained data in the time series of the metric: These limitations recite a mentally performable process with aid of pen and paper of using judgement to insert the collected data in the observed time series of the metric.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The additional elements of the claim are as follows:
wherein collecting actual values comprises: obtaining data for the time series of the metric received via logs or metrics data: These additional elements are recited at a high level of generality and amounts to no more than generally linking the use of the judicial exception, i.e. inserting the obtained data in the observed time series of the metric, to a particular technological environment wherein collecting actual values comprises: obtaining data for the time series of the metric received via logs or metrics data. See MPEP 2106.05(h).
Thus, the way in which the additional elements use or interact with the judicial exception do not integrate the judicial exception into a practical application.
Step 2B: The additional elements from Step 2A Prong 2 include generally linking the use of the judicial exception to a particular field of use or technological environment. Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself. See MPEP 2106.05.
Claim 4
Step 1: a process, as in claim 1.
Step 2A Prong 1: The claim recites, inter alia:
wherein comparing the actual values with the forecasted values comprises: calculating, for each time value in the time series of the metric, a difference between the forecasted value of the metric and the actual value of the metric: These limitations furthers the mentally performable process of comparing the observed actual values with the forecasted values to include added recitation of mathematical calculations, for each time value in the time series of the metric, a difference between the forecasted value of the metric and the actual value of the metric.
Step 2A Prong 2 & Step 2B: There are no additional elements recited so the claim does not provide a practical application and is not considered to be significantly more. As such, the claim is patent ineligible.
Claim 5
Step 1: a process, as in claim 1.
Step 2A Prong 1: The claim recites, inter alia:
wherein determining the anomaly further comprises: determining that an anomaly has occurred when a difference between the forecasted values and the actual values is above a predetermined threshold for a period greater than a predetermined time threshold: These limitations recite a mathematical relationship that is required when determining the anomaly has occurred via a difference between the forecasted values and the actual values is above a predetermined threshold for a period greater than a predetermined time threshold.
Step 2A Prong 2 & Step 2B: There are no additional elements recited so the claim does not provide a practical application and is not considered to be significantly more. As such, the claim is patent ineligible.
Claim 6
Step 1: a process, as in claim 1.
Step 2A Prong 1: The claim recites the same abstract ideas as in claim 1.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The additional elements of the claim are as follows:
wherein causing presentation in the UI further comprises: presenting in the UI a graph of a time series of the actual values and a time series of the forecasted values: These additional elements are recited at a high level of generality and amounts to no more than generally linking the use of the judicial exception to a particular technological environment wherein collecting actual values comprises: obtaining data for the time series of the metric received via logs or metrics data. See MPEP 2106.05(h).
Thus, the way in which the additional elements use or interact with the judicial exception do not integrate the judicial exception into a practical application.
Step 2B: The additional elements from Step 2A Prong 2 include generally linking the use of the judicial exception to a particular field of use or technological environment. Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself. See MPEP 2106.05.
Claim 7
Step 1: a process, as in claim 1.
Step 2A Prong 1: The claim recites the same abstract ideas as in claim 1.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The additional elements of the claim are as follows:
wherein the ML model is selected from a group comprising an AR model and a SARMA model, the AR model having a lags hyperparameter, the SARMA model having hyperparameters comprising a trend autoregressive order, a trend difference order, a trend moving average order, a number of time steps for a single seasonal period, a seasonal autoregressive order, a seasonal differencing order, and a seasonal moving average order: These additional elements are recited at a high level of generality and amounts to no more than generally linking the use of the judicial exception to a particular technological environment. See MPEP 2106.05(h).
Thus, the way in which the additional elements use or interact with the judicial exception do not integrate the judicial exception into a practical application.
Step 2B: The additional elements from Step 2A Prong 2 include generally linking the use of the judicial exception to a particular field of use or technological environment. Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself. See MPEP 2106.05.
Claim 8
Step 1: a process, as in claim 7.
Step 2A Prong 1: The claim recites, inter alia:
wherein selecting the ML model further comprises: utilizing gradient search to select hyperparameter values for the ML model: These limitations recite furthering the mentally performable process of selecting the ML model to included added recitation of mathematical calculations required to utilizing gradient search to select hyperparameter values for the ML model.
Step 2A Prong 2 & Step 2B: There are no additional elements recited so the claim does not provide a practical application and is not considered to be significantly more. As such, the claim is patent ineligible.
Claim 9
Step 1: a process, as in claim 1.
Step 2A Prong 1: The claim recites the same abstract ideas as in claim 1.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The additional elements of the claim are as follows:
wherein the anomaly is one of change detection, slow drift, sudden change from zero, sudden change to zero, or transient spike: These additional elements are recited at a high level of generality and amounts to no more than generally linking the use of the judicial exception to a particular technological environment. See MPEP 2106.05(h).
Thus, the way in which the additional elements use or interact with the judicial exception do not integrate the judicial exception into a practical application.
Step 2B: The additional elements from Step 2A Prong 2 include generally linking the use of the judicial exception to a particular field of use or technological environment. Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself. See MPEP 2106.05.
Claim 10
Step 1: a process, as in claim 1.
Step 2A Prong 1: The claim recites the same abstract ideas as in claim 1.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The additional elements of the claim are as follows:
wherein the ML model is configured to detect seasonalities in training data to forecast the values of the metric: These additional elements are mere instructions to implement the judicial exception because the additional elements only recite the idea of configuring the ML model to detect seasonalities in the training data to forecast the values of the metric but fail to recite any details of how configuring the ML model to detect seasonalities in the training data to forecast the values of the metric is accomplished, e.g. how many samples are needed to detect a plurality of seasonalities, what are the different seasonalities detected, how are the forecast values of the metric related to each or the set/subset of seasonalities, etc. See MPEP 2106.05(f).
Thus, the way in which the additional elements use or interact with the judicial exception do not integrate the judicial exception into a practical application.
Step 2B: The additional elements from Step 2A Prong 2 include adding the words “apply it” (or equivalent) with the judicial exception. Thus, the additional elements, viewed individually or in combination, do not provide an inventive concept or otherwise amount to significantly more than the abstract idea itself. See MPEP 2106.05.
Claims 11-15
Step 1: These claims are directed to “A system comprising:”; therefore, these claims are directed to the statutory category of machines.
Step 2A Prong 1: These claims recite the same abstract ideas as in claims 1-5, respectively.
Step 2A Prong 2: The judicial exceptions recited in these claims are not integrated into a practical application. The only difference between claims 11-15 and claims 1-5 is that claims 11-15 are directed to “A system comprising: a memory comprising instructions; and one or more computer processors, wherein the instructions, when executed by the one or more computer processors, cause the system to perform operations comprising:”. However, mere recitation that a judicial exception is to be performed using generic computer equipment in their ordinary capacity, i.e. a system comprising: a memory comprising instructions; and one or more computer processors, wherein the instructions, when executed by the one or more computer processors, cause the system to perform operations, cannot meaningfully integrate the judicial exception into a practical application. See MPEP 2106.05(f). With that exception, the analysis at this step mirrors that of claims 1-5, respectively.
Step 2B: These claims do not contain significantly more than the judicial exception. The only difference between claims 11-15 and claims 1-5 is that claims 11-15 are directed to “A system comprising: a memory comprising instructions; and one or more computer processors, wherein the instructions, when executed by the one or more computer processors, cause the system to perform operations comprising:”. However, mere recitation that a judicial exception is to be performed using generic computer equipment in their ordinary capacity, i.e. a system comprising: a memory comprising instructions; and one or more computer processors, wherein the instructions, when executed by the one or more computer processors, cause the system to perform operations, cannot amount to significantly more than the judicial exception. See MPEP 2106.05(f). With that exception, the analysis at this step mirrors that of claims 1-5, respectively.
Claims 16-20
Step 1: These claims are directed to “A tangible machine-readable storage medium including instructions that, when executed by a machine, cause the machine to perform operations comprising:”; while the “tangible machine-readable storage medium” claimed are directed to non-statutory subject matter as set forth in the 35 U.S.C. 101 rejections above, the recommended amendment of “non-transitory machine -readable storage medium” would appear to direct these claims to the statutory category of an article of manufacture and, per MPEP 2106.03(II), the 2019 PEG analysis proceeds to determine whether such amended claims would quality as patent eligible.
Step 2A Prong 1: These claims recite the same abstract ideas as in claims 1-5, respectively.
Step 2A Prong 2: The judicial exceptions recited in these claims are not integrated into a practical application. The only difference between claims 16-20 and claims 1-5 is that claims 16-20 are directed to “A tangible machine-readable storage medium including instructions that, when executed by a machine, cause the machine to perform operations comprising:”. However, mere recitation that a judicial exception is to be performed using generic computer equipment in their ordinary capacity, i.e. a tangible machine-readable storage medium including instructions that, when executed by a machine, cause the machine to perform operations, cannot meaningfully integrate the judicial exception into a practical application. See MPEP 2106.05(f). With that exception, the analysis at this step mirrors that of claims 1-5, respectively.
Step 2B: These claims do not contain significantly more than the judicial exception. The only difference between claims 16-20 and claims 1-5 is that claims 16-20 are directed to “A tangible machine-readable storage medium including instructions that, when executed by a machine, cause the machine to perform operations comprising:”. However, mere recitation that a judicial exception is to be performed using generic computer equipment in their ordinary capacity, i.e. a tangible machine-readable storage medium including instructions that, when executed by a machine, cause the machine to perform operations, cannot amount to significantly more than the judicial exception. See MPEP 2106.05(f). With that exception, the analysis at this step mirrors that of claims 1-5, respectively.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 4-6, 10-11, 14-16, and 19-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Mathis, US 2014/0108640 A1.
Regarding independent claim 1, Mathis discloses a computer-implemented method comprising ([0032] process in FIG. 4 performed by anomaly detector): selecting a machine learning (ML) model for predicting future values of a time series for a metric ([0033] model selection process in FIG. 5 generates a predictive model applying historical data (selecting a machine learning model) to predict expected upper and lower bounds calculated (for predicting future values) for a next time step of a specified segment involving forecasted value, standard error and a confidence (for a metric) of the time series data (of a time series)); forecasting, using the ML model, values of the metric for a forecast period ([0033] predicting using the generated predictive model that applies historical data (forecasting using the ML model) the calculated upper and lower bounds (values) of the specified segment involving forecasted value, standard error and a confidence (of the metric) of next time step of the time series data (for a forecast period)); collecting actual values of the metric during the forecast period ([0033] during a time step processing during the prediction error (of the metric during the forecast period) is updated (collecting) that squares the difference between the forecasted and actual value (actual values)); comparing the actual values to the forecasted values ([0033] rolling error squares the difference between the forecasted and actual value); determining an anomaly in a behavior of the metric based on the comparison ([0034] a determination is made if the actual value is outside the expected range to determine whether the actual value is an anomalous value); and causing presentation in a computer user interface (UI) of the anomaly ([0037] an anomaly indication is displayed on display screen of user interface 222).
Regarding dependent claim 4, Mathis discloses the method as recited in claim 1, wherein comparing the actual values with the forecasted values ([0033] rolling error squares the difference between the forecasted and actual value) comprises: calculating, for each time value in the time series of the metric, a difference between the forecasted value of the metric and the actual value of the metric ([0033] for each time step for a specified segment of the time series data, the forecasting module 270 will update a rolling standard error calculated with square difference between the forecasted and actual value).
Regarding dependent claim 5, Mathis discloses the method as recited in claim 1, wherein determining the anomaly further comprises ([0036] In another example, the same system may be configured to detect an anomaly): determining that an anomaly has occurred when a difference between the forecasted values and the actual values is above a predetermined threshold for a period greater than a predetermined time threshold ([0036] detect an anomaly when values exceed (and the actual values is above) a range, e.g. upper or lower bound predicted value range, (determining that an anomaly has occurred when a difference between the forecasted values a predetermined threshold) for two consecutive time values of the time window with a number of time periods (for a period greater than a predetermined time threshold, e.g. one time value)).
Regarding dependent claim 6, Mathis discloses the method as recited in claim 1, wherein causing presentation in the UI further comprises: presenting in the UI a graph of a time series of the actual values and a time series of the forecasted values ([0029] FIG. 3 the Correlations button will direct the software system to display an illustration of correlated metrics or allow a user to perform correlative activity (wherein causing presentation in the UI further comprises) such as illustration in 306 and 308 of FIG. 3 (presenting in the UI a graph) solid line extending from Fri 02 until Thu 08 connects the actual values of the same metric (of a time series of the actual values) and the highlighted area extending above and below the predicted value line represents a range between an upper bound and a lower bound around the expected value (and a time series of the forecasted values)).
Regarding dependent claim 10, Mathis discloses the method as recited in claim 1, wherein the ML model is configured to detect seasonalities in training data to forecast the values of the metric ([0039-0040] model configuration to handle cycles contrasting moving average analysis with time series analysis, noting that the latter is better suited because "automated time series analysis takes cycles into consideration by applying a mathematical model that represents such cycles", specific adjustment for cycles with process designed to handle seasonality, allowing it to "identify when weekend volume is abnormally high or low compared to most other weekends while ignoring weekend volume changes that are more closely tied to regular cyclical changes". Furthermore, time series analysis "adjusts the upper and lower bounds according to recognized cyclical behavior").
Regarding claims 11 and 14-15, these claims are system claims that are substantially the same as the computer-implemented method of claims 1 and 4-5, respectively. Thus, claims 11 and 14-15 are rejected for the same reasons as claims 1 and 4-5. Additionally, Mathis discloses a system comprising ([0052] one or more computer systems including): a memory comprising instructions ([0055] System memory 720 configured to store program instructions); and one or more computer processors ([0052] processors 710), wherein the instructions, when executed by the one or more computer processors, cause the system to perform operations comprising ([0052], [0055], [0059] program instructions when executed by processors cause computer system 700 to implement embodiments).
Regarding claims 16 and 19-20, these claims are tangible machine-readable storage medium claims that are substantially the same as the method of claims 1 and 4-5, respectively. Thus, claims 16 and 19-20 are rejected for the same reasons as claims 1 and 4-5. Additionally, Mathis discloses a tangible machine-readable storage medium including instructions ([0062] storage media or computer media storing instructions) that, when executed by a machine, cause the machine to perform operations comprising ([0052], [0055], [0059] program instructions when executed by computer system processor cause computer system to implement embodiments).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 2, 12 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Mathis, as applied in the rejections of claims 1, 11, and 16 above, in view of Yu et al. (hereinafter Yu), US 2018/0121814 A1.
Regarding dependent claim 2, Mathis teaches the method as recited in claim 1, wherein selecting the ML model further comprises ([0038] model generation to produce a model (wherein selecting the ML model) further comprise FIG. 5): testing a first model with several hyperparameter configurations ([0038] the system alter type of model (testing a first model) and/or model coefficients as other segments of the data are considered (with several hyperparameter configurations)).
Mathis does not expressly teach the testing of the first model with one of the hyperparameter configurations comprising: selecting values for one or more hyperparameters of the first model; training the first model with the selected values; and calculating an accuracy of the first model, using validation data, for the selected values for the one or more hyperparameters; and selecting the hyperparameter configuration with a highest accuracy (interpreted as and selecting a hyperparameter configuration with a highest accuracy per the 35 U.S.C. 112(b) rejection set forth above).
However Yu teaches testing of a first model with one of hyperparameter configurations ([0021-0022] a predictive learning model 130 with hyperparameter value sets is compared) comprising: selecting values for one or more hyperparameters of the first model ([0022] identifying the hyperparameter value sets of the predictive learning model 130); training the first model with the selected values ([0022] the predictive model’s hyperparameter value sets are evaluated against a training model); and calculating an accuracy of the first model, using validation data, for the selected values for one or more hyperparameters ([0022], [0031], [0042-0047] calculating the error and variance and uses validation error y in calculating a marginal negative logarithmic likelihood for the process hyperparameters); and selecting a hyperparameter configuration with a highest accuracy ([0022] the most accurate hyperparameter value sets are selected as the hyperparameter value sets to be analyzed in the next round).
Because Yu and Mathis address the issue of comparing the predictive model parameters iteratively, accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of testing of a first model with one of hyperparameter configurations comprising: selecting values for one or more hyperparameters of the first model; training the first model with the selected values; and calculating an accuracy of the first model, using validation data, for the selected values for one or more hyperparameters; and selecting a hyperparameter configuration with a highest accuracy as suggested by Yu into Mathis’s computer-implemented method, with a reasonable expectation of success, to teach the testing of the first model with one of the hyperparameter configurations comprising: selecting values for one or more hyperparameters of the first model; training the first model with the selected values; and calculating an accuracy of the first model, using validation data, for the selected values for the one or more hyperparameters; and selecting a hyperparameter configuration with a highest accuracy. This modification would have been motivated by the desire to tune the hyperparameters to rely on incomplete and/or limited information (Yu [0004]).
Regarding dependent claim 12, claim 12 is a system claim that is substantially the same as the method of claim 2. Thus, claim 12 is rejected for the same reasons as claim 2.
Regarding dependent claim 17, claim 17 is a tangible machine-readable storage medium claim that is substantially the same as the method of claim 2. Thus, claim 17 is rejected for the same reasons as claim 2.
Claims 3, 13 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Mathis, as applied in the rejections of claims 1, 11, and 16 above, in view of Bath et al. (hereinafter Bath), US 10,606,856 B2.
Regarding dependent claim 3, Mathis teaches all the elements of claim 1.
Mathis does not expressly teach wherein collecting actual values comprises: obtaining data for the time series of the metric received via logs or metrics data; and inserting the obtained data in the time series of the metric.
However, Bath teaches wherein collecting actual values comprises: obtaining data for a time series of a metric received via logs or metrics data (7:7-7:52 wherein data intake stores raw data (wherein collecting actual values) including application logs or metrics data in the time series (comprises obtaining data for a time series of the metric received via logs or metrics data)); and inserting the obtained data in the time series of the metric (7:7-7:60 system modified to retain the remaining raw data indexed by timestamps).
Because Bath and Mathis address the issue of collecting actual values, accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of wherein collecting actual values comprises: obtaining data for a time series of a metric received via logs or metrics data; and inserting the obtained data in the time series of the metric as suggested by Bath into Mathis’s method, with a reasonable expectation of success to teach wherein collecting actual values comprises: obtaining data for the time series of the metric received via logs or metrics data; and inserting the obtained data in the time series of the metric. This modification would have been motivated by the desire to address the need of analyzing metrics data and/or machine-generated data of computing resources (Bath 2:17-2:20).
Regarding dependent claim 13, claim 13 is a system claim that is substantially the same as the method of claim 3. Thus, claim 13 is rejected for the same reasons as claim 3.
Regarding dependent claim 18, claim 18 is a tangible machine-readable storage medium claim that is substantially the same as the method of claim 3. Thus, claim 18 is rejected for the same reasons as claim 3.
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Mathis, as applied in the rejections of claim 1 above, in view of Weissman et al. (hereinafter Weissman), US 2018/0196900 A1.
Regarding dependent claim 7, Mathis teaches all the elements of claim 1.
Mathis does not expressly teach wherein the ML model is selected from a group comprising an AR model and a SARMA model, the AR model having a lags hyperparameter, the SARMA model having hyperparameters comprising a trend autoregressive order, a trend difference order, a trend moving average order, a number of time steps for a single seasonal period, a seasonal autoregressive order, a seasonal differencing order, and a seasonal moving average order.
However, Weissman teaches a ML model selected from a group comprising an AR model and a SARMA model, the AR model having a lags hyperparameter, the SARMA model having hyperparameters comprising a trend autoregressive order, a trend difference order, a trend moving average order, a number of time steps for a single seasonal period, a seasonal autoregressive order, a seasonal differencing order, and a seasonal moving average order ([0029-0035] forecasting time series data using the Autoregressive Integrated Moving Average (ARIMA) model, which can be a seasonal ARIMA (SARIMA) model and provides the precise definition of the SARIMA model parameters in denoting non-seasonal ARIMA models and seasonal ARIMA models).
Because Weissman and Mathis address the issue of predicting models, accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of a ML model selected from a group comprising an AR model and a SARMA model, the AR model having a lags hyperparameter, the SARMA model having hyperparameters comprising a trend autoregressive order, a trend difference order, a trend moving average order, a number of time steps for a single seasonal period, a seasonal autoregressive order, a seasonal differencing order, and a seasonal moving average order as suggested by Weissman into Mathis’s method, with a reasonable expectation of success to teach wherein the ML model is selected from a group comprising an AR model and a SARMA model, the AR model having a lags hyperparameter, the SARMA model having hyperparameters comprising a trend autoregressive order, a trend difference order, a trend moving average order, a number of time steps for a single seasonal period, a seasonal autoregressive order, a seasonal differencing order, and a seasonal moving average order. This modification would have been motivated by the desire to provide a solution to improve on the conventional approaches of generating a forecast of values with respect to a performance indicator and comparing actual values to the forecast (Weissman [0004-0005]).
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Mathis, as applied in the rejections of claim 1 above, in view of Filimonov et al. (hereinafter Filimonov), US 2018/0196900 A1.
Regarding dependent claim 9, Mathis teaches all the elements of claim 1.
Mathis does not expressly teach wherein the anomaly is one of change detection, slow drift, sudden change from zero, sudden change to zero, or transient spike.
However, Filimonov teaches wherein the anomaly is one of change detection, slow drift, sudden change from zero, sudden change to zero, or transient spike (10:8-12:40 detecting a spike anomaly. This corresponds to the presence of a positive or a negative spike in the sequence, which is detected by comparing the second order derivative of average values to a calculated threshold. It also identifies large dips/spikes with short duration and detecting sudden variation or edge anomaly. An edge anomaly refers to the occurrence of a sharp drop or sharp rise in the time series. This is detected if the absolute value of the drop or rise is larger than the standard deviation multiplied by a calculated threshold. This directly teaches sudden change detection).
Because Filimonov and Mathis address the issue of predicting models, accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings wherein the anomaly is one of change detection, slow drift, sudden change from zero, sudden change to zero, or transient spike as suggested by Filimonov into Mathis’s method, with a reasonable expectation of success. This modification would have been motivated by the desire to enable automatic detection of values that are abnormal to a high degree of probability in any time series sequence (Filimonov 1:35-1:40).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
PANG, US 2020/0034733 A1 (Jan. 30, 2020) (ABSTRACT In a computer-implemented method for visualization of anomalies in time series data on a graphical user interface, a plurality of time series data is dynamically displayed in a graph of the graphical user interface, the plurality of time series data including data points represented as numerical measurements. An indication that a time series data of the plurality of time series data includes an anomaly is received. Responsive to receiving the indication that the time series data of the plurality of time series data includes an anomaly, a visualization of at least a region of the time series data including the anomaly is changed).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KUANG FU CHEN whose telephone number is (571)272-1393. The examiner can normally be reached M-F 9:00-5:30pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Welch can be reached on (571) 272-7212. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KC CHEN/Primary Patent Examiner, Art Unit 2143