Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claims 1-20 are rejected under 35 U.S.C. 101 because they are directed to an abstract idea without significantly more.
Regarding claims 1-20:
Step 1: With respect to claims 1-8, applying step 1, the preamble of claims 1-8 claims a method, which falls within the statutory category of a process. With respect to claims 9-20, applying step 1, the preamble of claims 9-20 claims a system, which falls within the statutory category of an apparatus.
Regarding claim 1,
Step 2A — Prong One: Claim 1 recites an abstract idea. The limitation of “forming a plurality of aggregated data sets, wherein each aggregated data set comprises information generated from the entire training data set, each aggregated data set comprises entries generated from an associated aggregate of data points from the training data set, each associated aggregate of data points comprises a unique granularity;” is an abstract idea directed to mental processes, for example, a human could use observation and judgement to form a plurality of aggregated data sets, wherein each aggregate data set comprises entries generated from an associated aggregate of data points from the training data set, and each associated aggregate of data points comprising a unique granularity. The limitation of “analyzing a plurality of operational data set data points, generated by the application during execution by one or more processor cores of the processor, wherein for each of the plurality of machine learning models, the plurality of operational data set data points are aggregated at the same granularity as that of the associated aggregated data set used to train the machine learning model” is an abstract idea directed to mental processes, for example, a human could use evaluation and judgement to analyze a plurality of operational data set points, generated by the application, wherein for each of the plurality of machine learning models, the plurality of operational data set data points are aggregated at the same granularity as that of the associated aggregated data set used to train the machine learning model.
Step 2A — Prong Two: Claim 1 fails to integrate the judicial exception into practical application. The element of “providing a training data set, wherein the training data set comprises data points associated with the normal behavior;” is insignificant extra-solution activity that amounts to no more than mere data gathering (See MPEP 2106.05(g)). The element of “using the plurality of machine learning models executing on the processor,” are mere attempts to generally link the judicial exception to the technological environment of a computer (See MPEP 2106.05(h)). The element of “training a plurality of machine learning models including at least a first machine learning model, a second machine learning model, and a third machine learning model, wherein each machine learning model of the plurality of machine learning models is trained using an associated aggregated data set of the plurality of aggregated data sets, wherein the first machine learning model is trained according to a first associated aggregated data set having a first granularity corresponding a first number of data points, the second machine learning model is trained according to a second associated aggregated data set having a second granularity corresponding to a second number of data points, and the third machine learning model is trained according to a third associated aggregated data set having a third granularity corresponding to a data period;” are mere instructions to implement an abstract idea on a generic computer (See MPEP 2106.05(f)).
Step 2B: Claim 1 does not contain any additional elements that would amount to significantly more than the judicial exception. The element of “providing a training data set, wherein the training data set comprises data points associated with the normal behavior,” is insignificant extra-solution activity that amounts to no more than mere data gathering (See MPEP 2106.05(g)). Data gathering is a well-understood, routine conventional activity as recognized by the courts (See MPEP 2106.05(d)(II)). Providing a training data that is associated with normal behavior does not impose meaningful limitations to detecting an anomaly. The element of “using the plurality of machine learning models executing on the processor,” are mere attempts to generally link the judicial exception to the technological environment of a computer (See MPEP 2106.05(h)). The element of “training a plurality of machine learning models including at least a first machine learning model, a second machine learning model, and a third machine learning model, wherein each machine learning model of the plurality of machine learning models is trained using an associated aggregated data set of the plurality of aggregated data sets, wherein the first machine learning model is trained according to a first associated aggregated data set having a first granularity corresponding a first number of data points, the second machine learning model is trained according to a second associated aggregated data set having a second granularity corresponding to a second number of data points, and the third machine learning model is trained according to a third associated aggregated data set having a third granularity corresponding to a data period;” are mere instructions to implement an abstract idea on a generic computer (See MPEP 2106.05(f)). The computer is recited at a high level of generality and imposes no meaningful limitations on the claim.
Regarding claim 9,
Step 2A — Prong One: Claim 9 recites an abstract idea. The limitation of “form a plurality of aggregated data sets, wherein each aggregated data set comprises information generated from the entire training data set, each aggregated data set comprises entries generated from an associated aggregate of data points from the training data set, each associated aggregate of data points comprises a unique granularity;” is an abstract idea directed to mental processes, for example, a human could use observation and judgement to form a plurality of aggregated data sets, wherein each aggregate data set comprises entries generated from an associated aggregate of data points from the training data set, and each associated aggregate of data points comprising a unique granularity. The limitation of “subsequently, analyze a plurality of operational data set data points[[,]] generated by the environment[[,]] wherein for each of the plurality of machine learning models, the plurality of operational data set data points are aggregated at the same granularity as that of the associated aggregated data set used to train the machine learning model” is an abstract idea directed to mental processes, for example, a human could use evaluation and judgement to analyze a plurality of operational data set points, generated by the application, wherein for each of the plurality of machine learning models, the plurality of operational data set data points are aggregated at the same granularity as that of the associated aggregated data set used to train the machine learning model.
Step 2A — Prong Two: Claim 9 fails to integrate the judicial exception into practical application. The element of “and storing a training data set comprising data points associated with the normal behavior;” is insignificant extra-solution activity that amounts to no more than mere data gathering (See MPEP 2106.05(g)). The elements of “a processor; a first memory[[,]] coupled to the processor[[,]]”, “a second memory[[,]] coupled to the processor[[,]] and storing instructions executable by the processor, the instructions, when executed, configured to cause the processor to:”, and “using the plurality of machine learning models,” are mere attempts to generally link the judicial exception to the technological environment of a computer (See MPEP 2106.05(h)). The element of “train a plurality of machine learning models including at least one first machine learning model and at least one second machine learning model, wherein each machine learning model of the plurality of machine learning models is trained using an associated aggregated data set having a unique granularity, wherein the at least one first machine learning model is trained using the associated aggregated data set having at least one first granularity having at least one first number of data points, wherein the at least one second machine learning model is training using the associated aggregated data set having at least one second granularity having at least one data period;” are mere instructions to implement the abstract idea on a generic computer (See MPEP 2106.05(f)).
Step 2B: Claim 9 does not contain any additional elements that would amount to significantly more than the judicial exception. The element of “and storing a training data set comprising data points associated with the normal behavior;” is insignificant extra-solution activity that amounts to no more than mere data gathering (See MPEP 2106.05(g)). Data gathering is a well-understood, routine conventional activity as recognized by the courts (See MPEP 2106.05(d)(II)). Storing training data that is associated with normal behavior does not impose meaningful limitations to detecting an anomaly. The elements of “a processor; a first memory[[,]] coupled to the processor[[,]]”, “a second memory[[,]] coupled to the processor[[,]] and storing instructions executable by the processor, the instructions, when executed, configured to cause the processor to:”, and “using the plurality of machine learning models,” are mere attempts to generally link the judicial exception to the technological environment of a computer (See MPEP 2106.05(h)). The element of “train a plurality of machine learning models including at least one first machine learning model and at least one second machine learning model, wherein each machine learning model of the plurality of machine learning models is trained using an associated aggregated data set having a unique granularity, wherein the at least one first machine learning model is trained using the associated aggregated data set having at least one first granularity having at least one first number of data points, wherein the at least one second machine learning model is training using the associated aggregated data set having at least one second granularity having at least one data period;” are mere instructions to implement the abstract idea on a generic computer (See MPEP 2106.05(f)). The computer is recited at a high level of generality and imposes no meaningful limitations on the claim.
Regarding claim 17,
Step 2A — Prong One: Claim 17 recites an abstract idea. The limitation of “analyze the performance statistic over time” and “the performance statistic is aggregated at the same granularity as that of the associated data set used to train the machine learning model, and said analyzing comprises determining, by the processor, whether the performance statistic exhibits anomalous behavior relative to the training data set” is an abstract idea directed to mental processes, for example, a human could use observation, evaluation, and judgement to analyze a performance statistic over time, where the performance statistic is aggregated at the same granularity as that of the associated data set used to train the machine learning model, and said analyzing comprises determining whether the performance statistic exhibits anomalous behavior. The limitation of “in response to determining the performance statistic exhibits anomalous behavior, the instructions cause the processor to selectively respond to the anomalous behavior” is an abstract idea directed to mental processes, for example, a human could use observation, evaluation, and judgement to determine the performance statistic exhibits anomalous behavior, and could selectively respond to, or cause the processor to selectively respond to, the anomalous behavior.
Step 2A — Prong Two: Claim 17 fails to integrate the judicial exception into practical application. The elements of “a processor; a performance monitoring unit configured to periodically track a performance statistic associated with the processor; a memory[[,]] coupled to the processor[[,]] and storing instructions executable by the processor, the instructions, when executed, configured to cause the processor to:”, and “using a plurality of machine learning models, wherein each machine learning model of the plurality of machine learning models is trained using an associated aggregated data set, each aggregated data set comprises information generated from an entire training data set, each aggregated data set comprises entries generated from an associated aggregate of data points from the training data set, for each of the plurality of machine learning models,” are mere attempts to generally link the judicial exception to the technological environment of a computer (See MPEP 2106.05(h)). The element of “each associated aggregate of data points comprises a unique granularity corresponding to at least one of a unique number of data points or a unique data period such that each machine learning model is trained according to the unique granularity,” are mere instructions to implement an abstract idea on a generic computer (See MPEP 2106.05(f)).
Step 2B: Claim 17 does not contain any additional elements that would amount to significantly more than the judicial exception. The elements of “a processor; a performance monitoring unit configured to periodically track a performance statistic associated with the processor; a memory[[,]] coupled to the processor[[,]] and storing instructions executable by the processor, the instructions, when executed, configured to cause the processor to:”, and “using a plurality of machine learning models, wherein each machine learning model of the plurality of machine learning models is trained using an associated aggregated data set, each aggregated data set comprises information generated from an entire training data set, each aggregated data set comprises entries generated from an associated aggregate of data points from the training data set, for each of the plurality of machine learning models,” are mere attempts to generally link the judicial exception to the technological environment of a computer (See MPEP 2106.05(h)). The element of “each associated aggregate of data points comprises a unique granularity corresponding to at least one of a unique number of data points or a unique data period such that each machine learning model is trained according to the unique granularity,” are mere instructions to implement an abstract idea on a generic computer (See MPEP 2106.05(f)). The computer is recited at a high level of generality and imposes no meaningful limitations on the claim.
Regarding claim 2,
Step 2A — Prong One: Claim 2 recites an abstract idea. The limitation of “wherein said analyzing comprises determining whether the operational data set data points exhibit anomalous behavior of the environment generating the operational data set from the normal behavior” is an abstract idea directed to mental processes, for example, a human could use observation, evaluation, and judgement to determine whether the operational data set data points exhibit anomalous behavior of the environment generating the operation data set from the normal behavior.
Step 2A — Prong Two: Claim 2 does not contain any additional elements that would integrate the judicial exception into practical application.
Step 2B: Claim 2 does not contain any additional elements that would amount to significantly more than the judicial exception.
Regarding claim 3,
Step 2A — Prong One: Claim 3 recites an abstract idea. The limitation of “wherein said determining comprises: examining results of said analyzing by each machine learning model for anomalous behavior at the associated granularity of that machine learning model; and determining whether the results from any one of the machine learning models exhibits anomalous behavior” is an abstract idea directed to mental processes, for example, a human could use observation, evaluation, and judgement to examine results of said analyzing by each machine learning model for anomalous behavior at the associated granularity of that machine learning model, and determine whether the results from any one of the machine learning models exhibits anomalous behavior.
Step 2A — Prong Two: Claim 3 does not contain any additional elements that would integrate the judicial exception into practical application.
Step 2B: Claim 3 does not contain any additional elements that would amount to significantly more than the judicial exception.
Regarding claim 4,
Step 2A — Prong Two: Claim 4 fails to integrate the judicial exception into practical application. The additional element of “wherein each of the machine learning models comprises a same machine learning algorithm for detecting anomalous behavior” is a mere attempt to generally link the judicial exception to the technological environment of a computer (See MPEP 2106.05(h)).
Step 2B: Claim 4 does not contain any additional elements that would amount to significantly more than the judicial exception. The additional element of “wherein each of the machine learning models comprises a same machine learning algorithm for detecting anomalous behavior” is a mere attempt to generally link the judicial exception to the technological environment of a computer (See MPEP 2106.05(h)).
Regarding claim 5,
Step 2A — Prong Two: Claim 5 fails to integrate the judicial exception into practical application. The additional element of “wherein each of the machine learning models comprises a unique machine learning algorithm for detecting anomalous behavior” is a mere attempt to generally link the judicial exception to the technological environment of a computer (See MPEP 2106.05(h)).
Step 2B: Claim 5 does not contain any additional elements that would amount to significantly more than the judicial exception. The additional element of “wherein each of the machine learning models comprises a unique machine learning algorithm for detecting anomalous behavior” is a mere attempt to generally link the judicial exception to the technological environment of a computer (See MPEP 2106.05(h)).
Regarding claim 6,
Step 2A — Prong Two: Claim 6 fails to integrate the judicial exception into practical application. The element of “wherein each of the machine learning models comprises a machine learning algorithm for detecting anomalous behavior at the granularity of the associated aggregated data set” is a mere attempt to generally link the judicial exception to the technological environment of a computer (See MPEP 2106.05(h)).
Step 2B: Claim 6 does not contain any additional elements that would amount to significantly more than the judicial exception. The element of “wherein each of the machine learning models comprises a machine learning algorithm for detecting anomalous behavior at the granularity of the associated aggregated data set” is a mere attempt to generally link the judicial exception to the technological environment of a computer (See MPEP 2106.05(h)).
Regarding claim 7,
Step 2A — Prong Two: Claim 7 fails to integrate the judicial exception into practical application. The element of “wherein a first machine learning model of the plurality of machine learning models is trained using an aggregated data set comprising single data points from the training data set” is a mere attempt to generally link the judicial exception to the technological environment of a computer (See MPEP 2106.05(h)).
Step 2B: Claim 7 does not contain any additional elements that would amount to significantly more than the judicial exception. The element of “wherein a first machine learning model of the plurality of machine learning models is trained using an aggregated data set comprising single data points from the training data set” is a mere attempt to generally link the judicial exception to the technological environment of a computer (See MPEP 2106.05(h)).
Regarding claim 8,
Step 2A — Prong Two: Claim 8 fails to integrate the judicial exception into practical application. The limitation of “wherein an environment generating the operational data set comprises one of a processor performance monitor, a transaction environment, imaging data, and three-dimensional data” is insignificant extra-solution activity that amounts to no more than mere data gathering (See MPEP 2106.05(g)).
Step 2B: Claim 8 does not contain any additional elements that would amount to significantly more than the judicial exception. The limitation of “wherein an environment generating the operational data set comprises one of a processor performance monitor, a transaction environment, imaging data, and three-dimensional data” is insignificant extra-solution activity that amounts to no more than mere data gathering (See MPEP 2106.05(g)). Data gathering is a well-understood, routine conventional activity as recognized by the courts (See MPEP 2106.05(d)(II)).
Claims 2-8 are therefore not drawn to eligible subject matter are they are directed to an abstract idea without amounting to significantly more than the judicial exception.
Claims 10-16 and 18-20 incorporate substantively all the limitations of claims 2-8 in a system and are rejected under the same rationale.
Claim Rejections - 35 USC § 103
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claim(s) 1-3, 7-11, and 15-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Morris, II et al. (US Pub. No. 2016/0350671, hereinafter "Morris") in view of Le et al. (NPL from IDS: Analyzing Data Granularity Levels for Insider Threat Detection Using Machine Learning, hereinafter "Le"), and further in view of Manadhata et al. (US Pub. No. 2018/0176241, hereinafter “Manadhata”).
Regarding claim 1, Morris teaches a method for detecting anomalies in an operational data set with respect to well- defined normal behavior of an application executing on a processor of a computing device, the method comprising:
providing a training data set, wherein the training data set comprises data points associated with the normal behavior (Morris, [0030] — ““Source data” comprises input data generated from or associated with operation of the operating system, hardware device, or machine in need of monitoring. Such source data can be derived from information obtained during the operation of the operating system as a whole, or it can comprise information that is specifically associated with one or more hardware devices or machines included with or associated with the operating system in need of monitoring.”— Morris teaches providing a training data set that comprises data points associated with normal behavior of a subject);
training a plurality of machine learning models, wherein each machine learning model of the plurality of machine learning models is trained using an associated aggregated data set of the plurality of aggregated data sets (Morris, [0073] — “the runtime process aggregates 188 the formatted source data 176 based upon the features defined by the contextualization 186 to generate an aggregated data set 190 including feature data context values, which is used to generate the models 194 and predicted outputs 196 for the outcome of interest for the operating system 110. The predictive models 194 are based upon the determined contextualization 186 and trained 192 using data from the aggregated data set 190” — Morris teaches creating a plurality of machine learning models, wherein each machine learning model of the plurality of machine learning models is trained using an associated aggregated data set);
analyzing a plurality of operational data set data points, generated by the application during execution by one or more processor cores of the processor, using the plurality of machine learning models executing on the processor (Morris, [0038] — “The generated one or more predictive models will then be self-modifiable or self-correctable when new operational data is generated from continued operation of the operating system” — Morris teaches analyzing a plurality of operational data generated by the application during execution using the plurality of machine learning models),
wherein for each of the plurality of machine learning models, the plurality of operational data set data points are aggregated at the same granularity as that of the associated aggregated data set used to train the machine learning model (Morris, [0050] — “The user device(s) and/or controller(s) 124, in addition to facilitating local storage of source data, can be configured to provide additional information and/or source data associated with the processes, operating conditions and/or equipment 112 associated with the environment 100. Data and/or readings entered or recorded by an operator, as well as aggregations and/or combinations of sensor data, can be collected and/or derived via the user device(s) and/or controller(s) 124 and transmitted to the on-site server or master controller 128 and/or stored in an on-site database 130. Such additional information can comprise historical source data or operational data or system/user input data as appropriate in the inventions herein.” — Morris teaches the operational data points being aggregated with the source data and thus shows that the operational data being aggregated at the same granularity as that of the associated aggregated data set used to train the machine learning model).
Morris fails to explicitly teach forming a plurality of aggregated data sets, wherein each aggregated data set comprises information generated from the entire training data set, each aggregated data set comprises entries generated from an associated aggregate of data points from the training data set, each associated aggregate of data points comprises a unique granularity.
However, analogous to the field of anomaly detection, Le teaches:
forming a plurality of aggregated data sets, wherein each aggregated data set comprises information generated from the entire training data set, each aggregated data set comprises entries generated from an associated aggregate of data points from the training data set, each associated aggregate of data points comprises a unique granularity (Le, Section III B Paragraph 4 — “Firstly, data from different sources are aggregated based on user id given an aggregation condition ¢” and in Section III B Paragraph 9 — “Based on aforementioned data aggregation condition c, extracted features could have different levels of granularity.” — Le teaches forming a plurality of aggregated data sets wherein each aggregated data set comprises information from the entire training data set, each aggregated data set comprises entries generated from an associated aggregate of data points from the training data set, and each associated aggregate of data points comprises a unique granularity based on the aggregation condition c);
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate the granular data sets of Le to the machine learning models and operational data of Morris in order to create an anomaly detection method using the levels of granularity in aggregated data. Doing so would allow quicker system responses when a malicious instance is detected (Le, Section B Paragraph 10).
The combination of Morris and Le fails to explicitly teach training a plurality of machine learning models including at least a first machine learning model, a second machine learning model, and a third machine learning model, wherein the first machine learning model is trained according to a first associated aggregated data set having a first granularity corresponding to a first number of data points, the second machine learning model is trained according to a second associated aggregated data set having a second granularity corresponding to a second number of data points, and the third machine learning model is trained according to a third associated aggregated data set having a third granularity corresponding to a data period.
However, analogous to the field of the claimed invention, Manadhata teaches:
training a plurality of machine learning models including at least a first machine learning model, a second machine learning model, and a third machine learning model (Manadhata, [0017] – “As shown in the illustrative example of FIG. 1, the prediction engine 108 may include engine components to extract time-series data of an enterprise entity from log data of an enterprise, wherein the time-series data of the enterprise entity includes measured feature values of a set of selected features over a series of time periods; and train predictive models specific to the enterprise entity using the time-series data, including training a separate predictive model for each selected feature using time-series data specific to the selected feature, wherein the separate predictive model is to output a predicted feature value of the selected feature for a particular time period” – teaches training a plurality of machine learning models, including at least a first machine learning model, a second machine learning model, and a third machine learning model, as in [0035] – “In some examples, the predictive model 320 may include individual models for each selected feature for which time-series data was extracted by the prediction engine 108. To illustrate, the predictive model 320 shown in FIG. 3 may include multiple individual models, such as the predictive model 322 specific to Feature.sub.1 trained using the Feature.sub.1 time-series data 312, the predictive model 324 specific to Feature.sub.2 trained using the Feature.sub.2 time-series data 314, and other predictive models for other features” – teaches a first (predictive model 322), second (predictive model 324), and third (other predictive models for other features) machine learning model ),
wherein the first machine learning model is trained according to a first associated aggregated data set having a first granularity corresponding to a first number of data points, the second machine learning model is trained according to a second associated aggregated data set having a second granularity corresponding to a second number of data points, and the third machine learning model is trained according to a third associated aggregated data set having a third granularity corresponding to a data period (Manadhata, [0023] – “From the enterprise log data 240, the prediction engine 108 may extract time-series data. Time-series data may refer to data characteristics of enterprise entities measured over multiple time periods. The specific time period duration or length at which the prediction engine 108 extracts time-series data may be configurable, and some examples of applicable time period durations include a 1-hour time period, a 4-hour time period, an 8-day time period, a 1-day time period, or any other specified time period duration. As such, the prediction engine 108 may extract time-series data from the enterprise log data 240 for multiple enterprise entities monitored by the security system 100, and in some examples do so on a per-entity basis” and in [0035] – “In some examples, the predictive model 320 may include individual models for each selected feature for which time-series data was extracted by the prediction engine 108. To illustrate, the predictive model 320 shown in FIG. 3 may include multiple individual models, such as the predictive model 322 specific to Feature.sub.1 trained using the Feature.sub.1 time-series data 312, the predictive model 324 specific to Feature.sub.2 trained using the Feature.sub.2 time-series data 314, and other predictive models for other features. In that regard, the prediction engine 108 may train separate predictive models for each monitored enterprise entity (e.g., at a per entity granularity), and further train individual predictive models for each selected feature from which time-series data extractions are performed (e.g., at a per-entity-per-feature granularity)” – teaches wherein the first (predictive model 322) machine learning model is trained according to a first associated aggregated data set having a first granularity corresponding to a first number of data points (trained at per-entity-per-feature granularity, across different time periods (such as 1 hour, 4 hours, 8 days, 1 day, or any other specified time period duration)), the second (predictive model 324) machine learning model is trained according to a second associated aggregated data set having a second granularity corresponding to a second number of data points (trained at per-entity-per-feature granularity, across different time periods), and the third (and other predictive models for other features) machine learning model is trained according to a third associated aggregated data set having a third granularity corresponding to a data period (log data acquired at 1 hour, 4 hours, 8 days, 1 day, or any other specified time period duration));
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate the training of a plurality of machine learning models associated with aggregated data sets having unique granularities of Manadhata to the plurality of aggregated data sets having unique granularities generated by Morris and Le in order to train a plurality of individual machine learning models on data sets with various granularities. Doing so would result in increased system security, accuracy, and efficiency, e.g., by reducing false-positives (e.g., benign behavior identified as malicious behavior thereby increasing the workload of security analysts), flexibly adapting to changing malware attack patterns, detecting active malware that was previously dormant, etc. (Manadhata, [011]).
Regarding claim 9, Morris teaches a system for detecting anomalies in an operational data set generated by an environment with respect to well-defined normal behavior, the system comprising:
a processor; a first memory[[,]] coupled to the processor (Morris, [0093] — “predictive system 160 may include one or more processors(s) 200, one or more input/output (I/O) interfaces 202, one or more network interface(s) 204, one or more storage interface(s) 206, and one or more storage or memories 210.” — Morris teaches the system including one or more processors and storage or memories)[[,]] and storing a training data set comprising data points associated with the normal behavior (Morris, [0030] — ““Source data” comprises input data generated from or associated with operation of the operating system, hardware device, or machine in need of monitoring. Such source data can be derived from information obtained during the operation of the operating system as a whole, or it can comprise information that is specifically associated with one or more hardware devices or machines included with or associated with the operating system in need of monitoring.”— Morris teaches providing a training data set that comprises data points associated with normal behavior of a subject); a second memory, coupled to the processor , and storing instructions executable by the processor (Morris, [0093] — “predictive system 160 may include one or more processors(s) 200, one or more input/output (I/O) interfaces 202, one or more network interface(s) 204, one or more storage interface(s) 206, and one or more storage or memories 210.” — Morris teaches the system including one or more processors and a second memory storing instructions);
a second memory[[,]] coupled to the processor[[,]] and storing instructions executable by the processor, the instructions, when executed, configured to cause the processor to (Morris, [0093] — “predictive system 160 may include one or more processors(s) 200, one or more input/output (I/O) interfaces 202, one or more network interface(s) 204, one or more storage interface(s) 206, and one or more storage or memories 210.” – teaches a second memory, coupled to the processor, storing instructions):
train a plurality of machine learning models, wherein each machine learning model of the plurality of machine learning models is trained using an associated aggregated data set (Morris, [0073] — “the runtime process aggregates 188 the formatted source data 176 based upon the features defined by the contextualization 186 to generate an aggregated data set 190 including feature data context values, which is used to generate the models 194 and predicted outputs 196 for the outcome of interest for the operating system 110. The predictive models 194 are based upon the determined contextualization 186 and trained 192 using data from the aggregated data set 190” — Morris teaches creating a plurality of machine learning models, wherein each machine learning model of the plurality of machine learning models is trained using an associated aggregated data set);
subsequently, analyze a plurality of operational data set data points[[,]] generated by the environment[[,]] using the plurality of machine learning models (Morris, [0038] — “The generated one or more predictive models will then be self-modifiable or self-correctable when new operational data is generated from continued operation of the operating system” — Morris teaches analyzing a plurality of operational data generated by the environment using the plurality of machine learning models), wherein: for each of the plurality of machine learning models, the plurality of operational data set data points are aggregated at the same granularity as that of the associated aggregated data set used to train the machine learning model (Morris, [0050] — “The user device(s) and/or controller(s) 124, in addition to facilitating local storage of source data, can be configured to provide additional information and/or source data associated with the processes, operating conditions and/or equipment 112 associated with the environment 100. Data and/or readings entered or recorded by an operator, as well as aggregations and/or combinations of sensor data, can be collected and/or derived via the user device(s) and/or controller(s) 124 and transmitted to the on-site server or master controller 128 and/or stored in an on-site database 130. Such additional information can comprise historical source data or operational data or system/user input data as appropriate in the inventions herein.” — Morris teaches the operational data points being aggregated with the source data and thus shows that the operational data being aggregated at the same granularity as that of the associated aggregated data set used to train the machine learning model).
Morris fails to explicitly teach the instructions configured to form a plurality of aggregated data sets, wherein each aggregated data set comprises information generated from the entire training data set, each aggregated data set comprises entries generated from an associated aggregate of data points from the training data set, each associated aggregate of data points comprises a unique granularity.
However, analogous to the field of anomaly detection, Le teaches:
form a plurality of aggregated data sets, wherein: each aggregated data set comprises information generated from the entire training data set, each aggregated data set comprises entries generated from an associated aggregate of data points from the training data set, each associated aggregate of data points comprises a unique granularity (Le, Section III B Paragraph 4 — “Firstly, data from different sources are aggregated based on user id given an aggregation condition ¢” and in Section III B Paragraph 9 — “Based on aforementioned data aggregation condition c, extracted features could have different levels of granularity.” — Le teaches forming a plurality of aggregated data sets wherein each aggregated data set comprises information from the entire training data set, each aggregated data set comprises entries generated from an associated aggregate of data points from the training data set, and each associated aggregate of data points comprises a unique granularity based on the aggregation condition c);
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate the granular data sets of Le to the system of Morris in order to create an anomaly detection system using the levels of granularity in aggregated data. Doing so would allow quicker system responses when a malicious instance is detected (Le, Section B Paragraph 10).
The combination of Morris and Le fails to explicitly teach at least one first machine learning model and at least one second machine learning model, wherein each machine learning model of the plurality of machine learning models is trained using an associated aggregated data set having a unique granularity, wherein the at least one first machine learning model is trained using the associated aggregated data set having at least one first granularity having at least one first number of data points, wherein the at least one second machine learning is trained using the associated aggregated data set having at least one second granularity having at least one data period.
However, analogous to the field of the claimed invention, Manadhata teaches:
train a plurality of machine learning models, including at least one first machine learning model and at least one second machine learning model, wherein each machine learning model of the plurality of machine learning models is trained using an associated aggregated data set having a unique granularity, wherein the at least one first machine learning model is trained using the associated aggregated data set having at least one first granularity having at least one first number of data points, wherein the at least one second machine learning is trained using the associated aggregated data set having at least one second granularity having at least one data period (Manadhata, [0023] – “From the enterprise log data 240, the prediction engine 108 may extract time-series data. Time-series data may refer to data characteristics of enterprise entities measured over multiple time periods. The specific time period duration or length at which the prediction engine 108 extracts time-series data may be configurable, and some examples of applicable time period durations include a 1-hour time period, a 4-hour time period, an 8-day time period, a 1-day time period, or any other specified time period duration. As such, the prediction engine 108 may extract time-series data from the enterprise log data 240 for multiple enterprise entities monitored by the security system 100, and in some examples do so on a per-entity basis” and in [0035] – “In some examples, the predictive model 320 may include individual models for each selected feature for which time-series data was extracted by the prediction engine 108. To illustrate, the predictive model 320 shown in FIG. 3 may include multiple individual models, such as the predictive model 322 specific to Feature.sub.1 trained using the Feature.sub.1 time-series data 312, the predictive model 324 specific to Feature.sub.2 trained using the Feature.sub.2 time-series data 314, and other predictive models for other features. In that regard, the prediction engine 108 may train separate predictive models for each monitored enterprise entity (e.g., at a per entity granularity), and further train individual predictive models for each selected feature from which time-series data extractions are performed (e.g., at a per-entity-per-feature granularity)” – teaches wherein at least one first machine learning model (predictive model 322) is trained according to an associated aggregated data set having at least one first granularity corresponding to at least one first number of data points (trained at per-entity-per-feature granularity, across different time periods (such as 1 hour, 4 hours, 8 days, 1 day, or any other specified time period duration) and at least one second machine learning model (predictive model 324) is trained according to an associated aggregated data set having at least one second granularity corresponding to a at least one data period (log data acquired at 1 hour, 4 hours, 8 days, 1 day, or any other specified time period duration));
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate the training of a plurality of machine learning models associated with aggregated data sets having unique granularities of Manadhata to the plurality of aggregated data sets having unique granularities generated by Morris and Le in order to train a plurality of individual machine learning models on data sets with various granularities. Doing so would result in increased system security, accuracy, and efficiency, e.g., by reducing false-positives (e.g., benign behavior identified as malicious behavior thereby increasing the workload of security analysts), flexibly adapting to changing malware attack patterns, detecting active malware that was previously dormant, etc. (Manadhata, [011]).
Regarding claim 17, Morris teaches a system comprising:
a processor (Morris, [0093] — “predictive system 160 may include one or more processors(s) 200, one or more input/output (I/O) interfaces 202, one or more network interface(s) 204, one or more storage interface(s) 206, and one or more storage or memories 210.” — Morris teaches the system including one or more processors and storage or memories);
a performance monitoring unit configured to periodically track a performance statistic associated with the processor (Morris, [0025] — “As used herein, groups of operating systems, hardware devices, machines and associated processes are also contemplated for monitoring of one or more operational outcomes of interest related thereto. Operating systems can include, but are not limited to, mobile systems or vehicles (e.g., locomotives, airplanes, etc.), industrial facilities or distributed equipment, or other monitored processing systems.” — Morris teaches monitoring operating systems, hardware devices (a processor), machines, and associated processes including monitoring processor performance);
a memory[[,]] coupled to the processor[[,]] and storing instructions executable by the processor, the instructions, when executed, configured to cause the processor to: (Morris, [0093] — “predictive system 160 may include one or more processors(s) 200, one or more input/output (I/O) interfaces 202, one or more network interface(s) 204, one or more storage interface(s) 206, and one or more storage or memories 210.” — Morris teaches the system including one or more processors and storage or memories containing instructions) wherein: each machine learning model of the plurality of machine learning models is trained using an associated aggregated data set (Morris, [0073] — “the runtime process aggregates 188 the formatted source data 176 based upon the features defined by the contextualization 186 to generate an aggregated data set 190 including feature data context values, which is used to generate the models 194 and predicted outputs 196 for the outcome of interest for the operating system 110. The predictive models 194 are based upon the determined contextualization 186 and trained 192 using data from the aggregated data set 190” — Morris teaches creating a plurality of machine learning models, wherein each machine learning model of the plurality of machine learning models is trained using an associated aggregated data set)
Morris fails to explicitly teach analyze the performance statistic over time using a plurality of machine learning models, each aggregated data set comprises information generated from an entire training data set, each aggregated data set comprises entries generated from an associated aggregate of data points from the training data set, each associated aggregate of data points comprises a unique granularity, for each of the plurality of machine learning models, the performance statistic is aggregated at the same granularity as that of the associated data set used to train the machine learning model, and said analyzing comprises determining, by the processor, whether the performance statistic exhibits anomalous behavior relative to the training data set and in response to determining the performance statistic exhibits anomalous behavior, the instructions cause the processor to selectively respond to the anomalous behavior.
However, analogous to the field of anomaly detection, Le teaches:
analyze the performance statistic over time using a plurality of machine learning models (Le, Section IV B2 Performance Metrics Paragraph 6 — “to further analyze the insider threat cases and provide better insights into the effectiveness of the approach, we introduce the following measures: detection delay (DD) and detection rate per detected malicious insider (DR/DMI). DD can be defined as the time duration between the first malicious action performed by a malicious insider until he/she is detected (if ever). On the other hand, DR/DMI is the percentage of malicious instances detected per malicious user” — Le teaches defining two performance statistics for the plurality of machine learning models, where the statistics are dete