DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . This action is responsive to pending claims 1-24 filed 3/7/2024.
Priority
Acknowledgment is made of applicant's claim for foreign priority based on an application GB2112756.8 filed in UK on 9/8/2021 .
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Claim Objections
Claim 23, which is a mirror of claim 5, recites calibrating the initial confidence values by processing the set of calibrated confidence values, where claim 5 has a process of processing the initial confidence values. Although Examiner believes the cited art holds on both variations and both can be interpreted equivalently (i.e., a processing at or before the calibrated confidence values to generate the confidence values), this change my raise confusion as to any intended difference in scope. Hence, clarification to “initial confidence values” is needed.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim(s) 19-24 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter.
Claim 19 recites a computer-readable storage medium. In light of the Specs (e.g., fig.7:702), a computer-readable medium is not limited to non-transitory forms, and hence may include non-statutory forms such as carrier waves. Hence, the claim is non-statutory.
Claim 20, 24 recites a system and a telecommunications network absent recitation of any hardware components. As such, under BRI, such a system may be software per se, carrier waves, etc. and hence is non-statutory.
The dependent claims 21-23 are rejected for failing to cure the deficiency of the parent.
Claim(s) 1-24 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The 35 U.S.C. 101 subject matter eligibility analysis first asks whether the claim is directed to one of the four statutory categories (Step 1). It next asks whether the claim is directed to an abstract idea (Step 2A), via Prong 1, whether an abstract idea (e.g., mathematical concept, mental process, certain methods of organizing human activity) is recited, and Prong 2, whether it is integrated into a practical application. It finally asks whether the claim as a whole includes additional elements that amount to significantly more than the judicial exception (Step 2B). See MPEP 2106.
STEP 1: Do the claims falls within one of the four statutory categories?
See above for statutory issues for claims 19-24. The remaining claims are statutory, but all claims are analyzed below.
STEP 2A PRONG 1: The claims recite a judicial exception:
The claims recite a process of analyzing and responding anomaly data in a computer system, akin to what a human network engineer may do. Hence, they recite a mental process.
The additional elements are underlined and analyzed subsequently. In particular:
For claim 1: A computer-implemented method comprising:
obtaining activity data indicative of an anomalous activity within a computer system (obtaining and aggregating data may be performed mentally);
processing the activity data to generate confidence data representative of a set of confidence values, each confidence value representative of a confidence that the anomalous activity comprises a respective type of activity (generating confidence levels is judgment and may be performed mentally); and
determining, based on at least the confidence data, mitigating action to take to mitigate the anomalous activity (Determining mitigating actions may be performed mentally).
For claim 2: The method of claim 1, wherein processing the activity data comprises processing the activity data using a machine learning (ML) model trained to generate output confidence data representative of a set of output confidence values in response to processing input activity data indicative of an input anomalous activity (using a trained model, e.g., a mental model o heuristic, to generate confidence judgments in response to observing data may be performed mentally) within at least one of: the computer system and a further computer system, each output confidence value representative of a confidence that the input anomalous activity comprises a respective type of activity (Generating confidence values is observation and judgment and may be performed mentally).
For claim 4: The method of claim 2, wherein the confidence data is calibrated confidence data, the set of confidence values is a set of calibrated confidence values, processing the activity data using the ML model comprises processing the activity data using the ML model to generate initial confidence data representative of a set of initial confidence values (Forming initial confidence estimates and adjusting such confidence may be performed mentally), and processing the activity data further comprises calibrating the set of initial confidence values to generate the set of calibrated confidence values (Updating confidence models may be performed mentally).
For claim 5: The method of claim 4, wherein calibrating the set of initial confidence values comprises obtaining the set of calibrated confidence values from the set of initial confidence values by processing the set of initial confidence values using a non-parametric calibration function (Adjusting confidence values non-parametrically, for example, via a general adjustment algorithm without considering parameters, may be performed mentally).
For claim 6: The method of claim 5, wherein the non-parametric calibration function is an isotonic function (Isotonic adjustments to a model may be performed mentally).
For claim 7: The method of claim 2, wherein the ML model is a calibrated ML model (Updating a calibrated model may be performed mentally).
For claim 8: The method of claim 1,wherein determining the mitigating action to take comprises determining the mitigating action to take based on a comparison between at least one of the set of confidence values and a threshold value (Making determinations based on a threshold comparison may be performed mentally).
For claim 9: The method of claim 8, wherein the threshold value is a first threshold value and determining the mitigating action to take comprises:
determining to take a first mitigating action in response to determining that a confidence value of the set of confidence values exceeds the first threshold value (Making determinations based on a threshold comparison may be performed mentally); and
determining to take a second mitigating action, different from the first mitigating action, in response to determining that the confidence value is between a second threshold value and the first threshold value, wherein the second threshold value is lower than the first threshold value (Making determinations based on a threshold comparison may be performed mentally).
For claim 10: The method of claim 1, wherein:
the set of confidence values comprises:
a first confidence value representative of a first confidence that the anomalous activity comprises a first type of activity (Determining a first confidence value may be performed mentally); and
a second confidence value representative of a second confidence that the anomalous activity comprises a second type of activity, different from the first type of activity (Determining a second confidence value may be performed mentally); and
determining the mitigating action to take comprises determining the mitigating action to take based on a comparison between the first confidence value and the second confidence value (Making determinations based on a value comparison may be performed mentally).
For claim 11: The method of claim 1,comprising:
computing, based on at least one of: a confidence value of the set of confidence values, a severity of the anomalous activity, and a criticality of the computer system to operation of a telecommunications network comprising the computer system, a risk metric indicative of a risk to the telecommunications network of the anomalous activity (Making judgments of severity may be performed mentally); and
determining to take the mitigating action based on the risk metric (Determining actions may be performed mentally).
For claim 12: The method of claim 1, wherein the activity data represents at least one intrusion detection alert (Processing alerts may be performed mentally).
For claim 13: The method of claim 1, wherein the types of activity represented by the set of confidence values comprise at least one malicious activity and at least one benign activity (Processing anomaly types may be performed mentally).
For claim 14: A computer-implemented method of calibrating a system comprising a machine learning (ML) model trained to generate output uncalibrated confidence data representative of a set of output uncalibrated confidence values in response to processing input activity data indicative of an input anomalous activity within a computer system, each output uncalibrated confidence value representative of an uncalibrated confidence that the input anomalous activity comprises a respective type of activity (Anomaly detection and confidence evaluation is a process of judgment and may be performed mentally), the method comprising:
processing, using the ML model, calibration activity data representative of an anomalous activity (Reevaluating confidence may be performed mentally) within at least one of: the computer system and a further computer system, to generate uncalibrated confidence data representative of a set of uncalibrated confidence values, each uncalibrated confidence value representative of an uncalibrated confidence that the anomalous activity comprises a respective type of activity (Generating judgments of confidence may be performed mentally); computing an uncertainty metric associated with the set of uncalibrated confidence values (Considering uncertainty of judgments may be performed mentally); and adjusting parameters associated with the ML model, based on the uncertainty metric, to calibrate the ML model, thereby generating a calibrated ML model (Adjusting parameters of a heuristic may be performed mentally).
For claim 15: The method according to claim 14, wherein the uncertainty metric is indicative of a dissimilarity between at least one of the set of uncalibrated confidence values and a corresponding at least one of a set of ground truth confidence values (Considering uncertainty based on known values may be performed mentally).
For claim 16. (Currently Amended) The method according to claim 14, wherein adjusting the parameters associated with the ML model comprises adjusting the parameters associated with the ML model to reduce the uncertainty metric (Making adjustments to a mental model to minimize a parameter may be performed mentally).
For claim 17: The method according to claim 14, wherein the system comprises a non-parametric calibration model (Making adjustments to mental models without further parameters may be performed mentally), and the method comprises:
processing the calibration activity data using the calibrated ML model to generate initial confidence data representative of a set of initial confidence values (Generating initial confidence may be performed mentally); and
adjusting the non-parametric calibration model to fit a non-parametric calibration function represented by the non-parametric calibration model to the set of initial confidence values (adjusting a model based on a function via the model to calibrate judgment may be performed mentally), wherein the non-parametric calibration function is useable to obtain the set of calibrated confidence values from the initial confidence values (Recalibrating or updating judgment may be performed mentally).
For claim 18: The method according to claim 17, wherein fitting the non-parametric calibration function comprises fitting the non-parametric calibration function using isotonic regression (Calibrating a mental judgment model on a case-by-case basis, so as to align a mental judgment with expected output, may be performed mentally).
For claim 19: A computer-readable medium storing thereon a program for carrying out the method of claim 1.
For claim 20: A computer system configured to implement:
an intrusion detection system (IDS) to:
obtain activity data indicative of an anomalous activity (Obtaining anomaly activity data may be performed mentally) within at least one of: the computer system and a further computer system; and
process the activity data to generate confidence data representative of a set of confidence values, each confidence value representative of a confidence that the anomalous activity comprises a respective type of activity (Generating judgments of confidence may be performed mentally); and
an intrusion response system (IRS) to: obtain the confidence data from the intrusion detection system; and determine, based on at least the confidence data, mitigating action to take to mitigate the anomalous activity (Determining mitigating action may be performed mentally).
Claims 21-23 recite systems analogous to the methods of 2, 4-5 and hence are similarly analyzed.
STEP 2A PRONG 2: The claims do not integrate the exception into a practical application:
For claims 1-2, the additional elements include implementation on a computer, via a machine learning model, and via a computer system.
However, this is mere instructions to implement the abstract idea on a computer or computer system, or a machine learning model and hence does not constitute an integration into a practical application.
For claim 3, the additional elements include implementation on a random-forest model.
However, this is mere instructions to implement the abstract idea on a random forest learning model and hence does not constitute an integration into a practical application.
For claim 14, 20-21, the additional elements include implementation on a computer or computer system, via a machine learning model, and via a computer system.
However, this is mere instructions to implement the abstract idea on a computer or computer system, or a machine learning model and hence does not constitute an integration into a practical application.
For claim 19, the additional elements include implementation via a computer-readable medium.
However, this is mere instructions to implement the abstract idea on a computer or computer system and hence does not constitute an integration into a practical application.
For claim 24, the additional elements include a telecommunications network.
However, this is mere application to a particular field and does not meaningfully limit the practice of the abstract idea, and hence does not constitute an integration into practical application.
STEP 2B: The claim as a whole do not include additional elements that amount to significantly more than the abstract idea:
For claim 1, the additional elements include implementation on a computer, a computer system, and a machine learning model.
However, implementation on a computer, computer system, and machine learning model is well understood, routine, and conventional in the field of data analysis and hence does not constitute significantly more.
For claim 3, the additional elements include implementation on a random-forest model.
However, implementation on a random forest learning model is well understood, routine, and conventional in the field of data analysis and hence does not constitute significantly more.
For claim 14, 20-21 the additional elements include implementation on a computer or computer system, via a machine learning model, and via a computer system.
However, implementation on a computer, computer system, and machine learning model is well understood, routine, and conventional in the field of data analysis and hence does not constitute significantly more.
For claim 19, the additional elements include implementation via a computer-readable medium.
However, implementation on a computer, computer system, and machine learning model is well understood, routine, and conventional in the field of data analysis and hence does not constitute significantly more.
For claim 24, the additional elements include a telecommunications network.
However, the implementation of data analysis techniques in telecommunication networks is well understood, routine, and conventional in the field of network data analysis and hence does not constitute significantly more.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claim(s) 20-24 are rejected under 35 U.S.C. 112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
Claim 20 recites a computer system “configured to implement” an IDS, i.e., defines a system by what it could potentially functionally perform without positively claiming actual implementation. A reader cannot form a clear understanding of the metes and bounds of such a system since the functions of such a system are not being positively claimed, is, for example, any general computing system capable of ingesting data to implement these steps, are does a computer necessarily have to be programmed with such data, etc. . Hence, the claim is indefinite.
The dependent claims 21-24 are rejected for the same reasons.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-2, 4-8, 10-16, 19-24 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Mergendahl ("Rapid: Robust and adaptive detection of distributed denial-of-service traffic from the internet of things", published 2020).
For claim 1, Mergendahl discloses: a computer-implemented method comprising:
obtaining activity data indicative of an anomalous activity within a computer system (fig.1 gives an overview of the method, with §III.B ¶1 contemplating use of sampled data streams);
processing the activity data to generate confidence data representative of a set of confidence values, each confidence value representative of a confidence that the anomalous activity comprises a respective type of activity (fig.1 shows aggregate and granular flows being processed by the Rapid model to generate a confidence metric, with high confidence samples going to direct labeling and low confidence samples going to further analysis by the mitigation unit for labeling, see §III.D ¶2: mitigation system); and
determining, based on at least the confidence data, mitigating action to take to mitigate the anomalous activity (fig.1, §III.D ¶2: deploying mitigating actions for attack data and low confidence data).
For claim 2, Mergendahl discloses the method of claim 1, as described above. Mergendahl further discloses: wherein processing the activity data comprises processing the activity data using a machine learning (ML) model trained to generate output confidence data representative of a set of output confidence values in response to processing input activity data indicative of an input anomalous activity (fig.1: Rapid, with fig.2 gives architectural overview of Rapid model, the model generating confidence values of fig.1 in response to input data, see §III.B, §III.C disclosing probabilistic severity degree output of LSTM) within at least one of: the computer system and a further computer system (fig.1), each output confidence value representative of a confidence that the input anomalous activity comprises a respective type of activity (§III.C ¶1: probabilistic severity degree confidence of DDoS).
For claim 4, Mergendahl discloses the method of claim 2, as described above. Mergendahl further discloses: wherein the confidence data is calibrated confidence data, the set of confidence values is a set of calibrated confidence values (§IV.C contemplates calibrating confidence data for deployment, with ¶2 disclosing using temperature scaling), processing the activity data using the ML model comprises processing the activity data using the ML model to generate initial confidence data representative of a set of initial confidence values (fig.1, §IV.C: initial values are generated for feeding into temperature scaling calibration), and processing the activity data further comprises calibrating the set of initial confidence values to generate the set of calibrated confidence values (§IV.C).
For claim 5, Mergendahl discloses the method of claim 4, as described above. Mergendahl further discloses: wherein calibrating the set of initial confidence values comprises obtaining the set of calibrated confidence values from the set of initial confidence values by processing the set of initial confidence values using a non-parametric calibration function (§IV.C: temperature scaling calibration; since temperature scaling is optimized via tuning the parameter T to minimize NLL loss (see Guo, reference 38 incorporated by reference, §2: Negative Log Likelihood; §4.1: Platt Scaling, §4.2: Temperature Scaling: optimizing T or a, b over a validation set via NLL, hence, function is adjusted to match validation set via tuning without additional parameters).
For claim 6, Mergendahl discloses the method of claim 5, as described above. Mergendahl further discloses: wherein the non-parametric calibration function is an isotonic function (§IV.C: temperature scaling, with Guo §4.1: Platt Scaling, §4.2: Temperature Scaling disclosing a scaling via a temperature parameter applied at the logit level that would not shuffle the order of the results, hence, isotonic).
For claim 7, Mergendahl discloses the method of claim 2, as described above. Mergendahl further discloses: wherein the ML model is a calibrated ML model (§IV.C).
For claim 8, Mergendahl discloses the method of claim 1, as described above. Mergendahl further discloses: wherein determining the mitigating action to take comprises determining the mitigating action to take based on a comparison between at least one of the set of confidence values and a threshold value (§III.D ¶2, fig.1: determining mitigating action based on low confidence values, hence, lower than a threshold).
For claim 10, Mergendahl discloses the method of claim 1, as described above. Mergendahl further discloses: wherein:
the set of confidence values comprises:
a first confidence value representative of a first confidence that the anomalous activity comprises a first type of activity (fig.1: mitigating actions are taken or not after processing by ML network (fig.2), with §III.C ¶2 disclosing confidence output as a probabilistic severity degree p, hence, a first value of p specifying anomalous activity of high confidence); and
a second confidence value representative of a second confidence that the anomalous activity comprises a second type of activity, different from the first type of activity (fig.1, §III.D: a second confidence value representing uncertainty behavior, to be routed to mitigation); and
determining the mitigating action to take comprises determining the mitigating action to take based on a comparison between the first confidence value and the second confidence value (ibid: mitigating action such as in §III.D taken based on determination of that confidence value is lower than certainty threshold).
For claim 11, Mergendahl discloses the method of claim 1, as described above. Mergendahl further discloses: comprising:
computing, based on at least one of: a confidence value of the set of confidence values (§III.D ¶2, fig.1: confidence value triggering mitigation system is computed), a severity of the anomalous activity, and a criticality of the computer system to operation of a telecommunications network comprising the computer system, a risk metric indicative of a risk to the telecommunications network of the anomalous activity (ibid: risk metric is determined if confidence based on comparison of confidence with threshold, i.e., too low, for routing to mitigation analysis); and
determining to take the mitigating action based on the risk metric (§III.D ¶2: mitigating action is performed).
For claim 12, Mergendahl discloses the method of claim 1, as described above. Mergendahl further discloses: wherein the activity data represents at least one intrusion detection alert (§III.A ¶2).
For claim 13, Mergendahl discloses the method of claim 1, as described above. Mergendahl further discloses: wherein the types of activity represented by the set of confidence values comprise at least one malicious activity and at least one benign activity (fig.1, §III.C ¶2).
For claim 14, Mergendahl discloses: a computer-implemented method of calibrating a system comprising a machine learning (ML) model trained to generate output uncalibrated confidence data representative of a set of output uncalibrated confidence values in response to processing input activity data indicative of an input anomalous activity within a computer system (fig.1 shows Rapid ML model trained to generate uncalibrated confidence values (high and low confidence paths) in response to input data (§III.B: sFlow packet data) indicative anomalous activity, see §III.A ¶2), each output uncalibrated confidence value representative of an uncalibrated confidence that the input anomalous activity comprises a respective type of activity (ibid), the method comprising:
processing, using the ML model, calibration activity data representative of an anomalous activity (fig.1: high and low output confidence levels would be calibration activity data) within at least one of: the computer system and a further computer system (fig.1 shows processing of data within the IoT computer system), to generate uncalibrated confidence data representative of a set of uncalibrated confidence values (fig.1 shows generation of uncalibrated confidence data for uncalibrated confidence values), each uncalibrated confidence value representative of an uncalibrated confidence that the anomalous activity comprises a respective type of activity (ibid); computing an uncertainty metric associated with the set of uncalibrated confidence values (§IV.C: ¶1 contemplates calibrating the model for deployment, with ¶2 contemplating computing uncertainty metrics for performing temperature scaling; with Guo (incorporated by reference as reference 38) disclosing temperature scaling (Guo §4.1: Platt scaling, §4.2: Temperature Scaling) involving the calculation of various uncertainty metrics, e.g., the NLL metric when optimizing the NLL on the validation set (§4.2: Temperature scaling, §4.2: Platt Scaling both contemplate optimizing NLL, see §2: Negative Log Likelihood) ); and adjusting parameters associated with the ML model, based on the uncertainty metric, to calibrate the ML model, thereby generating a calibrated ML model (ibid: temperature scaling calibration is applied to the ML model to adjust model parameters to generate a calibrated model).
For claim 15, Mergendahl discloses the method of claim 14, as described above. Mergendahl further discloses: wherein the uncertainty metric is indicative of a dissimilarity between at least one of the set of uncalibrated confidence values and a corresponding at least one of a set of ground truth confidence values (Gau §4.2, §2: NLL, incorporated by reference: NLL indicates disparity between uncalibrated value and ground truth label).
For claim 16, Mergendahl discloses the method of claim 14, as described above. Mergendahl further discloses: wherein adjusting the parameters associated with the ML model comprises adjusting the parameters associated with the ML model to reduce the uncertainty metric (Gau §4.2, §2: NLL, incorporated by reference: T is adjusted to reduce NLL uncertainty metric).
For claim 19, Mergendahl discloses the method of claim 1, as described above. Mergendahl further discloses: a computer-readable medium storing thereon a program for carrying out the method of claim 1 (§5 gives model details including computation details, see “computation time”, hence, computer-readable medium for storing data and programs).
For claim 20, Mergendahl discloses: a computer system configured to implement:
an intrusion detection system (IDS) (fig.1, §III.A ¶2 contemplates DDoS intrusion detection systems) to:
obtain activity data indicative of an anomalous activity within at least one of: the computer system and a further computer system (fig.1, §III.B contemplates obtaining activity data with the computing system); and
process the activity data to generate confidence data representative of a set of confidence values (fig.1: processing by Rapid model to generate confidence values, see also §III.D: mitigation system for low-confidence output), each confidence value representative of a confidence that the anomalous activity comprises a respective type of activity (ibid); and
an intrusion response system (IRS) (fig.1, §III.D: mitigation system) to: obtain the confidence data from the intrusion detection system (ibid); and determine, based on at least the confidence data, mitigating action to take to mitigate the anomalous activity (§III.D).
For claim 21, Mergendahl discloses the method of claim 20, as described above. Mergendahl further discloses: wherein, to process the activity data, the IDS is configured to process the activity data using a machine learning (ML) model trained to generate output confidence data representative of a set of output confidence values in response to processing input activity data indicative of an input anomalous activity (fig.1: Rapid, with fig.2 gives architectural overview of Rapid model, the model generating confidence values of fig.1 in response to input data, see §III.B, §III.C disclosing probabilistic severity degree output of LSTM) within at least one of: the computer system and a further computer system (fig.1), each output confidence value representative of a confidence that the input anomalous activity comprises a respective type of activity (§III.C ¶1: probabilistic severity degree confidence of DDoS).
For claim 22, Mergendahl discloses the method of claim 21, as described above. Mergendahl further discloses: wherein the confidence data is calibrated confidence data, the set of confidence values is a set of calibrated confidence values (§IV.C contemplates calibrating confidence data for deployment, with ¶2 disclosing using temperature scaling), and, to process the activity data, the IDS is configured to process the activity data using the ML model to generate initial confidence data representative of a set of initial confidence values (fig.1, §IV.C: initial values are generated for feeding into temperature scaling calibration), and processing the activity data further comprises calibrating the set of initial confidence values to generate the set of calibrated confidence values (§IV.C).
For claim 23, Mergendahl discloses the method of claim 22, as described above. Mergendahl further discloses: wherein, to calibrate the set of initial confidence values, the IDS is configured to obtain the set of calibrated confidence values from the set of initial confidence values by processing the set of calibrated confidence values using a non-parametric calibration function (§IV.C discloses using temperature scaling to calibrate the set of initial confidence values, see Guo §4.1-4.2, the set of calibrated confidence values hence being processed via a non-parametric calibration function to generate them).
For claim 24, Mergendahl discloses the method of claim 20, as described above. Mergendahl further discloses: a telecommunications network comprising the computer system of claim 20 (fig.1 shows implementation via telecommunications traffic, e.g., a WAN, IoT, internet).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 3, 17 are rejected under 35 U.S.C. 103 as being unpatentable over Mergendahl ("Rapid: Robust and adaptive detection of distributed denial-of-service traffic from the internet of things", published 2020) in view of Liu ("Opprentice: Towards practical and automatic anomaly detection through machine learning", published 2015).
For claim 3, Mergendahl discloses the method of claim 2, as described above. Mergendahl further discloses: wherein the ML model comprises a random forest model.
Liu discloses: wherein the ML model comprises a random forest model (fig.3, §4.1).
It would have been obvious before the effective filing date to one of ordinary skill in the art to modify the method of Guo by incorporating the isotonic regression calibration technique of Guo. Both concern the art of calibrated anomaly detection via machine learning, and the incorporation would have, according to Liu, allow for machine learning and user customizable methods to acquire accurate anomaly detection via monitored metrics (§1: Detector Challenges).
For claim 17, Mergendahl discloses the method of claim 14, as described above. Mergendahl further discloses: wherein the system comprises a non-parametric calibration model (§IV.C: temperature scaling calibration; since temperature scaling is optimized via tuning the parameter T to minimize NLL loss (see Guo, reference 38 incorporated by reference, §2: Negative Log Likelihood; §4.1: Platt Scaling, §4.2: Temperature Scaling: optimizing T or a, b over a validation set via NLL, hence, function is adjusted to match validation set via tuning without additional parameters), and the method comprises:
processing the calibration activity data using an ML model to generate initial confidence data representative of a set of initial confidence values (§IV.C: calibration activity monitoring data is processed via the MLP / LSTM (fig.2) to generate initial confidence values p); and
adjusting the non-parametric calibration model to fit a non-parametric calibration function represented by the non-parametric calibration model to the set of initial confidence values (§IV.C: temperature scaling is performed, with Guo §4.1 Platt Scaling and §4.2 Temperature Scaling disclosing the adjusting of the model temperature function to minimize NLL based on fit to initial confidence values), wherein the non-parametric calibration function is useable to obtain the set of calibrated confidence values from the initial confidence values (§IV.C ibid: initial values are calibrated).
Mergendahl does not disclose: wherein an ML model is the calibrated ML model
Liu discloses: wherein an ML model is the calibrated ML model (§3.2 “Incomplete anomaly cases” discloses incremental retraining hence, combination with Mergendahl yielding a previously calibrated model may be retrained and recalibrated).
It would have been obvious before the effective filing date to one of ordinary skill in the art to modify the method of Guo by incorporating the retraining technique of Liu. Both concern the art of calibrated anomaly detection via machine learning, and the incorporation would have, according to Liu, adapt to emergent new anomalies (§3.2 “Incomplete anomaly cases”).
Claim(s) 9 are rejected under 35 U.S.C. 103 as being unpatentable over Mergendahl ("Rapid: Robust and adaptive detection of distributed denial-of-service traffic from the internet of things", published 2020) in view of Basak (US 20190130101 A1).
For claim 9, Mergendahl discloses the method of claim 8, as described above. Mergendahl further discloses: wherein the threshold value is a first threshold value and determining the mitigating action to take comprises:
determining to take a second mitigating action, different from the first mitigating action, in response to determining that the confidence value is between a second threshold value and the first threshold value, wherein the second threshold value is lower than the first threshold value (fig.1: mitigating actions are taken or not after processing by ML network (fig.2), with §III.C ¶2 disclosing confidence output as a probabilistic severity degree p, hence, mitigating actions, such as in §III.D ¶2 are taken for confidences in an uncertain region between a certain threshold (high confidence of attack) and above a low threshold (low confidence attack, hence, benign).
Mergendahl does not disclose: determining to take a first mitigating action in response to determining that a confidence value of the set of confidence values exceeds the first threshold value.
Basak discloses: determining to take a first mitigating action in response to determining that a confidence value of the set of confidence values exceeds the first threshold value (fig.3:390, 0045: implementing mitigating actions on positive detection, hence, combination with Mergendahl yielding taking actions in response to detector of Mergendahl).
It would have been obvious before the effective filing date to one of ordinary skill in the art to modify the method of Mergendahl by incorporating the mitigating action of Basak. Both concern the art of anomaly detection, and the incorporation would have implemented appropriate response actions in run time settings (0045).
Claim(s) 18 are rejected under 35 U.S.C. 103 as being unpatentable over Mergendahl ("Rapid: Robust and adaptive detection of distributed denial-of-service traffic from the internet of things", published 2020) in view of Liu ("Opprentice: Towards practical and automatic anomaly detection through machine learning", published 2015) in view of Guo ("On calibration of modern neural networks", published 2017).
For claim 18, Mergendahl modified by Liu discloses the method of claim 14, as described above. Mergendahl does not disclose: wherein fitting the non-parametric calibration function comprises fitting the non-parametric calibration function using isotonic regression.
Guo discloses: wherein fitting the non-parametric calibration function comprises fitting the non-parametric calibration function using isotonic regression (Guo §4.1: Isotonic Regression).
It would have been obvious before the effective filing date to one of ordinary skill in the art to modify the method of Guo by incorporating the isotonic regression calibration technique of Guo. Both concern the art of calibration for machine learning, and the incorporation would have, according to Guo, apply a widely used common-place calibration method to improve calibration (Guo §4.1: Isotonic Regression).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Peroulas (US 20200342311 A1) discloses network anomaly detection based on machine learning.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LIANG LI whose telephone number is (303)297-4263. The examiner can normally be reached Mon-Fri 9-12p, 3-11p MT (11-2p, 5-1a ET).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. The examiner is available for interviews Mon-Fri 6-11a, 2-7p MT (8-1p, 4-9p ET).
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor Jennifer Welch can be reached on (571)272-7212. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center and the Private Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from Patent Center or Private PAIR. Status information for unpublished applications is available through Patent Center or Private PAIR to authorized users only. Should you have questions about access to Patent Center or the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
/LIANG LI/
Primary examiner AU 2143
Prosecution Insights