Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
The instant application having Application No. 18/568,923 is presented for examination by the examiner. Claims 47 and 65 are amended. Claims 47-66 have been examined.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant’s submission filed on 03/30/2026 has been entered.
Claims 47 and 65 are independent claims. The applicant has amended claims 47 and 65. Claims 47-56 and 59-66 have been examined and are pending. This Action is made Non-FINAL.
Response to Arguments
Applicant’s arguments with respect to claim(s) 47 and 65 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
The Examiner respectfully suggests that the claim be further amended; details in the specification be incorporated, to distinguish the claimed invention over prior art of record. Should the Applicant desire an interview to further clarify the claim interpretation/rejections, please contact the Examiner at (571) 272-1531 to schedule an interview.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 47-48, 54-55, 59-60 and 64-66 is/are rejected under 35 U.S.C. 103 as being unpatentable over Pierri (US 2021/0349897 A1), in view of Miller (US 20210019399 A1) and in view of Zang (US 8,396,451 B1).
Regarding Claim 47
Pierri discloses:
A device capable of operating in a communications network and predicting malicious threats by determining a (Pierri ¶0018, 0037–0039, 0070: discloses a computing system that receives environment sensor data over a network as time-series events and applies a machine-learning anomaly detection algorithm to identify subsets of measurands whose values indicate anomalous behavior, thereby associating each event with an indication of a threat condition.) events of the environment threat data, one or more anomaly predictions relating to if the one or more events are malicious or not, wherein the anomaly prediction comprises a probability value that an event is malicious (Pierri ¶0024, 0069–0070: discloses computing, for events of environment threat data, anomaly predictions using a trained machine-learning model (e.g., an autoencoder) that generates anomaly scores indicating whether events are anomalous, wherein the anomaly scores are normalized to a [0,1] range and thus represent a probability-like value indicating likelihood that an event corresponds to malicious behavior), and report the one or more anomaly predictions (Pierri ¶0018, 0036–0039, 0070: discloses outputting anomaly predictions by providing anomaly scores and anomaly indications generated by a trained machine-learning model to other system components (e.g., for root-cause analysis or further processing), thereby reporting the anomaly predictions).
Pierri teaches a device comprising one or more processors and memory executing instructions to compute anomaly predictions and current metric scores for environment threat data using a machine learning model, where anomaly scores are generated, normalized, and evaluated relative to expected or normal behavior ranges (Pierri ¶¶0016, 0069–0070). However, Pierri is silent with respect to obtaining one or more target metric scores based on cost and/or risk values associated with identifying anomalous events, deriving a decision-making threshold (DMT) from such target metric scores, comparing anomaly predictions with the DMT to obtain malicious threat predictions, computing current metric scores based on labels and prediction outcomes, and adapting the DMT based on whether the current metric scores fall within a range of the target metric scores. On the other hand, Miller teaches selecting, applying, and adapting decision thresholds based on cost and risk performance metrics. Miller discloses defining thresholds that upper-bound false positives and false negatives according to acceptable risk tolerances, which are performance metrics directly associated with the cost and risk of identifying anomalous events (¶0014, 0073). Miller further teaches comparing anomaly prediction values, such as detection statistics or p-values, with a decision threshold to determine whether an event constitutes a malicious threat, and computing current performance metrics using known labels and observed prediction outcomes (¶0073–0081). Miller also teaches adapting the decision threshold when observed performance deviates from acceptable bounds, in order to maintain desired cost and risk objectives over time (¶0073–0074).
It would have been obvious to one of ordinary skill in the art at the time of the invention to modify Pierri’s anomaly detection system to incorporate Miller’s cost and risk based threshold determination and adaptation techniques, because both references address automated detection of anomalous or malicious behavior using machine learning, and Miller’s threshold control techniques predictably improve Pierri by enabling dynamic control of false-positive and false-negative outcomes. Combining these known techniques merely applies a well understood decision threshold optimization to an existing anomaly detection system and would have yielded predictable results.
Pierri in view of Miller teaches an ML anomaly detection system that computes probability anomaly predictions for threat events and adapts a decision threshold based on cost and risk based false-positive and false-negative performance tolerances. However, Pierri in view of Miller does not teach or suggest wherein adapting the DMT comprises updating the DMT and repeatedly computing one or more current metric scores for the environment threat data using the updated DMT until the computed one or more current metric scores is within a range of the target metric scores. On the other hand, Zang teaches a method of determining a classification threshold for generating fraud indications in a communications network wherein the system computes a fraud metric against labeled fraudulent and non-fraudulent user data, compares those metrics to a classification threshold to generate tentative classifications, and obtains feedback comprising a false positive rate and false negative rate from the resulting classifications (Column 5, Lines 20-47). Zang further teaches that the threshold is adjusted in response to that feedback and that the computing, comparing, and feedback steps are repeated using the adjusted threshold, continuing until a threshold is found that simultaneously satisfies both the maximum permissible false positive rate and the maximum permissible false negative rate, at which point the classifications are finalized (Column 5, Line 40 - Column 6, Line 2).
It would have been obvious to one of ordinary skill in the art at the time of the invention to modify Pierri's anomaly detection system to incorporate Miller's cost and risk based threshold determination and adaptation techniques, and further to incorporate Zang's iterative threshold convergence mechanism, because all three references address automated detection of anomalous or malicious behavior in communications networks using machine learning classifiers that output probability scores compared against decision thresholds. Miller's threshold control techniques predictably improve Pierri by enabling dynamic control of false-positive and false-negative outcomes, and Zang's iterative convergence loop predictably improves the Pierri and Miller combination by automating the threshold adaptation process until the system's detection performance meets its desired cost and risk objectives. Combining these known techniques merely applies a well understood iterative threshold optimization technique to an existing anomaly detection system and would have yielded predictable results.
Regarding Claim 48
Pierri discloses:
The device according to claim 47, wherein the environment threat data is represented as a table consisting of one or more rows and one or more columns, each row representing the event and one column of the one or more columns representing the label (Pierri ¶0067–0069: teach processing input events xix_ixi as structured data points for anomaly detection using a machine-learning model, where each data point represents an individual event comprising multiple measured values, and where an anomaly score or anomaly indication is associated with each event, which corresponds to a table-based representation in which each row represents an event and at least one column represents a label indicating whether the event is anomalous.).
Regarding Claim 54
Pierri discloses:
The device according to claim 47, comprising one or more processor(s) and memory, said memory containing instructions which when executed on the one or more processor(s) cause the device to:
compute the one or more anomaly predictions using a single row in the table; or
compute the one or more anomaly predictions using multiple rows in the table (Pierri ¶67-69: teaches computing anomaly predictions using both individual input data points (e.g., xi) and subsets of multiple data points, which correspond to single or multiple rows in a table of environment threat data.).
Regarding Claim 55
Pierri discloses:
The device according to claim 47, comprising one or more processor(s) and memory, said memory containing instructions which when executed on the one or more processor(s) cause the device to:
compute the anomaly prediction by training a machine learning, ML, model with the environment threat data without one or more threat events (Pierri ¶69: teaches training an unsupervised machine learning model (specifically an autoencoder) on environment threat data representing only normal traffic behavior, thereby excluding one or more threat events during training.).
Regarding Claim 59
Pierri in view of Zang teaches computing anomaly predictions and associated metric values for environment threat data using a machine learning anomaly detection system, where anomaly scores are evaluated relative to expected or normal behavior ranges derived from reference values (Pierri ¶¶0016, 0018, 0069–0070). Miller further teaches defining decision-making thresholds and acceptable ranges based on target metric scores, such as false-positive and false-negative tolerances, where detection outcomes are evaluated based on whether computed values fall within predefined limits associated with those target metrics (Miller ¶¶0014, 0073–0075).
It would have been obvious to one of ordinary skill in the art at the time of the invention to apply Miller to Pierri and Zang teaching, as Pierri teaches anomaly detection metrics so that the evaluated range falls within limits defined by target metric scores, because the references address improving detection accuracy by bounding decision outcomes within acceptable performance limits, and their combination yields the predictable result of evaluating anomaly decisions against defined target metric limits.
Regarding Claim 60
Pierri in view of Zang teaches a device comprising one or more processors and memory executing instructions to perform anomaly detection on environment threat data and to generate anomaly predictions and associated metric values using a machine learning model (Pierri ¶¶0016, 0069–0070). Although Pierri and Zang does not explicitly teach selecting a new decision-making threshold, Miller expressly teaches adapting or updating a decision-making threshold by selecting a new threshold value based on detection performance criteria, such as false-positive and false-negative tolerances or other target metrics (Miller ¶¶0014, 0073–0075).
It would have been obvious to one of ordinary skill in the art at the time of the invention to modify the teachings of Pierri and Zang to adapt or update the decision making threshold by selecting a new DMT as taught by Miller, because the references are directed to improving detection accuracy through threshold adjustment, and combining these teachings yields the predictable result of dynamically updating the threshold used to determine malicious events.
Regarding Claim 64
Pierri in view of Zang teaches a device comprising one or more processors and memory executing instructions to perform anomaly detection on environment threat data using a machine-learning model and to generate anomaly predictions used to identify malicious events (Pierri ¶¶0016, 0069–0070). Miller expressly teaches selecting and adjusting decision-making thresholds in order to control and reduce false positives in anomaly or attack detection, including setting new threshold values based on acceptable false-positive tolerances and deployment-specific requirements (Miller ¶¶0014, 0073–0075).
It would have been obvious to one of ordinary skill in the art at the time of the invention to modify Pierri and Zang anomaly detection system to select a new decision-making threshold for the purpose of reducing the number of false positives, as taught by Miller, because the references address improving detection accuracy by managing false-positive rates, and their combination yields the predictable result of reducing false positives in anomaly detection through threshold adjustment.
Regarding Claim 65
Claim 65 is directed to a method corresponding to the computer-implemented method in claim 47. Claim 65 is similar in scope to claim 47 and is therefore rejected under similar rationale.
Regarding Claim 66
Pierri in view of Zang teaches a computer implemented anomaly detection system comprising one or more processors and memory executing instructions to obtain environment threat data, compute anomaly predictions, and generate malicious threat determinations using a machine-learning model (Pierri ¶¶0016, 0069–0070). Miller further teaches determining, selecting, and updating a decision-making threshold based on target metrics such as false-positive and false-negative tolerances, including adapting the threshold to improve detection performance (Miller ¶¶0014, 0073–0075). These teachings require executable program instructions stored on a non-transitory computer-readable storage medium to cause a device to perform the claimed threshold-determination and adaptation method.
It would have been obvious to one of ordinary skill in the art at the time of the invention to implement the combined methods of Pierri, Miller and Zang as a computer program product stored on a computer-readable storage medium, because such an implementation is a routine and predictable way of deploying software-based anomaly detection and threshold-selection techniques, and yields no more than the expected result of executing the claimed method on a computing device.
Claims 49 is/are rejected under 35 U.S.C. 103 as being unpatentable over Pierri (US 2021/0349897 A1), in view of Miller (US 20210019399 A1), in view of Zang (US 8,396,451 B1) as applied to claim 47 above, and further in view of Gates (US 2019/0098039 A1).
Regarding Claim 49
Pierri discloses:
The device according to claim 47, comprising one or more processor(s) and memory, said memory containing instructions which when executed on the one or more processor(s) cause the device to:
indicate the threat event for the environment threat data if the label is 1 (Pierri ¶70: teaches using a trained machine learning model to output normalized anomaly scores between 0 and 1, where a score of 1 indicates a strong anomaly (threat event), effectively serving as a label indicating the presence of a threat event in the environment threat data.);
Pierri, Miller and Zang combined teaches using an anomaly detection system that processes time-series sensor data to compute anomaly scores, compares them to thresholds and adapts the model based on root cause feedback to detect threats in real time. However, they do not disclose the following limitation “obtain the threat event which is generated by a function simulating threat data and inserted in the environment data; and/or obtain the threat event which is generated by a dataset of real threat captures and inserted in the environment data”.
However, in an analogous art, Gates discloses a threat detection engine system/method that includes:
obtain the threat event which is generated by a dataset of real threat captures and inserted in the environment data (Gates ¶51-53: teach generating threat events by forming combined entities (super-entities) from input data using techniques such as masking attributes and text processing, and inserting these entities into an environment data universe for real-time threat detection and analysis.).
Given the teachings of Gates a person having ordinary skill in the art before the effective filing date of the claimed invention would have found it obvious to modify the teachings of Pierri, Miller and Zang by forming entities representing threat events by combining multiple data items with masked or ignored attributes to capture coordinated attacks. Gates further discloses inserting these entities into an environment data universe for real time threat detection and action recommendation, trading some accuracy for speed. These disclosures collectively teach generating threat events from processed and combined data inserted into environment threat data for detection purposes. It would have been obvious to one of ordinary skill in the art to generate such threat events by combining and processing input data to improve real time cybersecurity threat detection, consistent with known practices in threat modeling and data conditioning (Gates ¶51-53).
Claims 50 is/are rejected under 35 U.S.C. 103 as being unpatentable over Pierri (US 2021/0349897 A1), in view of Miller (US 20210019399 A1), in view of Zang (US 8,396,451 B1) as applied to claim 47 above, and in further view of Lijuan (US 2023/0231871 A1).
Regarding Claim 50
Pierri, Miller and Zang combined teaches using an anomaly detection system that processes time-series sensor data to compute anomaly scores, compares them to thresholds and adapts the model based on root cause feedback to detect threats in real time. However, they do not disclose the following limitation “wherein the threat event comprises one or more threats such as initial access, lateral movement, credential access and denial of service”.
However, in an analogous art, Lijuan discloses a threat event system/method that includes:
The device according to claim 47, wherein the threat event comprises one or more threats such as initial access, lateral movement, credential access and denial of service (Lijuan ¶243, 246: teaches identifying threat events including intranet brute force cracking (initial access), sensitive information leakage (credential access), and external network DDoS attacks (denial of service) within a monitored environment. These correspond directly to the claimed threat types, demonstrating the detection and classification of such threats in network traffic data.).
Given the teachings of Lijuan a person having ordinary skill in the art before the effective filing date of the claimed invention would have found it obvious to modify the teachings of Pierri, Miller and Zang by identifying specific cybersecurity threat events such as intranet brute force cracking attacks, which correspond to initial access; sensitive information leakage involving usernames or passwords, which corresponds to credential access; and external network DDoS attacks, which correspond to denial of service. These threat events are detected in real time based on analysis of traffic flow metadata and are used to classify the flow as malicious. The system adds such flows to a malicious sample set for further processing. These disclosures collectively teach that threat events can comprise well known categories such as initial access, credential access, and denial of service. It would have been obvious to one of ordinary skill in the art to include such standard threat types as part of a threat event and is common practices in threat modeling (Lijuan ¶243-246).
Claims 51-53, 56, 61 and 63 is/are rejected under 35 U.S.C. 103 as being unpatentable over Pierri (US 2021/0349897 A1), in view of Miller (US 20210019399 A1), in view of Zang (US 8,396,451 B1) as applied to claim 47 above, and further in view of LIPOSKY (US 2022/0108238 A1).
Regarding Claim 51
Pierri, Miller and Zang teaches using an anomaly detection system that processes time-series sensor data to compute anomaly scores, compares them to thresholds and adapts the model based on root cause feedback to detect threats in real time. However, they do not disclose the following limitation “wherein the cost value comprises at least one of number of false positives relating to the event of the environment threat data”.
However, in an analogous art, LIPOSKY discloses a cost value system/method that includes:
The device according to claim 47, wherein:
the target metric score is calculated using one or more cost values,
wherein the cost value comprises at least one of number of false positives relating to the event of the environment threat data (LIPOSKY ¶184-188: Teaches calculating model evaluation metrics that include the number of false positives, as part of a system that evaluates AI-generated risk predictions for operational loss events; specifically, the system tracks false positives during model evaluation using precision and F1-score calculations to inform performance and adaptation of predictive models.),
number of false negatives relating to the event (LIPOSKY ¶184-188: Teaches calculating model evaluation metrics that include the number of false negatives, as part of a system that assesses the accuracy and reliability of AI models used to predict operational loss events; the number of false negatives is explicitly tracked through recall and F1-score calculations, informing risk evaluation and model retraining.),
cost of false positives relating to the event (LIPOSKY ¶184-188: Teaches considering the cost of false positives relating to an event as part of a model performance evaluation process, where the system accounts for differing impacts of prediction errors by recommending analysis of both precision and F1-score when false positive and false negative costs diverge.),
cost of false negatives relating to the event (LIPOSKY ¶187-188: Teaches considering the cost of false negatives relating to an event by evaluating predictive model performance in scenarios where the cost of misclassifying an actual threat (i.e., false negative) is significant, and by using F1-score and recall metrics to balance and reduce such high-impact errors.),
cost of the threat event (LIPOSKY ¶62, 71 and 85: Teaches considering the cost of the threat event by generating risk scores that estimate the probability of an operational loss occurring from a transaction, and using those scores to drive targeted mitigations and recommendations aimed at reducing the financial and procedural impact of such events.),
probability of occurrence of the threat event (LIPOSKY ¶64 and 71: Teaches calculating the probability of occurrence of a threat event by generating risk scores using predictive models trained on historical and current data, where each risk score represents the likelihood that a transaction or user action will result in an operational loss.),
frequency of occurrence of the threat event (LIPOSKY ¶92 and 247: Teaches determining the frequency of occurrence of a threat event by identifying process steps or business areas that are frequently associated with operational loss, and increasing or decreasing monitoring accordingly; additionally, the system supports periodic execution of AI models based on administrator-defined frequencies to evaluate risk likelihoods.),
importance weight associated with the false positives and importance weight associated with false negatives (LIPOSKY ¶188: Teaches assigning importance weights to false positives and false negatives by evaluating predictive models using F1-score, a metric that combines precision and recall based on their relative significance, and further emphasizes adjusting performance analysis when the cost or impact of false positives differs from that of false negatives.).
Given the teachings of LIPOSKY a person having ordinary skill in the art before the effective filing date of the claimed invention would have found it obvious to modify the teachings of Pierri, Miller and Zang by generates risk scores for operational loss events using predictive models evaluated by precision, recall, and F1-score metrics, which incorporate both false positives and false negatives. LIPOSKY further discloses that these metrics are selected based on the relative cost or importance of false positives and false negatives, and that model training and adaptation use organization-specific data tied to historical error patterns. These disclosures collectively teach assigning importance weights to false positives and false negatives when computing target metric scores. It would have been obvious to one of ordinary skill in the art to include such weighting to improve risk model relevance in cost-sensitive decision contexts, consistent with standard practices in model evaluation and tuning (LIPOSKY ¶64, 71, 92, 181 and 247).
Regarding Claim 52
Pierri, Miller and Zang teaches using an anomaly detection system that processes time-series sensor data to compute anomaly scores, compares them to thresholds and adapts the model based on root cause feedback to detect threats in real time. However, they do not disclose the following limitation “wherein the current metric score and the target metric score comprise at least one of the following metrics: precision, recall and F1 score”.
However, in an analogous art, LIPOSKY discloses a cost value system/method that includes:
The device according to claim 47, wherein the current metric score and the target metric score comprise at least one of the following metrics: precision, recall and F1 score (LIPOSKY ¶184–¶189: Teaches calculating precision, recall, and F1 score as model evaluation metrics for predictive models assessing operational loss risks. Precision and recall are defined using true/false positives and negatives and the F1 score is calculated as their weighted average, enabling performance assessment and model adaptation.), wherein each metric is calculated using at least one of number of False Positives, number of False Negatives and number of True Negatives (LIPOSKY ¶184–¶189: Teaches that precision, recall, and accuracy are calculated using standard classification components, including false positives (FP), false negatives (FN), and true negatives (TN). Specifically, precision uses FP, recall uses FN, and accuracy uses TN along with other values, establishing that each metric relies on at least one of these values during evaluation.).
Given the teachings of LIPOSKY a person having ordinary skill in the art before the effective filing date of the claimed invention would have found it obvious to modify the teachings of Pierri, Miller and Zang by using standard classification metrics, including precision, recall, and F1 score, to assess model performance in detecting operational loss risks. These metrics are explicitly calculated using values such as false positives, false negatives, and true negatives, and are displayed within a model evaluation interface to guide system tuning and retraining. For instance, precision is calculated as the ratio of true positives to the sum of true positives and false positives, recall as the ratio of true positives to the sum of true positives and false negatives, and the F1 score as the harmonic mean of precision and recall. A person of ordinary skill in the art would have found it obvious to apply these well-established metrics and classification components when evaluating model performance in threat detection or risk classification systems, as doing so reflects standard practices in supervised machine learning and model evaluation to ensure reliable decision-making based on labeled outcomes (LIPOSKY ¶184–¶189).
Regarding Claim 53
Pierri, Miller and Zang teaches using an anomaly detection system that processes time-series sensor data to compute anomaly scores, compares them to thresholds and adapts the model based on root cause feedback to detect threats in real time. However, they do not disclose the following limitation “wherein the F1 score is a harmonic mean of recall and precision”.
However, in an analogous art, LIPOSKY discloses a cost value system/method that includes:
The device according to claim 52, wherein the F1 score is a harmonic mean of recall and precision (LIPOSKY ¶0189: Teaches that the F1 score is a weighted average of precision and recall, which in standard machine learning practice refers to the harmonic mean of the two. This metric is used to evaluate model performance by balancing false positives and false negatives.).
Given the teachings of LIPOSKY a person having ordinary skill in the art before the effective filing date of the claimed invention would have found it obvious to modify the teachings of Pierri, Miller and Zang by using standard classification metrics, including precision, recall, and F1 score, to assess model performance in detecting operational loss risks. These metrics are explicitly calculated using values such as false positives, false negatives, and true negatives, and are displayed within a model evaluation interface to guide system tuning and retraining. For instance, precision is calculated as the ratio of true positives to the sum of true positives and false positives, recall as the ratio of true positives to the sum of true positives and false negatives, and the F1 score as a weighted average of precision and recall. A person of ordinary skill in the art would have understood the F1 score to be the harmonic mean of precision and recall, as this is the standard definition in machine learning, and would have found it obvious to implement or interpret the F1 score accordingly to ensure balanced evaluation of both false positives and false negatives during predictive model assessment (LIPOSKY ¶184–¶189).
Regarding Claim 56
Pierri, Miller and Zang teaches using an anomaly detection system that processes time-series sensor data to compute anomaly scores, compares them to thresholds and adapts the model based on root cause feedback to detect threats in real time. However, they do not disclose the following limitation “comprising one or more processor(s) and memory, said memory containing instructions which when executed on the one or more processor(s) cause the device to compute the anomaly prediction based on a length of a leaf in a decision tree-based ML model”.
However, in an analogous art, LIPOSKY discloses a cost value system/method that includes:
The device according to claim 47, comprising one or more processor(s) and memory, said memory containing instructions which when executed on the one or more processor(s) cause the device to compute the anomaly prediction based on a length of a leaf in a decision tree-based ML model (LIPOSKY ¶181 and 189: Teaches generating anomaly predictions using a Two-Class Boosted Decision Tree Regression model, where each prediction is derived by traversing a sequence of learned regression trees built to minimize residual error using a differentiable loss function. This ensemble technique inherently relies on structural elements such as leaf nodes, and the influence of each tree’s depth and residual fit would be understood by a POSITA to contribute to the prediction outcome.).
Given the teachings of LIPOSKY a person having ordinary skill in the art before the effective filing date of the claimed invention would have found it obvious to modify the teachings of Pierri, Miller and Zang by generating anomaly or risk predictions using a Two-Class Boosted Decision Tree Regression model, where each regression tree is built in a step-wise fashion and optimized using residual error minimization and a differentiable loss function. Boosted decision tree ensembles inherently rely on structural elements such as the depth and configuration of leaves, as predictions are derived by traversing these trees and aggregating outputs based on the terminal nodes (leaves) reached. A person of ordinary skill in the art would understand that the length of a leaf (i.e., its depth in the tree) impacts how finely the model partitions the input space and thus directly affects the prediction output. It would have been obvious to such a person to base anomaly prediction at least in part on the structural depth of decision tree leaves, as this is a well-known parameter influencing model generalization and interpretability in ensemble tree models (LIPOSKY ¶181 and 189).
Regarding Claim 61
Pierri, Miller and Zang teaches using an anomaly detection system that processes time-series sensor data to compute anomaly scores, compares them to thresholds and adapts the model based on root cause feedback to detect threats in real time. However, they do not disclose the following limitation “wherein the new DMT is selected based on: a criterion of achieving the highest recall and/or precision; or a criterion of achieving the highest F1 score”.
However, in an analogous art, LIPOSKY discloses a cost value system/method that includes:
The device according to claim 60, wherein the new DMT is selected based on:
a criterion of achieving the highest recall and/or precision; or
a criterion of achieving the highest F1 score (LIPOSKY ¶185-187:Teaches selecting and evaluating predictive models using criteria based on achieving high recall, precision, or F1 score, with the F1 score specifically identified as a preferred metric when class distributions are imbalanced or when false positives and false negatives carry different costs.).
Given the teachings of LIPOSKY a person having ordinary skill in the art before the effective filing date of the claimed invention would have found it obvious to modify the teachings of Pierri, Miller and Zang by evaluating predictive models using precision, recall, and F1 score, where F1 is specifically recommended when class distributions are imbalanced or when the cost of false positives and false negatives differs. LIPOSKY explains that recall and precision may be prioritized when error costs diverge, and that F1 is a preferred metric in such cases, suggesting its use as a selection criterion. These disclosures collectively teach using performance metrics as a basis for selecting and tuning predictive models. It would have been obvious to one of ordinary skill in the art to apply a criterion of maximizing recall, precision, or F1 score to optimize model performance in a cost-sensitive classification system, as such use reflects standard model evaluation practices in the field of machine learning (LIPOSKY ¶185-187).
Regarding Claim 63
Pierri, Miller and Zang teaches using an anomaly detection system that processes time-series sensor data to compute anomaly scores, compares them to thresholds and adapts the model based on root cause feedback to detect threats in real time. However, they do not disclose the following limitation “wherein the new DMT is selected based on at least one of risk, cost and time available for current metric score computation”.
However, in an analogous art, LIPOSKY discloses a cost value system/method that includes:
The device according to claim 60, wherein the new DMT is selected based on at least one of risk, cost and time available for current metric score computation (LIPOSKY ¶80, 87, ¶90: Teaches threshold-based decision logic by comparing model-generated risk scores to predefined thresholds and triggering actions such as issuing recommendations, delaying trades, or reallocating tasks when thresholds are exceeded. LIPOSKY ¶62 and 85: further considers cost and operational complexity when deciding which processes to monitor or optimize, and adjusts decisions based on timing factors such as when during a user’s shift certain tasks are performed).
Given the teachings of LIPOSKY a person having ordinary skill in the art before the effective filing date of the claimed invention would have found it obvious to modify the teachings of Pierri, Miller and Zang by comparing model-generated risk scores against predefined thresholds to trigger automated actions such as issuing recommendations, reallocating tasks, or delaying trades when the threshold is exceeded. These threshold-based decisions are influenced by factors including operational risk, cost, and time. For instance, the system determines when and where to intervene based on the cost and complexity of operations to improve return on investment and reduce unnecessary monitoring, and adapts recommendations based on time-related factors such as user shift timing to minimize the likelihood of error. A person of ordinary skill in the art would have found it obvious to configure such a threshold-based decision mechanism to select values based on any one or more of risk, cost, or time, as doing so aligns with common design practices in intelligent process optimization systems to balance accuracy, efficiency, and operational impact (LIPOSKY ¶62, 80, 87, 90).
Claims 62 is/are rejected under 35 U.S.C. 103 as being unpatentable over Pierri (US 2021/0349897 A1), in view of Miller (US 20210019399 A1), in view of Zang (US 8,396,451 B1), in view of LIPOSKY (US 2022/0108238 A1) as applied to claim 61 above, and further in view of Duppils (US 2023/0036159 A1).
Regarding Claim 62
Pierri, Miller, Zang and LIPOSKY teaches using an anomaly detection system that processes time-series sensor data to compute anomaly scores, compares them to thresholds and adapts the model based on root cause feedback to detect threats in real time. However, they do not disclose the following limitation “wherein achieving the highest F1 score comprises performing, within a time period, iterations of the current metric scores computation using different DMTs and selecting, from among the DMTs, a DMT that achieves the highest F1 score as the new DMT”.
However, in an analogous art, Duppils discloses a multi-model system/method that includes:
The device according to claim 61, wherein achieving the highest F1 score comprises performing, within a time period, iterations of the current metric scores computation using different DMTs and selecting, from among the DMTs, a DMT that achieves the highest F1 score as the new DMT (Duppils ¶121, 209-211, 231: Teaches iteratively evaluating multiple models with different hyperparameters and selecting the model that achieves the highest F1 score, which aligns with performing iterations over decision metric thresholds (DMTs) and choosing the DMT that yields the best F1 score as the new threshold.).
Given the teachings of Duppils a person having ordinary skill in the art before the effective filing date of the claimed invention would have found it obvious to modify the teachings of Pierri, Miller, Zang and LIPOSKY by evaluating predictive models using standard classification metrics, including precision, recall, and F1 score, to assess model performance in classifying security-related content. Specifically, the F1 score is described as the harmonic mean of precision and recall, providing a balanced metric that accounts for both false positives and false negatives. Duppils further explains that classifiers are evaluated on test datasets with precision, recall, and F1 scores recorded for relevant classes to guide model comparison and selection. A person of ordinary skill in the art would have understood the use of the F1 score as a key metric for tuning and selecting the best performing model, making it obvious to perform iterative evaluations and select a decision metric threshold (DMT) that achieves the highest F1 score to optimize model performance (Duppils ¶121, 209-211, 231).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SAAD ABDULLAH whose telephone number is 571-272-1531. The examiner can normally be reached on Monday-Friday 9am-5pm EST. If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, LYNN FIELD can be reached on 571-272-2092.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SAAD AHMAD ABDULLAH/ Examiner, Art Unit 2431
/SHIN-HON (ERIC) CHEN/ Primary Examiner, Art Unit 2431
Prosecution Insights