Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
2. This action is in response to the original filing on 01/30/2024. Claims 1-20 are pending and have been considered below.
Information Disclosure Statement
3. The information disclosure statement (IDS(s)) submitted on 05/02/2024 is/are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 U.S.C. § 112
4. The following is a quotation of 35 U.S.C. § 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 7, 14, and 20 are rejected under 35 U.S.C. § 112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor regards as the invention. Dependent claims 7, 14, and 20 recite
PNG
media_image1.png
160
338
media_image1.png
Greyscale
The claims recite a mathematical formula for the bi-level optimization problem, including several variables above. However, the claims do not define these variables. Thus, the scope of is unclear.
Claim Rejections – 35 USC § 103
5. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
6. Claims 1-3, 6, 8-10, 13, and 15-17 are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al. (U.S. Patent Application Pub. No. US 20210067549 A1) in view of Qin (U.S. Patent Application Pub. No. US 20200372353 A1).
Claim 1: Chen teaches a computer-implemented method, comprising:
receiving (i.e. a data distributor 41 that receives the data from backend server 20 and distributes the corresponding information to non-GNN-based intrusion detection 42 and to adversarial GNN defense 43; non-GNN intrusion detection 42, which detects abnormal communication events using non-GNN detection systems; adversarial GNN defense 43, which provides adversarial training for GNN-based detection system 44; GNN-based intrusion detection 44, which uses a trained GNN to detect anomalous behavior in network gathered network information; para. [0025]), with at least one processor (i.e. A system for detecting and responding to an intrusion in a computer network includes a hardware processor and a memory; para. [0005]), a training dataset comprising graph data associated with a graph (i.e. a GNN can be a neural network parameterized function that leverages both graph structure and node features to learn the representation of a node/graph … both graph structure and node features can be represented with binary values, for example characterizing a connection between nodes with a ‘1’, and representing the fact that a node has a particular feature or attribute with a ‘1’; para. [0028, 0032]), the graph comprising a plurality of nodes and the graph data comprising node data associated with each node of the graph (i.e. The graph 1100 captures the topological structure of a dynamic network of objects, represented as nodes 1104 … Each node 1104 in the network 1100 includes one or more attributes or labels. These labels identify some characteristic of the node 1104 … Given a network's historical records, a sequence of communication graphs can be constructed, where each node is a computational device and each edge indicates a communication. Each node can be associated with characteristic features, such as a network address (e.g., an IP address or MAC address) and a device type; para. [0082, 0084, 0087]);
training, with at least one processor, a graph neural network (GNN) machine learning model (i.e. graph neural networks (GNNs) may be trained to automatically detect adversarial sample attacks, improving the robustness of the GNNs … adversarial GNN defense 43, which provides adversarial training for GNN-based detection system 44; GNN-based intrusion detection 44, which uses a trained GNN to detect anomalous behavior in network gathered network information; para. [0020, 0025]) using a loss equation according to a bi-level optimization problem and based on the training dataset (i.e. Adversarial contrastive learning 304 models the graph adversarial learning as a min-max optimization, to overcome the discrete unknown adversarial space problem … The parameters of a GNN may be trained in a supervised manner, with the aim being to learn a new representation of a node, which can be used to predict the node label … the classifier may thus be trained with a mixture of original samples and adversarial samples, with an overall loss function that may be defined as: {tilde over (L)}=βL(f GNN , G, y)+(1−β)Σv*ϵV* l(f GNN , v*, 1) ; para. [0027, 0028, 0034]), wherein training the GNN machine learning model using the loss equation according to the bi-level optimization problem (i.e. The objective function can then be optimized in a min-max adversarial learning manner. This formulation includes a discriminator function S and an adversarial sample generator function G. Contrasted to a generative adversarial network (GAN), which generates adversarial samples that are close to the original samples, the present generator function G may generate adversarial samples as hard negative samples. These hard negative samples can help the system to learn a better discriminator for distinguishing the positive and negative pairs. The functions S and G are trained in a joint manner, adjusting the parameters of G to maximize log(1−S(G(x+|y−))), and adjusting the parameters of S to minimize log S(x, y); para. [0042]), loss equation according to a bi-level optimization problem is met by the disclosed adversarial min-max objective because it includes an adversarial optimization structure, comprises:
determining a solution to an inner loss problem, wherein determining the solution to the inner loss problem comprises (i.e. The integrated gradients of the prediction score can be determined for a winning class c with respect to the entries of A and X. The integrated gradients can then be used as metrics to measure the priority of perturbing specific features or edges in the graph G … adjusting the parameters of G to maximize log(1−S(G(x+|y−))); para. [0031, 0039, 0042, 0044]), using integrated gradients to determine perturbation priority of graph features or edges, generating adversarial samples, and using generator G that is trained to maximize an adversarial objective:
a first loss of the GNN machine learning model based on model parameters and a second loss of the GNN machine learning model based on the model parameters and a perturbation value (i.e. the classifier may thus be trained with a mixture of original samples and adversarial samples, with an overall loss function that may be defined as: {tilde over (L)}=βL(f GNN , G, y)+(1−β)Σv*ϵV* l(f GNN , v*, 1); para. [0034]);
determining a solution to an outer loss problem, wherein determining the solution to the outer loss problem comprises (i.e. After the effective graph adversarial samples are generated in block 402, they can be used to augment the set of anomaly examples, and to retrain a classifier of the adversarial sample detection … adjusting the parameters of S to minimize log S(x, y); para. [0034, 0042]):
the first loss of the GNN machine learning model based on the model parameters and the second loss of the GNN machine learning model based on the model parameters and the perturbation value (i.e. The objective function can then be optimized in a min-max adversarial learning manner. This formulation includes a discriminator function S and an adversarial sample generator function G. Contrasted to a generative adversarial network (GAN), which generates adversarial samples that are close to the original samples, the present generator function G may generate adversarial samples as hard negative samples. These hard negative samples can help the system to learn a better discriminator for distinguishing the positive and negative pairs. The functions S and G are trained in a joint manner, adjusting the parameters of G to maximize log(1−S(G(x+|y−))), and adjusting the parameters of S to minimize log S(x, y); para. [0035, 0042]); and
providing, with at least one processor, a trained GNN machine learning model based on training the GNN machine learning model (i.e. GNN-based detection system 44; GNN-based intrusion detection 44, which uses a trained GNN to detect anomalous behavior in network gathered network information; para. [0025, 0080]).
Chen does not explicitly teach determining a maximum value of a difference between a first loss of the machine learning model based on model parameters and a second loss of the machine learning model based on the model parameters and a perturbation value; determining model parameters that minimize the maximum value of the difference between the first loss of the machine learning model based on the model parameters and the second loss of the machine learning model based on the model parameters and the perturbation value.
However, Qin teaches determining a solution to an inner loss problem, wherein determining the solution to the inner loss problem comprises: determining a maximum value of a difference between a first loss of the NN machine learning model based on model parameters and a second loss of the NN machine learning model based on the model parameters and a perturbation value (i.e. the local linearity measure can be an absolute difference between (1) the loss function evaluated at the input-output pair that includes (i) the perturbed training input and (ii) the target output for the training input and (2) a first-order Taylor expansion of the loss function evaluated at the input-output pair … perform the inner optimization to identify an adversarial perturbation that results in the largest change in the task loss of any possible perturbation; para. [0080, 0088]); determining a solution to an outer loss problem, wherein determining the solution to the outer loss problem comprises: determining model parameters that minimize the maximum value of the difference between the first loss of the NN machine learning model based on the model parameters and the second loss of the NN machine learning model based on the model parameters and the perturbation value (i.e. the system can perform the iteration of the neural network training procedure to minimize a local linearity regularized loss function that measures at least the respective losses for the plurality of training inputs and the non-linearity for the identified maximally non-linear perturbation; para. [0063-0069).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Chen to include the feature of Qin. One would have been motivated to make this modification because it improves robustness of the NN against adversarial perturbations while maintaining computational efficiency.
Claim 2: Chen and Qin teach the computer-implemented method of claim 1. Chen does not explicitly teach training the machine learning model according to a first learning rate for the inner loss problem.
However, Qin further teaches training the NN machine learning model according to a first learning rate for the inner loss problem (i.e. the gradient descent technique can be a projected gradient descent (PGD) technique, which updates the perturbation as follows: δ←Proj(δ−s×Optimizer(gradient)), where gradient the averaged gradient, s is a step size hyperparameter, and Optimizer is an update rule that is applied to the averaged gradient, e.g., the Adam update rule or the rmsProp update rule … Thus, like some existing techniques for training neural networks to be more robust to adversarial attack, the described techniques also require an inner optimization to be performed to identify a perturbation that satisfies some criteria; para. [0084, 0087]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Chen to include the feature of Qin. One would have been motivated to make this modification because it improves robustness of the NN against adversarial perturbations while maintaining computational efficiency.
Claim 3: Chen and Qin teach the computer-implemented method of claim 1. Chen does not explicitly teach training the machine learning model according to a second learning rate for the outer loss problem
However, Qin further teaches training the GNN machine learning model according to a second learning rate for the outer loss problem (i.e. At a given iteration, the training engine 150 then determines the update to the current values of the network parameters 118 by performing an iteration of a neural network training procedure to minimize the local linearity regularized loss function, i.e., to decrease losses for the training inputs (as measured by the task loss function) and to decrease the non-linearity of the task loss function for the maximally non-linear perturbation identified by the perturbation engine 160 … The system can then determine an update to the current values of the network parameters from the averaged gradient, e.g., by applying an update rule, e.g., a learning rate, an Adam optimizer update rule, or an rmsProp update rule, to the gradient to generate an update; para. [0046, 0065]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Chen to include the feature of Qin. One would have been motivated to make this modification because it improves robustness of the NN against adversarial perturbations while maintaining computational efficiency.
Claim 6: Chen and Qin teach the computer-implemented method of claim 1. Chen further teaches comprising: modifying the training dataset (i.e. generate an adversarial training data set that includes original samples and adversarial samples, by perturbing one or more of the original samples with an integrated gradient attack to generate the adversarial samples; para. [0005]), wherein modifying the training dataset comprises: generating a perturbation value of at least one data instance of the graph data associated with the graph (i.e. For a feature or an edge with a high perturbation priority, the value may be perturbed simply by flipping it to a different binary value; para. [0031-0033]).
Claims 8 is similar in scope to Claims 1 and is rejected under a similar rationale.
Chen teaches a system, comprising: at least one processor configured to (i.e. A system for detecting and responding to an intrusion in a computer network includes a hardware processor and a memory; para. [0005]).
Claims 9, 10, 13 are similar in scope to Claims 2, 3, 6 and are rejected under a similar rationale.
Claims 15 is similar in scope to Claims 1 and is rejected under a similar rationale.
Chen teaches a computer program product comprising at least one non-transitory computer-readable medium including one or more instructions that (i.e. Embodiments may include a computer program product accessible from a computer-usable or computer-readable medium providing program code for use by or in connection with a computer or any instruction execution system; para. [0058]), when executed by at least one processor, cause the at least one processor to (i.e. A system for detecting and responding to an intrusion in a computer network includes a hardware processor and a memory; para. [0005]).
Claims 16 and 17 are similar in scope to Claims 2, 3 and are rejected under a similar rationale.
7. Claims 4, 11, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Chen in view of Qin, and further in view of Chen et al. (U.S. Patent Application Pub. No. US 20200242250 A1).
Claim 4: Chen and Qin teach the computer-implemented method of claim 1. Chen does not explicitly teach wherein determining the maximum value of the difference between the first loss of the machine learning model based on the model parameters and the second loss of the machine learning model based on the model parameters and the perturbation value comprises: determining, using stochastic gradient descent (SGD), the maximum value of the difference between the first loss of the machine learning model based on the model parameters and the second loss of the machine learning model based on the model parameters and the perturbation value.
However, Qin further teaches wherein determining the maximum value of the difference between the first loss of the NN machine learning model based on the model parameters and the second loss of the NN machine learning model based on the model parameters and the perturbation value comprises: determining, using gradient descent (GD) (i.e. the gradient descent technique can be a projected gradient descent (PGD) technique, which updates the perturbation as follows: δ←Proj(δ−s×Optimizer(gradient)), where gradient the averaged gradient, s is a step size hyperparameter, and Optimizer is an update rule that is applied to the averaged gradient, e.g., the Adam update rule or the rmsProp update rule; para. [0084]), the maximum value of the difference between the first loss of the NN machine learning model based on the model parameters and the second loss of the NN machine learning model based on the model parameters and the perturbation value (i.e. the described techniques also require an inner optimization to be performed to identify a perturbation that satisfies some criteria … perform the inner optimization to identify an adversarial perturbation that results in the largest change in the task loss of any possible perturbation; para. [0087, 0088]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Chen to include the feature of Qin. One would have been motivated to make this modification because it improves robustness of the NN against adversarial perturbations while maintaining computational efficiency.
However, Chen ’250 teaches using stochastic gradient descent (SGD) (i.e. where b=zk+1+(1/ρ)uk. In the white-box setting, since the gradients of ƒ(x0+δ, t) are directly accessible, gradient descent method like stochastic gradient descent (SGD) can be applied straight-forwardly to solve equation (15); para. [0053, 0054]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to modify the combination of Chen and Qin to include the feature of Chen ’250. One would have been motivated to make this modification because it provides a known optimization technique for solving adversarial perturbation optimization problem.
Claims 11 and 18 are similar in scope to Claim 4 and are rejected under a similar rationale.
8. Claims 5, 12, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Chen in view of Qin, and further in view of Baydin et al. (Online Learning Rate Adaptation with Hypergradient Descent, arXiv, published 2018, pages 1-11).
Claim 5: Chen and Qin teach the computer-implemented method of claim 1. Chen does not explicitly teach wherein determining the model parameters that minimize the maximum value of the difference between the first loss of the machine learning model based on the model parameters and the second loss of the machine learning model based on the model parameters and the perturbation value comprises: determining, using hypergradient descent, the model parameters that minimize the maximum value of the difference between the first loss of the machine learning model based on the model parameters and the second loss of the machine learning model based on the model parameters and the perturbation value.
However, Qin further teaches wherein determining the model parameters that minimize the maximum value of the difference between the first loss of the GNN machine learning model based on the model parameters and the second loss of the GNN machine learning model based on the model parameters and the perturbation value comprises: determining, using gradient descent, the model parameters that minimize the maximum value of the difference between the first loss of the GNN machine learning model based on the model parameters and the second loss of the GNN machine learning model based on the model parameters and the perturbation value (i.e. At a given iteration, the training engine 150 then determines the update to the current values of the network parameters 118 by performing an iteration of a neural network training procedure to minimize the local linearity regularized loss function, i.e., to decrease losses for the training inputs (as measured by the task loss function) and to decrease the non-linearity of the task loss function for the maximally non-linear perturbation identified by the perturbation engine 160 … the system can perform the iteration of the neural network training procedure to minimize a local linearity regularized loss function that measures at least the respective losses for the plurality of training inputs and the non-linearity for the identified maximally non-linear perturbation … The system can then determine an update to the current values of the network parameters from the averaged gradient, e.g., by applying an update rule, e.g., a learning rate, an Adam optimizer update rule, or an rmsProp update rule, to the gradient to generate an update; para. [0046, 0063-0066]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Chen to include the feature of Qin. One would have been motivated to make this modification because it improves robustness of the NN against adversarial perturbations while maintaining computational efficiency.
However, Baydin teaches using hypergradient descent (i.e. hypergradient descent; Section 2, pages 2-4).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to modify the combination of Chen and Qin to include the feature of Baydin. One would have been motivated to make this modification because hypergradient descent as an improvement to gradient-based optimizers by dynamically adapting the learning rate during optimization.
Claims 12 and 19 are similar in scope to Claim 5 and are rejected under a similar rationale.
9. Claims 7, 14, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Chen in view of Qin, and further in view of Foret et al. (Sharpness-Aware Minimization for Efficiently Improving Generalization, arXiv, published 2021, pages 1-20).
Claim 7: Chen and Qin teach the computer-implemented method of claim 1. Chen does not explicitly teach wherein the bi-level optimization problem comprises the following:
PNG
media_image1.png
160
338
media_image1.png
Greyscale
However, Foret teaches wherein the bi-level optimization problem comprises the following:
PNG
media_image1.png
160
338
media_image1.png
Greyscale
(i.e. rather than seeking out parameter values w that simply have low training loss value LS(w), we seek out parameter values whose entire neighborhoods have uniformly low training loss value (equivalently, neighborhoods having both low loss and low curvature), min w LSAM S (w) +λ||w||2 2 where LSAM S (w) max || ||p≤ρ LS(w + ), where ρ ≥ 0 is a hyperparameter and p ∈ [1,∞] (we have generalized slightly from an L2-norm to a p-norm in the maximization over, though we show empirically in appendix C.5 that p = 2 is typically optimal). Figure 1 shows1 the loss landscape for a model that converged to minima found by minimizing either LS(w) or LSAM S (w), illustrating that the sharpness-aware loss prevents the model from converging to a sharp minimum. In order to minimize LSAM S ∇wLSAM S (w), we derive an efficient and effective approximation to (w) by differentiating through the inner maximization, which in turn enables us to apply stochastic gradient descent directly to the SAM objective. Proceeding down this path, we first ap proximate the inner maximization problem via a first-order Taylor expansion of LS(w + ) w.r.t. around 0, obtaining ∗ (w) argmax p≤ρ LS(w + )≈argmax p≤ρ LS(w)+ T∇wLS(w) =argmax p≤ρ T∇wLS(w); Section 2 SAM, pages 3-4).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to modify the combination of Chen and Qin to include the feature of Foret. One would have been motivated to make this modification because it improves generalization by simultaneously minimizing loss value and loss sharpness.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure.
Liu et al. (Pub. No. US 20220261626 A1), the process of adversarial training can be employed to mitigate the negative impact of adversarial perturbations using a min-max robust optimization-based training method that minimizes the worst-case training loss at adversarially perturbed examples. A min-max optimization-based training method is generally able to offer significant gains in robustness.
It is noted that any citation to specific pages, columns, lines, or figures in the prior art references and any interpretation of the references should not be considered to be limiting in any way. A reference is relevant for all it contains and may be relied upon for all that it would have reasonably suggested to one having ordinary skill in the art. In re Heck, 699 F.2d 1331, 1332-33, 216 U.S.P.Q. 1038, 1039 (Fed. Cir. 1983) (quoting In re Lemelson, 397 F.2d 1006, 1009, 158 U.S.P.Q. 275, 277 (C.C.P.A. 1968)).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TAN TRAN whose telephone number is (303)297-4266. The examiner can normally be reached on Monday - Thursday - 8:00 am - 5:00 pm MT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matt Ell can be reached on 571-270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TAN H TRAN/Primary Examiner, Art Unit 2141
Prosecution Insights