Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Remarks
This Office Action is responsive to Applicants' Amendment filed on 03/30/2026, in which claims 6, 7, and 13 are amended. No claims are newly cancelled or added.
Claims 1-14 are currently pending.
Response to Arguments
With regards to the rejections of claims 6, 7, and 13 under 35 U.S.C. 112(b) as being indefinite, Applicant’s arguments that the claims as amended overcome the rejections are partially persuasive. Claim 7 as amended now has antecedent basis for the term “the data protector” in new parent claim 5 and thus the 112(b) rejection of claim 7 is withdrawn. Claims 6 and 13 as amended now have antecedent basis for the terms “the data protector module” and “the activated prototypical parts” in new parent claims 5 and 8, respectively. However, the term “the explainable prediction” used within claims 6 and 13 does not have clear antecedent basis in new parent claims 5 and 8, respectively, as explained in the 112(b) rejections below. Therefore the 112(b) rejections of claims 6 and 13 are maintained.
With regards to the rejections of claims 1 and 8 under 35 U.S.C. 103 as being unpatentable over Lucia and Cotton “A Network Security Classifier Defense: Against Adversarial Machine Learning Attacks” (Lucia), in view of Amini et al. (U.S. Patent Application Publication No. 2012/0096549) (Amini), further in view of Xiao et al. “Generating Adversarial Examples with Adversarial Networks” (Xiao), Applicant argues that the combination of Lucia, Amini, and Xiao fails to teach several elements recited by claim 1, with equivalent reasoning applicable to claim 8. Examiner respectfully disagrees.
Applicant first argues on page 3 of the Remarks that “Lucia teaches a sequence of classifiers which will always be used until one classifier identifies an attack, but does not do anything additional. In contrast to that, claim 1 defines that the dynamic ensemble may be adapted based on a control function, which decides which types and sizes of ML models are deployed. A static sequence of classifiers as in Lucia does not constitute a dynamic ensemble of ML models” and “Lucia does not disclose any equivalent to the dynamic ensemble and no control function which can influence such a dynamic ensemble”. Examiner notes Lucia Pg. 5, Fig. 1 and Algorithm 1 show that during inference (use of the ensemble to classify an input outside of training), an algorithm (falling under the broadest reasonable interpretation of a function) controls the ensemble to deploy additional models to assist in classifying an input if the input is not classified earlier during inference as malicious. Lucia further discloses (Lucia Pg. 4) “Our approach is composed of a hierarchical ensemble of heterogeneous classifiers using disparate feature sets”, heterogenous and disparate both meaning “varying” and encompassing various types and sizes of models. Therefore, the approach of Lucia discloses a dynamic ensemble, which is dynamically adapted via a control function during inference to adapt what models with varying types and sizes are deployed, as claimed in claim 1. Although some elements of claim 1 are not taught by Lucia, as acknowledged by Examiner in the mapping of the limitations of claim 1, these are taught by Amini or Xiao, as described in the rejections below.
PNG
media_image1.png
285
412
media_image1.png
Greyscale
Applicant further argues on page 3 of the Remarks that “patent claim 1 states that the ML models in the ensemble are ‘trained to perform a machine learning based prediction’. The classifiers in the Lucia reference merely classify, i.e. categorize currently present data, but do not predict anything”. Examiner respectfully disagrees that Lucia does not teach ensemble models that perform machine learning predictions. One of ordinary skill in the art understands a “classification” in a machine learning context is a type of machine learning prediction, wherein one of several discrete categories for a given input is predicted, as opposed to regression, which is a type of machine learning prediction wherein a numerical value from a continuous range is predicted for a given input. Prediction is a general term for all outputs of a machine learning model, including classifications. See Brownlee “Difference Between Classification and Regression in Machine Learning”, which is included to provide a factual basis for definitions for common machine learning terminology but is not formally applied to any rejections.
Applicant further argues on pages 3 and 4 of the Remarks that “Even if the static sequence of classifiers H1, H2, ... in Lucia were to be construed to be an equivalent to the attack detector of claim 1, it does not generate anything that would decide which ML models further downstream are to be deployed during an inference stage of the claimed system. Lucia merely teaches a binary detection if a network is under attack or not. In contrast to that, the attack detector according to claim 1 allows for deploying different members of the ensemble of ML models depending on the alertness score, which may indicate that the underlying system may be under attack. Therefore, the disclosed system is not merely configured to detect if it is under attack, but also to continue its intended operation in a manner that adapts to the current threat”. Examiner respectfully disagrees that Lucia “does not generate anything that would decide which ML models further downstream are to be deployed during an inference stage”. Lucia Pg. 5, Fig. 1 and Algorithm 1 show that if a classification of “malicious” is generated by classifier H1 when given input x1, no additional models are deployed. If a classification of “malicious” is not generated by classifier H1 when given input x1, classifier H2 is deployed and given input x2. Examiner acknowledges that Lucia does not teach an alertness score as claimed, however this is taught by Amini, as described in the rejections below.
Applicant further argues on page 4 of the Remarks that neither Amini nor Xiao teach features of a dynamic ensemble dynamically adapted by a control function as claimed by claim 1. Examiner acknowledges this, however as Examiner asserts above, these claim elements are adequately taught by Lucia. Amini and Xiao are relied upon to teach other claim elements, as described in the rejections below.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 6 and 13 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 6,
Claim 6 recites the term the explainable prediction. It is unclear what this term refers to, as no “explainable prediction” is referred to previously within claim 6, or within parent claims 1 or 5. Claim 5 does recite the similar terminology “interpretable neural network models to learn prototypes for explaining class prediction”, however while the recited “interpretable neural network models” and “prototypes” are said to have the purpose of explaining class prediction, it is not clear whether the interpretable models or the prototypes themselves are explainable predictions, or if they are used to explain predictions that are implicitly explainable, but not explicitly recited as such. Therefore the term the explainable prediction lacks antecedent basis.
For examination purposes, the claim will be interpreted as reading “The system of claim 12, wherein the data protector module is further configured to: identify an anomaly in latent space geometry, and send a visualization of an explainable prediction to a user interface to guide additional training localized to the activated prototypical parts”.
Regarding claim 13,
Claim 13 recites a method for performing the function of the system of claim 6. All other limitations in claims 13 are substantially the same as those in claim 6, therefore claim 13 is considered indefinite with an equivalent rationale and is interpreted for examination purposes in the same way.
Prior Art
The following references are used for prior art claim rejections:
Lucia and Cotton “A Network Security Classifier Defense: Against Adversarial Machine Learning Attacks”
Amini et al. (U.S. Patent Application Publication No. 2012/0096549)
Xiao et al. “Generating Adversarial Examples with Adversarial Networks”
Preuveneers et al. “Resource Usage and Performance Trade-offs for Machine Learning Models in Smart Environment”
Sun et al. “AI-Enhanced Offloading in Edge Computing: When Machine Learning Meets Industrial IoT”
Echauz et al. (U.S. Patent No. 11,551,137)
Chen et al. “This Looks Like That: Deep Learning for Interpretable Image Recognition”
Chou et al. “SentiNet: Detecting Localized Universal Attacks Against Deep Learning Systems”
Ma et al. “Explaining Vulnerabilities to Adversarial Machine Learning through Visual Analytics”
Eskin et al. “A Geometric Framework For Unsupervised Anomaly Detection: Detecting Intrusions in Unlabeled Data”
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1 and 8 are rejected under 35 U.S.C. 103 as being unpatentable over Lucia and Cotton “A Network Security Classifier Defense: Against Adversarial Machine Learning Attacks”, hereinafter Lucia, in view of Amini et al. (U.S. Patent Application Publication No. 2012/0096549), hereinafter Amini, further in view of Xiao et al. “Generating Adversarial Examples with Adversarial Networks”, hereinafter Xiao.
Regarding claim 1,
Lucia teaches:
an attack detector comprising one or more [deep neural networks] ((Lucia Pg. 4) “We propose a novel method to defend a network security classifier using machine learning against AML attacks. Our approach is composed of a hierarchical ensemble of heterogeneous classifiers using disparate feature sets”, Lucia does not explicitly teach deep neural networks) trained using adversarial examples… ((Lucia Pg. 3) “Adversarial training employs AML to optimize the adversarial objective to create modified examples via perturbation of attack examples to cause the classifier to change its prediction from the original sample”)
and a dynamic ensemble of individually robust machine learning (ML) models ((Lucia Pg. 4) “In our proposed AML defense, each successive classifier must be resilient to an adversarial attack against the proceeding classifier in the ensemble”) of various types and sizes ((Lucia Pg. 4) “Our approach is composed of a hierarchical ensemble of heterogeneous classifiers using disparate feature sets”, heterogenous classifiers on disparate feature sets correspond to classifiers of various types and sizes) and all being trained to perform a machine learning based prediction, ((Lucia Pg. 4) “an ensemble of classifiers trained using loosely correlated disparate feature sets is more effective than using a single classifier containing all features”, (Lucia Pg. 6) “The objective of the existing network security classifier is to detect the presence of network scanning activity in a network”, a classifier produces classifications, which are a type of machine learning prediction)
wherein a control function dynamically adapts which types and sizes of ML models are deployed for the dynamic ensemble during the inference stage of operation, (Lucia Pg. 5, Fig. 1 and Algorithm 1 show that their algorithm performs inference dynamically, only using the next model in the ensemble if the subsequent model does not detect an input as malicious, an algorithm to control the models used in an ensemble during inference is a control function)
PNG
media_image1.png
285
412
media_image1.png
Greyscale
wherein the control function is responsive to [the alertness score] received from the attack detector (Lucia Pg. 5, Fig. 1 and Algorithm 1 show that their algorithm, which corresponds to a control function, responds to the classification of “malicious” or not “malicious” produced by the attack detector, however Lucia does not teach a score to measure alertness)
Amini teaches A system for robust machine learning, comprising: ((Amini [0006]) “FIG. 1 illustrates a block diagram of a system for adaptive security analytics in accordance with an embodiment”) as well as the following further limitations that Lucia does not explicitly teach:
a processor; ((Amini [0045]) “These computer program instructions may be provided to a processor”)
and a non-transitory memory having stored thereon modules executed by the processor, the modules comprising: ((Amini [0040]-[0041]) “aspects of the present invention may take the form of a computer program product embodied in one or more computer readable medium(s)… More specific examples (a nonexhaustive list) of the computer readable storage medium would include the following:…a random access memory (RAM)”)
the attack detector configured to produce an alertness score based on a likelihood of an input being adversarial; ((Amini [0005]) “A score responsive to the network activity and to a scoring model is computed at a computer. The score indicates a likelihood of a security violation”)
wherein the control function is responsive to the alertness score received from the attack detector ((Amini [0005]) “The score is validated and the scoring model is automatically updated responsive to results of the validating”, validation that causes an update based on a score from a network security model corresponds to a control function corresponds to a control function responsive to an alertness score)
At the time of filing, one of ordinary skill in the art would have motivation to combine Lucia and Amini by taking the system for detecting attacks using an adversarial trained machine learning model ensemble, including a control function to control the machine learning model ensemble, taught by Lucia, and including use of a score quantifying the likelihood of an input being adversarial which the control function is responsive to, taught by Amini, as use of a numerical score output of a machine learning model is well-known within the art, and allows the output to be compared to a threshold in order to automatically make a decision, increasing efficiency via automating the task of doing so. Such a combination would be obvious.
Xiao teaches the following further limitations that neither Lucia nor Amini teach:
an attack detector comprising one or more deep neural networks ((Xiao Pg. 5) “We use three adversarial training defenses to train different models for each model architecture:…ensemble adversarial training (Ens.)”, Xiao Pg. 5, Table 3 shows for the CIFAR10 dataset, ensemble defense with ResNet models is used, ResNet is a deep neural network)
PNG
media_image2.png
357
410
media_image2.png
Greyscale
… adversarial examples generated from multiple models including a generative adversarial network (GAN), ((Xiao Abstract) “In this paper, we propose AdvGAN to generate adversarial examples with generative adversarial networks (GANs), which can learn and approximate the distribution of original instances. For AdvGAN, once the generator is trained, it can generate perturbations efficiently for any instance, so as to potentially accelerate adversarial training as defense”)
At the time of filing, one of ordinary skill in the art would have motivation to combine Lucia, Amini, and Xiao by taking the system for detecting attacks using an adversarial trained machine learning model ensemble, including a control function to control the machine learning model ensemble and responsive to a model produced score, taught jointly by Lucia and Amini, and including use of deep neural networks in the ensemble and use of a generative adversarial network to produce adversarial examples for training, taught by Xiao, taught by Xiao, as Xiao teaches: (Xiao Pg. 1) “Deep Neural Networks (DNNs) have achieved great successes in a variety of applications” and (Xiao Pgs. 1-2) “we train a conditional adversarial network to directly produce adversarial examples, which not only results in perceptually realistic examples that achieve state-of-the-art attack success rate against different target models, but also the generation process is more efficient”. Such a combination would be obvious.
Regarding claim 8,
Lucia teaches A computer implemented method for robust machine learning, comprising: ((Lucia Pg. 4) “In our proposed AML defense, each successive classifier must be resilient to an adversarial attack against the proceeding classifier in the ensemble”)
training an attack detector configured as one or more [deep neural networks] ((Lucia Pg. 4) “We propose a novel method to defend a network security classifier using machine learning against AML attacks. Our approach is composed of a hierarchical ensemble of heterogeneous classifiers using disparate feature sets”, Lucia does not explicitly teach deep neural networks) trained using adversarial examples … ((Lucia Pg. 3) “Adversarial training employs AML to optimize the adversarial objective to create modified examples via perturbation of attack examples to cause the classifier to change its prediction from the original sample”)
training a plurality of machine learning (ML) models ((Lucia Pg. 4) “an ensemble of classifiers trained using loosely correlated disparate feature sets is more effective than using a single classifier containing all features”), of various types and sizes ((Lucia Pg. 4) “Our approach is composed of a hierarchical ensemble of heterogeneous classifiers using disparate feature sets”, heterogenous classifiers on disparate feature sets correspond to classifiers of various types and sizes) to perform a ML-based prediction task for given inputs; ((Lucia Pg. 6) “The objective of the existing network security classifier is to detect the presence of network scanning activity in a network”, a classifier produces classifications, which are a type of machine learning prediction)
monitoring, by the trained attack detector, inputs intended for a dynamic ensemble of a subset of the plurality of ML models during an inference stage of operation; (Lucia Pg. 5, Fig. 1 and Algorithm 1 show that their algorithm for attack detection performs inference using a subset of a dynamic ensemble on inputs x1 and x2, only using the next model in the ensemble if the subsequent model does not detect an input as malicious)
and dynamically adapting, by a control function, which types and sizes of ML models are deployed for the dynamic ensemble during the inference stage of operation, responsive to [the alertness score] (Lucia Pg. 5, Fig. 1 and Algorithm 1 show that their algorithm performs inference dynamically, only using the next model in the ensemble if the subsequent model does not detect an input as malicious, which is a response, and an algorithm to control the models used in an ensemble during inference is a control function, however Lucia does not teach a score to measure alertness)
Amini teaches the following further limitations that Lucia does not explicitly teach:
producing an alertness score for each input based on a likelihood of the input being adversarial; ((Amini [0005]) “A score responsive to the network activity and to a scoring model is computed at a computer. The score indicates a likelihood of a security violation”)
dynamically adapting, by a control function, [which types and sizes of ML models are deployed for the dynamic ensemble during the inference stage of operation,] responsive to the alertness score ((Amini [0005]) “The score is validated and the scoring model is automatically updated responsive to results of the validating”, validation that causes an update based on a score from a network security model corresponds to a control function corresponds to a control function responsive to an alertness score, Lucia but not Amini does not teach a dynamic ensemble of various types and sizes of ML models)
At the time of filing, one of ordinary skill in the art would have motivation to combine Lucia and Amini by taking the method for detecting attacks using an adversarial trained machine learning model ensemble, including a control function to control the machine learning model ensemble, taught by Lucia, and including use of a score quantifying the likelihood of an input being adversarial which the control function is responsive to, taught by Amini, as use of a numerical score output of a machine learning model is well-known within the art, and allows the output to be compared to a threshold in order to automatically make a decision, increasing efficiency via automating the task of doing so. Such a combination would be obvious.
Xiao teaches the following further limitations that neither Lucia nor Amini teach:
training an attack detector configured as one or more deep neural networks ((Xiao Pg. 5) “We use three adversarial training defenses to train different models for each model architecture:..ensemble adversarial training (Ens.)”, Xiao Pg. 5, Table 3 shows for the CIFAR10 dataset, ensemble defense with ResNet models is used, ResNet is a deep neural network)
… adversarial examples generated from multiple models including a generative adversarial network (GAN), ((Xiao Abstract) “In this paper, we propose AdvGAN to generate adversarial examples with generative adversarial networks (GANs), which can learn and approximate the distribution of original instances. For AdvGAN, once the generator is trained, it can generate perturbations efficiently for any instance, so as to potentially accelerate adversarial training as defense”)
At the time of filing, one of ordinary skill in the art would have motivation to combine Lucia, Amini, and Xiao by taking the method for detecting attacks using an adversarial trained machine learning model ensemble, including a control function to control the machine learning model ensemble responsive to a model produced score, taught jointly by Lucia and Amini, and including use of deep neural networks in the ensemble and use of a generative adversarial network to produce adversarial examples for training, taught by Xiao, taught by Xiao, as Xiao teaches: (Xiao Pg. 1) “Deep Neural Networks (DNNs) have achieved great successes in a variety of applications” and (Xiao Pgs. 1-2) “we train a conditional adversarial network to directly produce adversarial examples, which not only results in perceptually realistic examples that achieve state-of-the-art attack success rate against different target models, but also the generation process is more efficient”. Such a combination would be obvious.
Claims 2 and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Lucia, in view of Amini, further in view of Xiao, further in view of Preuveneers et al. “Resource Usage and Performance Trade-offs for Machine Learning Models in Smart Environment”, hereinafter Preuveneers.
Regarding claim 2,
Lucia, Amini, and Xiao jointly teach The system of claim 1,
Preuveneers teaches the following further limitation that neither Lucia, nor Amini, nor Xiao teaches:
wherein the control function selects the type and size of ML model further based on parameters including one of available system memory and maximum time to compute the prediction according to a level of urgency for the prediction ((Preuveneers Pg. 8) “It follows then that a systematic analysis should vary one constraint at a time, and then tune the models with different parameters and configurations, e.g., fixing a memory budget which is a commonplace constraint in some deployment environments”, (Preuveneers Pgs. 8-9) “MONAS and DPP-Net [36], on the other hand, are natural extensions that search and optimize for multiple device-agnostic and device-aware constraints, resulting in gradually better models for all optimization objectives. The outcome of this process are tuples of objective performances where we can select the ones that are Pareto-optimal”, MONAS and DPP-Net are control functions that can select hyperparameters for an ML model, including available memory)
At the time of filing, one of ordinary skill in the art would have motivation to combine Lucia, Amini, Xiao, and Preuveneers by taking the system for detecting attacks of claim 1, including a control function, taught jointly by Lucia, Amini, and Xiao, and including having the control function select machine learning models based on available system memory, taught by Preuveneers, as Preuveneers teaches: (Preuveneers Pg. 8) “fixing a memory budget…is a commonplace constraint in some deployment environments”. Such a combination would be obvious.
Regarding claim 9,
Claim 9 recites a method for performing the function of the system of claim 2. All other limitations in claim 9 are substantially the same as those in claim 2, therefore the same rationale for rejection applies.
Claims 3 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Lucia, in view of Amini, further in view of Xiao, further in view of Sun et al. “AI-Enhanced Offloading in Edge Computing: When Machine Learning Meets Industrial IoT”, hereinafter Sun.
Regarding claim 3,
Lucia, Amini, and Xiao jointly teach The system of claim 1,
Sun teaches the following further limitation that neither Lucia, nor Amini, nor Xiao teaches:
wherein the [trained attack detector] reacts to rapidity of inputs during an inference stage of operation by adjusting the [alertness] score to require less robustness and leaner ML models for more rapid response ((Sun Pg. 4) “computing tasks of MD i, we first estimate the latency if offloading to the available edge servers or through some paths to the remote cloud, respectively, and drop those computing tasks whose delay requirements cannot be satisfied (i.e., the access delay is greater than di)”, (Sun Pg. 6) “in this case the edge servers with high accuracy get crowded, and the other MDs will turn to the edge servers with less accuracy”, (Sun Pg. 2) “Deploying AI-based computing service at the edge servers could effectively extend the computing capability of MDs and enable MDs’ access to machine intelligent service with low latency”, computing on edge servers with less accurate AI models when delay of offloading through a network path is too high corresponds to reaction to rapid inputs by using less robust ML models with more rapid responses, Lucia teaches a trained attack detector, Amini teaches an alertness score)
At the time of filing, one of ordinary skill in the art would have motivation to combine Lucia, Amini, Xiao, and Sun by taking the system for detecting attacks of claim 1, taught jointly by Lucia, Amini, and Xiao, and including using an adjustment to use less robust models capable of faster response when a network is congested with input traffic, taught by Sun, as Sun teaches: (Sun Pg. 6) “AI-enabled edge servers could effectively process the delay-intolerant traffic at the edge of networks”. Such a combination would be obvious.
Regarding claim 10,
Claim 10 recites a method for performing the function of the system of claim 3. All other limitations in claim 10 are substantially the same as those in claim 3, therefore the same rationale for rejection applies.
Claims 4 and 11 are rejected under 35 U.S.C. 103 as being unpatentable over Lucia, in view of Amini, further in view of Xiao, further in view of Echauz et al. (U.S. Patent No. 11,551,137), hereinafter Echauz.
Regarding claim 4,
Lucia, Amini, and Xiao jointly teach The system of claim 1,
Echauz teaches the following further limitation that neither Lucia, nor Amini, nor Xiao teaches:
wherein the attack detector reacts to a high likelihood of input being adversarial by adjusting the [alertness score] to require more robustness ((Echauz Abstract) “detecting, by the classification monitor, a campaign of adversarial classification decision outputs in the machine learning model; applying a transformation function to the machine learning model in the model environment to transform the adversarial classification decision outputs to thwart the campaign of adversarial classification decision outputs”, a model being transformed to thwart an adversarial campaign is more robust, Amini teaches an alertness score)
At the time of filing, one of ordinary skill in the art would have motivation to combine Lucia, Amini, Xiao, and Echauz by taking the system for detecting attacks of claim 1, including an alertness score, taught jointly by Lucia, Amini, and Xiao, and including making a model more robust in response to a high likelihood of an adversarial input, taught by Echauz, as it is well-known in the art that models that have been made robust against adversarial inputs usually do so at the cost of reduced accuracy on non-adversarial inputs, and thus increasing the robustness only when adversarial inputs are likely provides the predictable benefit of providing protection against adversarial inputs without reducing accuracy when adversarial inputs are unlikely. Such a combination would be obvious.
Regarding claim 11,
Claim 11 recites a method for performing the function of the system of claim 4. Claim 11 also specifically states that the robustness recited in claim 4 should be robustness in the dynamic ensemble. Lucia at Pg. 5, Fig. 1 and Algorithm 1 teaches a dynamic ensemble. All other limitations in claim 11 are substantially the same as those in claim 4, therefore the same rationale for rejection applies.
Claims 5 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Lucia, in view of Amini, further in view of Xiao, further in view of Chen et al. “This Looks Like That: Deep Learning for Interpretable Image Recognition”, hereinafter Chen, further in view of Chou et al. “SentiNet: Detecting Localized Universal Attacks Against Deep Learning Systems”, hereinafter Chou.
Regarding claim 5,
Lucia, Amini, and Xiao jointly teach The system of claim 1, the modules further comprising:
Chen teaches the following further limitations that neither Lucia, nor Amini, nor Xiao teach:
a data protector module comprising interpretable neural network models configured to: learn prototypes for explaining class prediction; ((Chen Pg. 2) “we introduce a network architecture – prototypical part network (ProtoPNet), that accommodates this definition of interpretability, where comparison of image parts to learned prototypes is integral to the way our network reasons about new examples”)
form class predictions of initial training data relying on geometry of latent space, ((Chen Pg. 5) “In the first training stage, we aim to learn a meaningful latent space, where the most important patches for classifying images are clustered (in L2-distance) around semantically similar prototypes of the images’ true classes, and the clusters that are centered at prototypes from different classes are well-separated”)
wherein the class predictions determine how a test input is similar to prototypical parts of inputs from each class, ((Chen Pg. 7) “Figure 3 shows the reasoning process of our ProtoPNet in reaching a classification decision on a test image of a red-bellied woodpecker at the top of the figure. Given this test image x, our model compares its latent features f(x) against the learned prototypes. In particular, for each class k, our network tries to find evidence for x to be of class k by comparing its latent patch representations with every learned prototype pj of class k”)
At the time of filing, one of ordinary skill in the art would have motivation to combine Lucia, Amini, Xiao, and Chen by taking the system for detecting attacks of claim 1, taught jointly by Lucia, Amini, and Xiao, and including an interpretable neural network architecture with prototypes that explain class predictions via prototypical parts and via the geometry of a latent space, taught by Chen, as Chen teaches: (Chen Pg. 2) “our ProtoPNet provides a level of interpretability that is absent in other interpretable deep models”, and increased interpretability provides for the predictable benefit of allowing for a human monitoring the attack detector to better understand the nature of attacks and the model’s response to them. Such a combination would be obvious.
Chou teaches the following further limitation that neither Lucia, nor Amini, nor Xiao, nor Chen teaches:
and detect potential data poisoning or backdoor triggers in the initial training data ((Chou Pg. 2) “We evaluate SentiNet to protect pre-existing compromised and uncompromised networks against three known attack vectors, i.e., backdoors for poisoned networks, triggers for trojaned networks”) on a condition that [prototypical] parts from unrelated classes are activated ((Chou Pg. 3) “The idea is to identify the parts of the input x that contribute to the model prediction y”, (Chou Pg. 3) Once the class proposal C is obtained, the second step of SentiNet consists of identifying the regions of x that most highly influence the predictions C”, (Chou Pg. 4) “we use these additional heatmaps to generate secondary masks to improve the original mask for the prediction y by subtracting common regions. This results in a set of masks that highlight only the localized attack”, Chen teaches prototypes)
At the time of filing, one of ordinary skill in the art would have motivation to combine Lucia, Amini, Xiao, Chen, and Chou by taking the system for detecting attacks of claim 1, including an interpretable neural network architecture that learns prototypes to predict classification, taught jointly by Lucia, Amini, Xiao, and Chen, and further including detecting adversarial attacks such as data poisoning and backdoor triggers when regions from unrelated classes are found to be salient, taught by Chou, as Chou teaches: (Chou Pg. 3) “Because the attack is small and localized, we can hope to recover the true class of input x if we evaluate the model on a segmented input that contains no part of the attack” and (Chou Pg. 13) “Our method is notable because it only relies on the malicious behavior of an adversarial attack to perform classifications, without requiring prior knowledge of the attack vector”, that is, that detecting adversarial inputs via specific, localized parts allows not only for the original class of an adversarially modified input to be recovered, it also can be done for all means of adversarial attack. Such a combination would be obvious.
Regarding claim 12,
Claim 12 recites a method for performing the function of the system of claim 5. All other limitations in claim 12 are substantially the same as those in claim 5, therefore the same rationale for rejection applies.
Claims 6 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Lucia, in view of Amini, further in view of Xiao, further in view of Chen, further in view of Chou, further in view of Ma et al. “Explaining Vulnerabilities to Adversarial Machine Learning through Visual Analytics”, hereinafter Ma.
Claims 6 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Lucia, in view of Amini, further in view of Xiao, further in view of Chen, further in view of Chou, further in view of Ma et al. “Explaining Vulnerabilities to Adversarial Machine Learning through Visual Analytics”, hereinafter Ma.
Regarding claim 6,
Lucia, Amini, Xiao, Chen, and Chou jointly teach The system of claim 5, wherein the data protector module is further configured to:
Ma teaches the following further limitations that neither Lucia, nor Amini, nor Xiao, nor Chen, nor Chou teach:
identify an anomaly in latent space geometry, (Ma Pg. 7, Fig. 8 shows in the 4th element that there is an anomaly in the projection view, which has geometry)
PNG
media_image3.png
397
903
media_image3.png
Greyscale
and send a visualization of the explainable prediction to a user interface ((Ma Pg. 2) “A multi-faceted visualization scheme summarizes the attack results from the perspective of the machine learning model and its corresponding training dataset, and coordinated views are designed to help users quickly identify model vulnerabilities and explore potential attack vectors”) to guide additional training ((Ma Pg. 8) “For defenders who want to alleviate the sparsity issue and improve the security of the victim model, possible solutions could be to add more validated labeled samples into the original training dataset”) localized ((Ma Pg. 2) “For an in-depth analysis of specific data instances affected by the attack, a locality-based visualization is designed to reveal neighborhood structure changes due to an adversarial attack”) to the activated prototypical parts ((Ma Pg. 6) “Feature View: The feature view is designed to reflect the relationship between class features and prediction outputs to help users understand the effects of data poisoning (T3.2, D2.3)”, identified activated features when investigating data poisoning correspond to activated prototypical parts)
At the time of filing, one of ordinary skill in the art would have motivation to combine Lucia, Amini, Xiao, Chen, Chou, and Ma by taking the system for detecting attacks of claim 5, including a data protector and prototypical parts, taught jointly by Lucia, Amini, Xiao, Chen, and Chou, and including making a visualization of a prediction for a user interface in order to guide additional training, taught by Ma, as Ma teaches: (Ma Pg. 9) “our framework enables users to examine potential weak points in the training dataset and explore the impacts of poisoning attacks on model performance…This can enable domain scientists to design more reliable machine learning models and data processing pipelines”. Such a combination would be obvious.
Regarding claim 13,
Claim 13 recites a method for performing the function of the system of claim 6. All other limitations in claim 13 are substantially the same as those in claim 6, therefore the same rationale for rejection applies.
Claims 7 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Lucia, in view of Amini, further in view of Xiao, further in view of Chen, further in view of Chou, further in view of Eskin et al. “A Geometric Framework For Unsupervised Anomaly Detection: Detecting Intrusions in Unlabeled Data”, hereinafter Eskin.
Regarding claim 7,
Lucia, Amini, Xiao, Chen, and Chou jointly teach The system of claim 5, wherein the data protector is further configured to:
Eskin teaches the following further limitation that neither Lucia, nor Amini, nor Xiao, nor Chen, nor Chou teaches:
employ latent space embedding ((Eskin Pg. 4) “we present a geometric framework for unsupervised anomaly detection. Our frameworks maps the data to a feature space”, mapping to a feature space correspond to latent space embedding) of training data ((Eskin Pg. 4) “unsupervised anomaly detection has many advantages over supervised anomaly detection. The main advantage is that they do not require a purely normal training set”) where distances correspond to an amount of change in perception or meaning within a current context ((Eskin Pg. 5) “in general, our algorithms will detect anomalies because they will tend to be distant from other points”, distance being an amount of anomalousness corresponds to distances being an amount of change in meaning within a context)
At the time of filing, one of ordinary skill in the art would have motivation to combine Lucia, Amini, Xiao, Chen, Chou, and Eskin by taking the system for detecting attacks of claim 5, taught jointly by Lucia, Amini, Xiao, Chen, and Chou, and including embedding training data into a latent space with distance that corresponds to change in meaning, taught by Eskin, as Eskin teaches: (Eskin Pg. 5) “A major advantage of our framework is its flexibility. We can define our mappings to feature spaces that better capture intrusions as outliers in the feature space”, that is, that using distance in a latent space is advantageous for detecting outliers, a well-known and relatively simple to implement technique within the art for anomaly detection, including attack detection. Such a combination would be obvious.
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Lucia, in view of Amini, further in view of Xiao, further in view of Eskin.
Regarding claim 14,
Lucia, Amini, and Xiao jointly teach The method of claim 8, further comprising:
Eskin teaches the following further limitation that neither Lucia, nor Amini, nor Xiao teaches:
employing latent space embedding ((Eskin Pg. 4) “we present a geometric framework for unsupervised anomaly detection. Our frameworks maps the data to a feature space”, mapping to a feature space correspond to latent space embedding) of training data ((Eskin Pg. 4) “unsupervised anomaly detection has many advantages over supervised anomaly detection. The main advantage is that they do not require a purely normal training set”) where distances correspond to an amount of change in perception or meaning within a current context ((Eskin Pg. 5) “in general, our algorithms will detect anomalies because they will tend to be distant from other points”, distance being an amount of anomalousness corresponds to distances being an amount of change in meaning within a context)
At the time of filing, one of ordinary skill in the art would have motivation to combine Lucia, Amini, Xiao, and Eskin by taking the method for detecting attacks of claim 1, taught jointly by Lucia, Amini, and Xiao, and including embedding training data into a latent space with distance that corresponds to change in meaning, taught by Eskin, as Eskin teaches: (Eskin Pg. 5) “A major advantage of our framework is its flexibility. We can define our mappings to feature spaces that better capture intrusions as outliers in the feature space”, that is, that using distance in a latent space is advantageous for detecting outliers, a well-known and relatively simple to implement technique within the art for anomaly detection, including attack detection. Such a combination would be obvious.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Sen et al. “EMPIR: Ensembles of Mixed Precision Deep Networks for Increased Robustness against Adversarial Attacks” discloses a method of creating an ensemble of machine learning models with different precisions to increase robustness against adversarial inputs.
Maimo et al. “A Self-Adaptive Deep Learning-Based System for Anomaly Detection in 5G Networks” discloses a method of anomaly detection that can self-adapt based on changing memory requirements and network flows.
Di Pietro et al. (U.S. Patent Application Publication No. 2019/0370218) discloses a method of monitoring a machine learning model and replacing the machine learning model in response to changes in circumstances.
Srivastava et al. (U.S. Patent Application Publication No. 2022/0171848) discloses a method and device for an adaptive ensemble to defend against adversarial attacks using dynamic selection of diverse weak defenses (WDs).
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to VICTOR A NAULT whose telephone number is (703) 756-5745. The examiner can normally be reached M - F, 12 - 8.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Miranda Huang can be reached at (571) 270-7092. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/V.A.N./Examiner, Art Unit 2124
/Kevin W Figueroa/ Primary Examiner, Art Unit 2124
Prosecution Insights