Prosecution Insights
Last updated: November 18, 2026
Application No. 18/909,902

Universal Post-Training Backdoor Detection and Mitigation for Classifiers

Non-Final OA §103§112
Filed
Oct 08, 2024
Priority
Oct 12, 2022 — provisional 63/415,597 +3 more
Examiner
HONORE, EVEL NMN
Art Unit
2142
Tech Center
2100 — Computer Architecture & Software
Assignee
Anomalee Inc.
OA Round
1 (Non-Final)
52%
Grant Probability
Moderate
1-2
OA Rounds
84%
With Interview

Examiner Intelligence

Grants 52% of resolved cases
52%
Career Allowance Rate
15 granted / 29 resolved
-3.3% vs TC avg
Strong +32% interview lift
Without
With
+32.1%
Interview Lift
resolved cases with interview
Typical timeline
Avg Prosecution
20 currently pending
Career history
70
Total Applications
across all art units

Office Action

§103 §112
Notice of Pre-AIA or AIA Status The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . DETAILED ACTION This Action is responsive to the Application filed on 04/21/2022. Claims 1-20 are pending in this case. Claims 1, 11 and 19-20 are independent claim. No claims have been amended and/or canceled. Claim Rejections - 35 USC § 112 The following is a quotation of 35 U.S.C. 112(b): (b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention. The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph: The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention. The limitation “correct responses” renders the claim indefinite because the claim does not provide an objective standard for determining when a response is considered “correct”. The term “correct” is a relative term whose scope depends upon an unstated standard or criterion. Neither the claim language nor the specification provides sufficient guidance as to what constitutes a “correct response” or how correctness is determined. Claim Rejections - 35 USC § 103 The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. Claims 1 and 8-9 are rejected under 35 U.S.C. 103 as being unpatentable over “ABS: Scanning Neural Networks for Back-doors by Artificial Brain Stimulation”, https://dl.acm.org/doi/10.1145/3319535.3363216, Liu et al, 2019, hereinafter referred to as Liu, in view of Sijia et al. (US Pub No.: 20220053005 A1), hereinafter referred to as Sijia and further in view of “Approximate Feature Collisions in Neural Nets”, https://proceedings.neurips.cc/paper_files/paper/2019/file/7eea1f266bfc82028683ad15da46e05e-Paper.pdf, Li et al, 2019, hereinafter referred to as Li. With respect to claim 1, Liu disclose: A computer-implemented method for testing for overfitting bias for a trained neural network, the method comprising: receiving the trained neural network (On page 1 of the Abstract, Liu teaches a technique to scan neural network-based AI models to determine if they are trojaned. Pre-trained AI models may contain back-doors that are injected through training or by transforming inner neuron weights. On page 2 of the first bullet point, Liu teaches scanning AI models for back-doors by analyzing inner neuron behaviors through a stimulation method.) Testing for overfitting bias for the possible target response based on at least one of (i) the perturbations of the first layer, (ii) the perturbations at the second layer, and (iii) the output logits (Examiner selects: (iii) On page 11 (last paragraph), Liu evaluates the target-label output logit resulting from that simulation to determine whether a particular target response is abnormally elevated.) With respect to claim 1, Liu do not explicitly disclose: Receiving a clean dataset of input samples and their correct responses For two or more input samples of the clean dataset, perturbing their activations at a first layer of the trained neural network to both (i) achieve a common perturbation at a second layer of the trained neural network located between the first layer and the output of the trained neural network Change the output of the trained neural network to a possible target response However, it is known by Sijia to disclose: Receiving a clean dataset of input samples and their correct responses (In Fig. 4 and paragraph [0061], Sijia disclose that the detector has access to at least one clean test image per prediction class and defines Dk as the set of data belonging to prediction class k. It further says that the detector requires clean data for all K prediction labels, with one data point for each class. In Figs. 13–14 and paragraphs [0081 and 0098], Liu discloses the respective class/prediction labels associated with those clean test images.) Change the output of the trained neural network to a possible target response (In paragraphs [0042 and 0065-0066], Sijia teaches perturbing the output of a neural network to a possible target response by generating an adversarial perturbation for an input that causes the inputs original prediction label to be altered to a target label k. Furthermore, teaches that where k corresponds to a Trojan target label, the perturbation exploits a learned shortcut that causes inputs having non-k prediction labels to be predicted as the target label k. Thus, teaching changing output of the neural network to a possible target response.) Liu and Sijia are analogous pieces of art because both address post-training detection of trojan/backdoor behavior in neural networks. Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Liu, with scanning neural network based AI models to determine if they are trojaned as taught by Liu, with determining the model is a Trojan model by leveraging one or more similarities associated with the maximized neuron activations and the generated adversarial perturbations as taught by Sijia. The motivation for doing so would have been to improve post-training detection of backdoored neural networks by analyzing internal neuron behavior and identifying neurons exhibiting an abnormal bias toward a particular target output (See (Page 17, last paragraph) of Liu.) With respect to claim 1, Liu in view of Sijia do not explicitly disclose: For two or more input samples of the clean dataset, perturbing their activations at a first layer of the trained neural network to both (i) achieve a common perturbation at a second layer of the trained neural network located between the first layer and the output of the trained neural network However, it is known by Li to disclose: For two or more input samples of the clean dataset, perturbing their activations at a first layer of the trained neural network to both (i) achieve a common perturbation at a second layer of the trained neural network located between the first layer and the output of the trained neural network (On page 1 (last paragraph), Li teaches multiple different examples can share the same feature activation at an intermediate layer. The same feature activation in one layer, will have the same activation at subsequent layers and therefore the same classification decision. On page 7 (Table 6), Li teaches that two very different examples can have the same activation and therefore the same classification decision. Liu in view of Sijia and Li are analogous pieces of art because both address post-training detection of trojan/backdoor behavior in neural networks. Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Li, with multiple examples sharing the same feature activations as feature collisions as taught by Li. The motivation for doing so would have been to improve the identification and analysis of invariance and vulnerabilities by identifying different input example that produce the same or approximately the same feature activation at an intermediate layer (See (Page 7, Table 5) of Li.) Regarding claim 8, Liu, in view of Seija and Li disclose the elements of claim 1. In addition, Sijia disclose: The computer-implemented method of claim 1, wherein testing for overfitting bias further comprises determining a frequency with which a given perturbation at the first layer that causes an input sample from the clean dataset to produce the possible target response also causes other clean inputs from the clean dataset to also produce the possible target response (In paragraph [0045], Sijia discloses both a universal perturbation and a per-image perturbation for each candidate target label. Critically, the universal perturbation is one perturbation intended to operate across multiple samples, and Liu evaluates its behavior on samples associated with the candidate target label.) Regarding claim 9, Liu, in view of Seija and Li disclose the elements of claim 1. In addition, Sijia Liu: The computer-implemented method of claim 1, wherein overfitting bias indicates a backdoor data-poisoning attack (On page 2, Liu teaches a back-doors by analyzing inner neuron behaviors through a stimulation method. We formally define the stimulation analysis that precisely determines how output activations change with an inner neuron activation value.) Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Liu, in view of Sijia, Li and further in view of “GangSweep: Sweep out Neural Backdoors by GAN”, https://dl.acm.org/doi/10.1145/3394171.3413546, et al, 2020, hereinafter referred to as Zhu. Regarding claim 2, Liu, in view of Seija and Li disclose the elements of claim 1. Liu, in view of Seija and Li do not disclose: The computer-implemented method of claim 1, wherein testing for overfitting bias further comprises measuring for possible target responses at least one of: the largeness of the average perturbation at the second layer The smallness of the variance of the perturbations at the second layer The average smallness of the perturbations at the first layer The largeness of the average decision margin However, Zhu disclose the limitation: The smallness of the variance of the perturbations at the second layer (Examiner select:b On page 3 and Fig. 2, Zhu determines a shifting variance based on the difference between feature/logit representation before and after application of generated perturbations corresponding to a backdoor target exhibiting characteristically to a backdoor target exhibiting.) Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, having the teaching of Liu in view of Seija and Li to include Zhu, with revealing that the perturbation masks generated by GAN as taught by Zhu. The motivation for doing so would have been to improve neural cleanse to detect backdoor by using gradient optimization (See (Page 2: Back Door Detection) of Zhu.) Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Liu, in view of Sijia, Li, Zhu and further in view of “Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks”, https://ieeexplore.ieee.org/document/8835365, Wang et al, 2019, hereinafter referred to as Wang. Regarding claim 3, Liu, in view of Seija, Li and Zhu disclose the elements of claim 2. Liu, in view of Seija, Li and Zhu do not disclose: The computer-implemented method of claim 2, wherein testing for overfitting bias further comprises determining with statistical significance that a measurement associated with the possible target response is anomalous compared to those of one or more other possible target responses and hence that the trained neural network has overfitting bias to the anomalous target response However, Wang disclose the limitation (On page 5 (Detect Backdoor via Outlier Detection), Wang discloses determining with statistical significance that a measurement associated with a possible target response is anomalous by calculating an anomaly index using Median Absolute Deviation (MAD) and identifying a target label as an outlier when its anomaly index exceeds a threshold of 2, wherein an anomaly index greater than 2 corresponds to greater than 95% probability of the measurement being an outlier under the assumed distribution.) Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, having the teaching of Liu in view of Seija, Li and Zhu to include Wang, Identifying and Mitigating Backdoor Attacks in Neural Networks as taught by Wang. The motivation for doing so would have been to experimentally evaluate their impact on our detection and mitigation techniques, and where necessary, propose optimizations to improve performance (See (Page 1) of Wang.) Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Liu, in view of Sijia, Li, Zhu, Wang and further in view of Sokhandan (US Patent No. 12,223,422 B2 ), hereinafter referred to as Sokhandan. Regarding claim 4, Liu, in view of Seija, Li, Zhu and Wang disclose the elements of claim 3. Liu, in view of Seija, Li, Zhu and Wang do not disclose: The computer-implemented method of claim 3, wherein statistical significance for an anomalous measurement comprises two standard deviations beyond at least one of the mean or median across two or more possible target responses However, Sokhandan disclose the limitation (In Col. 15, lines 6–13, Sokhandan discloses a statistically sufficient sample of information contained in the vectors that contain the mean and the standard deviation.) Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, having the teaching of Liu in view of Seija, Li, Zhu and Wang to include Sokhandan, with each anomalous image is used to calculate vectors containing means and standard deviations for each first anomaly types as taught by Sokhandan. The motivation for doing so would have been to increase performance and accuracy a type of an anomaly in the input image of the particular object, directly from the generated image model (See (Col.7, lines 44-47) of Sokhandan.) Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Liu, in view of Sijia, Li, Zhu, and further in view of Grosse (US Pub No.: 20210303695 A1), hereinafter referred to as Grosse. Regarding claim 5, Liu, in view of Seija, Li and Zhu disclose the elements of claim 2. Liu, in view of Seija, Li and Zhu do not disclose: The computer-implemented method of claim 2, wherein testing for overfitting bias further comprises determining with statistical significance that no measurements associated with any of the possible target responses of the trained neural network are anomalous and hence deciding that the trained neural network does not have overfitting bias However, Grosse disclose the limitation (In paragraph [0042], Grosse discloses determining an over fit measure of the trained machine learning model based on the generated test results output data. The over fit measure quantifies the amount of overfitting of the trained machine learning model to a specific sub-portion of the test dataset.) Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, having the teaching of Liu in view of Seija, Li and Zhu to include Grosse, with determining an overfit measure of the trained machine learning model based on the generated test results output data as taught by Grosse. The motivation for doing so would have been to improve data processing apparatus and method and more specifically to mechanisms for measuring overfitting of machine learning computer models and their susceptibility to security threats (See [0001] of Grosse.) Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Liu, in view of Sijia, Li, and further in view of ZHAO (US Pub No.: 20260170819 A1), hereinafter referred to as ZHAO. Regarding claim 6, Liu, in view of Seija and Li disclose the elements of claim 1. Liu, in view of Seija and Li do not disclose: The computer-implemented method of claim 1, wherein: the trained neural network is a classifier The target responses are classes A classification margin is determined by output class logits However, ZHAO disclose the limitation: The computer-implemented method of claim 1, wherein: the trained neural network is a classifier (In paragraph [0063], ZHAO disclose a classifier may be implemented on top of the transformer encoder 506 without positional encoding. Thereby, the transformer encoder 506 may generate an estimate per image set element.) The target responses are classes (In paragraph [0062], ZHAO discloses a logit represents probability values from zero to one. The logit sets 508 include one logit for each image of the image set 502 to provide an estimate that is permutation-equivariant.) A classification margin is determined by output class logits (In paragraph [0067], ZHAO discloses that the logit sets 508 include one logit for each image of the image set 502 to provide an estimate that is permutation-equivariant. That is, if two of the elements in an image set (e.g., 502a) are switched in the sequence, the output would be the same as if the elements had not been switched in the sequence.) Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, having the teaching of Liu in view of Seija and Li to include ZHAO, estimating a score for each image of the image set as taught by ZHAO. The motivation for doing so would have been for means for generating, by a transformer encoder, an estimate for each image of the image set (See [0007] of ZHAO.) Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Liu, in view of Sijia, Li, and further in view of KIM (US Pub No.: 20200364574 A1), hereinafter referred to as KIM. Regarding claim 7, Liu, in view of Seija and Li disclose the elements of claim 1. Liu, in view of Seija and Li do not disclose: The computer-implemented method of claim 1, wherein the first layer is the input layer of the trained neural network However, KIM disclose the limitation (In paragraph [0117], KIM discloses raising data pairs including a first input value and a first output value of the neural network model acquired based on the first input value.) Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, having the teaching of Liu in view of Seija and Li to include KIM, with compressing a neural network model may acquire training data pairs as taught by KIM. The motivation for doing so would have been by optimizing a part of the original model 1 in the model compression process, problems of deterioration in performance compared to the original model (See [0011] of KIM.) Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Liu, in view of Sijia, Li, and further in view of DRAIN (US Pub No.: 20220066747 A1), hereinafter referred to as DRAIN. Regarding claim 10, Liu, in view of Seija and Li disclose the elements of claim 1. Liu, in view of Seija and Li do not disclose: The computer-implemented method of claim 1, wherein: the trained neural network is a transformer The target responses are token sequences The first layer is a token-embedding layer in the trained neural network that precedes a first attention layer However, DRAIN disclose the limitation: The computer-implemented method of claim 1, wherein: the trained neural network is a transformer (In paragraph [0042], DRAIN disclose a neural transformer model with attention in an encoder-decoder configuration) The target responses are token sequences (In paragraph [0042], DRAIN discloses the initial inputs to an encoder block 302 are the input embeddings 306 of an input sequence of a training dataset. In order to retain the order of the tokens in the input sequence, positional embeddings) The first layer is a token-embedding layer in the trained neural network that precedes a first attention layer (In paragraph [0080], DRAIN discloses that an embedding is a learned representation of text-based sub tokens, where sub tokens that have a common meaning have a common representation.) Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, having the teaching of Liu in view of Seija and Li to include DRAIN, with increasing the model's capacity allowing the model to learn increasing levels of abstraction as taught by DRAIN. The motivation for doing so would have been to optimize objective during fine tuning includes unit test case prediction given the focal method as an input (See [0088] of DRAIN.) Claims 11-12, 16-17 and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Sijia et al. (US Pub No.: 20220053005 A1), hereinafter referred to as Sijia, in view of Kolouri et al. (US Pub No.: 20200410098 A1), hereinafter referred to as Kolouri and further in view of “ABS: Scanning Neural Networks for Back-doors by Artificial Brain Stimulation”, https://dl.acm.org/doi/10.1145/3319535.3363216, Liu et al, 2019, hereinafter referred to as Liu. With respect to claim 11, Sijia disclose: A computer-implemented method for testing for overfitting bias for a trained neural network, the method comprising: receiving the trained neural network (In paragraph [0035], Sijia discloses that the objective of program 150 is to detect models that have been poisoned by the Trojan attack, as well as finding the target labels. In an embodiment, program 150 has access to the weights of a CNN model (either a Trojan one and/or a clean one). In paragraph [0038], Liu discloses when a user inputs a model into a program, the program monitors and/or retrieves one or more models for indications that they are poisoned.) Calculating a maximum for the detection criterion over a set of inputs to the trained neural network for two or more of the trained neural network’s output logits (In paragraph [0060], Sijia disclose the trained network as mapping input to the logits of K classes, and states that the final prediction is obtained by taking the argmax of those logits.) With respect to claim 11, Sijia do not explicitly disclose: Defining a detection criterion for possible responses of the trained neural network based on the trained neural network’s output logits Testing for overfitting bias to a corresponding output logit based on the calculated maximum detection criteria However, it is known by Kolouri to disclose: Defining a detection criterion for possible responses of the trained neural network based on the trained neural network’s output logits (In paragraph [0089], Kolouri discloses that the Universal Litmus Patterns (ULPs) are fed through the network and the corresponding outputs (i.e., logits) are linearly classified to reveal backdoor attacks. It further describes pooling the logit outputs and supplying them to a classifier that determines whether the network is poisoned or clean.) Sijia and Kolouri are analogous pieces of art because both address post-training detection of trojan/backdoor behavior in neural networks. Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Sijia, with determining the model is a Trojan model by leveraging one or more similarities associated with the maximized neuron activations and the generated adversarial perturbations as taught by Sijia, with detecting attacks in convolutional neural networks (CNNs), and more particularly, to a system for detecting backdoor attacks in CNNs as taught by Kolouri. The motivation for doing so would have been to improve detection of Trojan-poisoned neural networks by providing techniques capable of detecting Trojan behavior when only limited clean validation data is available (See [0035] of Sijia.) With respect to claim 11, Sijia in view of Kolouri do not explicitly disclose: Testing for overfitting bias to a corresponding output logit based on the calculated maximum detection criteria However, it is known by Liu to disclose: Testing for overfitting bias to a corresponding output logit based on the calculated maximum detection criteria (On page 11 (last paragraph), Liu evaluates the target-label output logit resulting from that simulation to determine whether a particular target response is abnormally elevated.) Sijia in view of Kolouri and Liu are analogous pieces of art because both address post-training detection of trojan/backdoor behavior in neural networks. Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Liu, with scanning neural network-based AI models to determine if they are trojaned as taught by Liu. The motivation for doing so would have been to improve post-training detection of backdoored neural networks by analyzing internal neuron behavior and identifying neurons exhibiting an abnormal bias toward a particular target output (See (Page 17, last paragraph) of Liu.) Regarding claim 12, Sijia, in view of Kolouri and Liu disclose the elements of claim 11. In addition, Sijia disclose: The computer-implemented method of claim 11, wherein detecting overfitting bias further involves determining with statistical significance that the maximum detection criterion associated with a specific output logit is anomalous compared to those of one or more other output logits (In paragraph [0074], Sijia discloses determining with statistical significance that a criterion associated with a particular output label is anomalous relative to criteria associated with other output label. In particular, Liu calculates a detection index for each class, determines an anomaly among the calculated indices, and employs median absolute deviation (MAD) for anomaly detection.) Regarding claim 16, Sijia, in view of Kolouri and Liu disclose the elements of claim 11. In addition, Sijia disclose: The computer-implemented method of claim 11, wherein calculating a maximum detection criterion involves gradient ascent with respect to the activations of a layer of the trained neural network (In paragraph [0075], Sijia discloses leveraging the idea of proximal gradient, calculating a perturbation needed for each label and detecting the target label if the corresponding perturbation is significantly smaller than the perturbations for other labels.) Regarding claim 17, Sijia, in view of Kolouri and Liu disclose the elements of claim 11. In addition, Liu disclose: The computer-implemented method of claim 11, wherein overfitting bias indicates a backdoor data-poisoning attack (On page 2, Liu teaches a back-doors by analyzing inner neuron behaviors through a stimulation method. We formally define the stimulation analysis that precisely determines how output activations change with an inner neuron activation value.) With respect to claim 19, Sijia disclose: A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method for testing for overfitting bias for a trained neural network, the method comprising: receiving the trained neural network (In paragraph [0035], Sijia discloses that the objective of program 150 is to detect models that have been poisoned by the Trojan attack, as well as finding the target labels. In an embodiment, program 150 has access to the weights of a CNN model (either a Trojan one and/or a clean one). In paragraph [0038], Liu discloses when a user inputs a model into a program, the program monitors and/or retrieves one or more models for indications that they are poisoned.) Calculating a maximum for the detection criterion over a set of inputs to the trained neural network for two or more of the trained neural network’s output logits (In paragraph [0060], Sijia disclose the trained network as mapping input to the logits of K classes, and states that the final prediction is obtained by taking the argmax of those logits.) With respect to claim 19, Sijia do not explicitly disclose: Defining a detection criterion for possible responses of the trained neural network based on the trained neural network’s output logits Testing for overfitting bias to a corresponding output logit based on the calculated maximum detection criteria However, it is known by Kolouri to disclose: Defining a detection criterion for possible responses of the trained neural network based on the trained neural network’s output logits (In paragraph [0089], Kolouri discloses that the Universal Litmus Patterns (ULPs) are fed through the network and the corresponding outputs (i.e., logits) are linearly classified to reveal backdoor attacks. It further describes pooling the logit outputs and supplying them to a classifier that determines whether the network is poisoned or clean.) Sijia and Kolouri are analogous pieces of art because both address post-training detection of trojan/backdoor behavior in neural networks. Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Sijia, with determining the model is a Trojan model by leveraging one or more similarities associated with the maximized neuron activations and the generated adversarial perturbations as taught by Sijia, with detecting attacks in convolutional neural networks (CNNs), and more particularly, to a system for detecting backdoor attacks in CNNs as taught by Kolouri. The motivation for doing so would have been to improve detection of Trojan-poisoned neural networks by providing techniques capable of detecting Trojan behavior when only limited clean validation data is available (See [0035] of Sijia.) With respect to claim 19, Sijia in view of Kolouri do not explicitly disclose: Testing for overfitting bias to a corresponding output logit based on the calculated maximum detection criteria However, it is known by Liu to disclose: Testing for overfitting bias to a corresponding output logit based on the calculated maximum detection criteria (On page 11 (last paragraph), Liu evaluates the target-label output logit resulting from that simulation to determine whether a particular target response is abnormally elevated.) Sijia in view of Kolouri and Liu are analogous pieces of art because both address post-training detection of trojan/backdoor behavior in neural networks. Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Liu, with scanning neural network-based AI models to determine if they are trojaned as taught by Liu. The motivation for doing so would have been to improve post-training detection of backdoored neural networks by analyzing internal neuron behavior and identifying neurons exhibiting an abnormal bias toward a particular target output (See (Page 17, last paragraph) of Liu.) With respect to claim 20, Sijia disclose: A testing system that tests for overfitting bias for a trained neural network, comprising: a processor (In paragraph [0109], Sijia disclose a processor to carry out aspects of the present invention.) A memory (In paragraph [0102], Sijia disclose memory and persistent storage.) and a detection mechanism; wherein at least one of the processor and the detection mechanism are configured to receive the trained neural network and store the program instructions associated with the trained neural network in the memory (In paragraph [0035], Sijia discloses that the objective of program 150 is to detect models that have been poisoned by the Trojan attack, as well as finding the target labels. In an embodiment, program 150 has access to the weights of a CNN model (either a Trojan one and/or a clean one). In paragraph [0038], Liu discloses when a user inputs a model into a program, the program monitors and/or retrieves one or more models for indications that they are poisoned.) Calculate a maximum for the detection criterion over a set of inputs to the trained neural network for two or more of the trained neural network’s output logits (In paragraph [0060], Sijia disclose the trained network as mapping input to the logits of K classes, and states that the final prediction is obtained by taking the argmax of those logits.) With respect to claim 20, Sijia do not explicitly disclose: Wherein the testing system is configured to: define a detection criterion for possible responses of the trained neural network based on the trained neural network’s output logits Test for overfitting bias to a corresponding output logit based on the calculated maximum detection criteria However, it is known by Kolouri to disclose: Wherein the testing system is configured to: define a detection criterion for possible responses of the trained neural network based on the trained neural network’s output logits (In paragraph [0089], Kolouri discloses that the Universal Litmus Patterns (ULPs) are fed through the network and the corresponding outputs (i.e., logits) are linearly classified to reveal backdoor attacks. It further describes pooling the logit outputs and supplying them to a classifier that determines whether the network is poisoned or clean.) Sijia and Kolouri are analogous pieces of art because both address post-training detection of trojan/backdoor behavior in neural networks. Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Sijia, with determining the model is a Trojan model by leveraging one or more similarities associated with the maximized neuron activations and the generated adversarial perturbations as taught by Sijia, with detecting attacks in convolutional neural networks (CNNs), and more particularly, to a system for detecting backdoor attacks in CNNs as taught by Kolouri. The motivation for doing so would have been to improve detection of Trojan-poisoned neural networks by providing techniques capable of detecting Trojan behavior when only limited clean validation data is available (See [0035] of Sijia.) With respect to claim 20, Sijia in view of Kolouri do not explicitly disclose: Test for overfitting bias to a corresponding output logit based on the calculated maximum detection criteria However, it is known by Liu to disclose: Test for overfitting bias to a corresponding output logit based on the calculated maximum detection criteria (On page 11 (last paragraph), Liu evaluates the target-label output logit resulting from that simulation to determine whether a particular target response is abnormally elevated.) Sijia in view of Kolouri and Liu are analogous pieces of art because both address post-training detection of trojan/backdoor behavior in neural networks. Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Liu, with scanning neural network-based AI models to determine if they are trojaned as taught by Liu. The motivation for doing so would have been to improve post-training detection of backdoored neural networks by analyzing internal neuron behavior and identifying neurons exhibiting an abnormal bias toward a particular target output (See (Page 17, last paragraph) of Liu.) Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Sijia, in view of Kolouri, Liu and further in view of Sokhandan (US Patent No. 12,223,422 B2 ), hereinafter referred to as Sokhandan. Regarding claim 13, Liu, in view of Kolouri and Liu disclose the elements of claim 12. Liu, in view of Kolouri and Liu do not disclose: The computer-implemented method of claim 12, wherein statistical significance comprises at least two standard deviations beyond at least one of the mean or median of the maximum detection criteria across at least two output logits However, Sokhandan disclose the limitation (In Col. 15, lines 6–13, Sokhandan discloses a statistically sufficient sample of information contained in the vectors that contain the mean and the standard deviation.) Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, having the teaching of Sijia in view of Kolouri, and Liu to include Sokhandan, with each anomalous image is used to calculate vectors containing means and standard deviations for each first anomaly types as taught by Sokhandan. The motivation for doing so would have been to increase performance and accuracy a type of an anomaly in the input image of the particular object, directly from the generated image model (See (Col.7, lines 44-47) of Sokhandan.) Claims 14-15 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Liu, in view of Kolouri, Sijia and further in view of ZHAO (US Pub No.: 20260170819 A1), hereinafter referred to as ZHAO. Regarding claim 14, Liu, in view of Kolouri and Sijia disclose the elements of claim 11. Liu, in view of Kolouri and Sijia do not disclose: The computer-implemented method of claim 11, wherein the trained neural network is a classifier Each output logit corresponds to a different class Each detection criterion is a class logit However, ZHAO disclose the limitation: The computer-implemented method of claim 11, wherein the trained neural network is a classifier (In paragraph [0063], ZHAO discloses that a classifier may be implemented on top of the transformer encoder 506 without positional encoding. Thereby, the transformer encoder 506 may generate an estimate per image set element.) Each output logit corresponds to a different class(In paragraph [0062], ZHAO discloses a logit represents probability values from zero to one. The logit sets 508 include one logit for each image of the image set 502 to provide an estimate that is permutation-equivariant.) Each detection criterion is a class logit (In paragraph [0067], ZHAO discloses that the logit sets 508 include one logit for each image of the image set 502 to provide an estimate that is permutation-equivariant. That is, if two of the elements in an image set (e.g., 502a) are switched in the sequence, the output would be the same as if the elements had not been switched in the sequence.) Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, having the teaching of Liu in view of Kolouri and Sijia to include ZHAO, estimating a score for each image of the image set as taught by ZHAO. The motivation for doing so would have been for means for generating, by a transformer encoder, an estimate for each image of the image set (See [0007] of ZHAO.) Regarding claim 15, Liu, in view of Kolouri and Sijia disclose the elements of claim 11. Liu, in view of Kolouri and Sijia do not disclose: The computer-implemented method of claim 11, wherein the trained neural network is a classifier Each output logit corresponds to a different class Each detection criterion is a classification margin However, ZHAO disclose the limitation: The computer-implemented method of claim 11, wherein the trained neural network is a classifier (In paragraph [0063], ZHAO discloses that a classifier may be implemented on top of the transformer encoder 506 without positional encoding. Thereby, the transformer encoder 506 may generate an estimate per image set element.) Each output logit corresponds to a different class (In paragraph [0062], ZHAO disclose a logit represents probability values from zero to one. The logit sets 508 include one logit for each image of the image set 502 to provide an estimate that is permutation-equivariant.) Each detection criterion is a classification margin (In paragraph [0067], ZHAO discloses that the logit sets 508 include one logit for each image of the image set 502 to provide an estimate that is permutation-equivariant. That is, if two of the elements in an image set (e.g., 502a) are switched in the sequence, the output would be the same as if the elements had not been switched in the sequence.) Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, having the teaching of Liu in view of Kolouri and Sijia to include ZHAO, estimating a score for each image of the image set as taught by ZHAO. The motivation for doing so would have been for means for generating, by a transformer encoder, an estimate for each image of the image set (See [0007] of ZHAO.) Regarding claim 18, Liu, in view of Kolouri and Sijia disclose the elements of claim 11. Liu, in view of Kolouri and Sijia do not disclose: The computer-implemented method of claim 11, wherein: the trained neural network is a transformer Different logits correspond to different output tokens Calculating a maximum detection criterion involves using gradient ascent with respect to the token-embedding activations of the input However, ZHAO disclose the limitation: The computer-implemented method of claim 11, wherein: the trained neural network is a transformer (In paragraph [0057], ZHAO discloses image set anomaly detection using transformer neural networks.) Different logits correspond to different output tokens (In paragraph [0062], ZHAO discloses a logit represents probability values from zero to one. The logit sets 508 include one logit for each image of the image set 502 to provide an estimate that is permutation-equivariant.) Calculating a maximum detection criterion involves using gradient ascent with respect to the token-embedding activations of the input (In paragraph [0044], ZHAO discloses that the error gradient of weights may be calculated over a few examples, so that the calculated gradient approximates the true error gradient.) Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, having the teaching of Liu in view of Kolouri and Sijia to include ZHAO, estimating a score for each image of the image set as taught by ZHAO. The motivation for doing so would have been for means for generating, by a transformer encoder, an estimate for each image of the image set (See [0007] of ZHAO.) Conclusion Any inquiry concerning this communication or earlier communications from the examiner should be directed to EVEL HONORE whose telephone number is (703)756-1179. The examiner can normally be reached Monday-Friday 8 a.m. -5:30 p.m. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Mariela D Reyes can be reached at (571) 270-1006. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. EVEL HONORE Examiner Art Unit 2142 /Mariela Reyes/Supervisory Patent Examiner, Art Unit 2142
Read full office action

Prosecution Timeline

Oct 08, 2024
Application Filed
Sep 21, 2026
Non-Final Rejection mailed — §103, §112 (current)

Precedent Cases

Applications granted by this same examiner with similar technology

Patent 12783848
METHOD FOR IMPROVING CONVOLUTIONAL NEURAL NETWORK TO PERFORM COMPUTATIONS
5y 0m to grant Granted Nov 03, 2026
Patent 12783917
SCALABLE KNOWLEDGE DATABASE GENERATION AND TRANSACTIONS PROCESSING
4y 9m to grant Granted Nov 03, 2026
Patent 12737632
METHOD AND DEVICE FOR TRAINING A MACHINE LEARNING SYSTEM
4y 11m to grant Granted Sep 15, 2026
Patent 12725029
INFORMATION PROCESSING APPARATUS, INFORMATION PROCESSING METHOD, AND STORAGE MEDIUM
5y 6m to grant Granted Sep 01, 2026
Patent 12725056
GENERATING MACHINE LEARNING BASED MODELS FOR TIME SERIES FORECASTING
4y 11m to grant Granted Sep 01, 2026
Study what changed to get past this examiner. Based on 5 most recent grants.

Strategy Recommendation AI-generated — please review before filing

Get a prosecution strategy drawn from examiner precedents, rejection analysis, and claim mapping.
Typically takes 5-10 seconds — AI-generated, attorney review required before filing

Prosecution Projections

1-2
Expected OA Rounds
52%
Grant Probability
84%
With Interview (+32.1%)
Low
PTA Risk
Based on 29 resolved cases by this examiner. Grant probability derived from career allowance rate.

Sign in with your work email

Enter your email to receive a magic link. No password needed.

Personal email addresses (Gmail, Yahoo, etc.) are not accepted.

Free tier: 3 strategy analyses per month