Detailed Action
This action is in response to the RCE filed on 03/30/2026 for the amended claims filed 03/27/2026 for application 17/331,570, in which:
Claims 1 and 8 are independent claims.
Claims 1 and 8 are amended.
Claim 9 is canceled.
Claims 1-8 are currently pending.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 03/30/2026 has been entered.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 03/27/2026 have been fully considered but they are not persuasive.
Regarding the 35 USC § 103 Rejections:
Applicant's arguments regarding the 35 U.S.C. 103 rejections of the previous office action have been fully considered, but are unpersuasive.
Applicant disagrees with the 103 rejections (Pages 5-6) and the newly cited Song prior art in view of the newly amended claims. Applicant supports their disagreements with asserting that Zhang fails to disclose the features of independent claim, where each image is associated with supplying, to said neural network … a result label associated with an object of said adversarial image and indicating a desired response associated with this adversarial image; and a target label associated with the object of said adversarial image, as a same object, and indicating an incorrect specific result that the neural network should not return when it receives said adversarial image as input. Thus, the Zhang NPL fails to disclose or suggest the claimed limitations and the limitations of claim 1 are novel thereover.
Examiner respectfully disagrees. Applicant’s arguments with respect to the amended independent claim(s) have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Applicant asserts (Page 6), that the Song prior art relates to a method for attacking a machine learning model but does not disclose a method for training a neural network. Applicant further supports their assertions by noting that Song also fails to disclose at least one iteration of an adversarial learning step. Song focuses on attacking deep neural networks and uses trained models but does not train the pre-trained models.
Examiner respectfully disagrees. Song does teach a method for attacking a model and also training as Table 2 shows the performance after retraining as noted by Song on Page 1245, Column 2, Paragraph 1, “After retraining, the final testing performance regarding five commonly used measures [20] is reported in Table II”. However, Song does not need to teach training of the neural network explicitly as this is covered within Zhang. In terms of the adversarial learning step, Song does not need to disclose as it is covered by Zhang.
Applicant asserts (Pages 6-8), that the Song prior art also fails to disclose the limitations each image is associated with a result label associated with an object of said adversarial image and indicating a desired response associated with this adversarial image; and a target label associated with the object of said adversarial image, as a same object, and indicating an incorrect specific result that the neural network should not return when it receives said adversarial image as input. Even if Song NPL discloses several labels, said labels are each associated with a respective object. Within the Song prior art, x is a feature vector of ℝd, which means that x includes "d" different objects. Each object of an instance x is either associated with the label -1 or the label 1. Song does not teach that either one of "1" indicates a desired response, or that ''-1" indicates an incorrect response that a neural network should not return. Table 1 is also erroneous within the Song prior art. Table 1 only specifies that there are two label groups; where Song notes that each object of the feature vector x is associated with a single respective label, whereas claim 1 requires that each object of a given image is associated with two different labels: one that is correct (and that is indicated as such) and one that is incorrect (and also that is indicated as such). Thus, Applicant respectfully notes wherein Song NPL fails to disclose or suggest the claimed limitations and the limitations of claim 1 are novel thereover.
Examiner respectfully disagrees. The examiner does agree that each instance of an image within Song is associated with either -1 or 1 which indicates incorrect or ground truth labels; where the image is associated with an object. An object of an image, is interpreted by the examiner, as any structure within an image. The positive and negative label indicate the object of said adversarial image; as the object of the image is either correctly labeled as the ground truth or incorrectly labeled (ex. from Song: Person or Sheep). Thus, Song does teach the labeling of the object within the image to be correctly or incorrectly classified by the model.
Applicant asserts (Pages 8-9), as noted above, for example the aforementioned distinguishing features are not disclosed in Song NPL, and cannot be considered as being part of the general knowledge of the person skilled in the art. Even if the person skilled in the art applied to Zhang NPL the teachings of Song NPL, it would only result in a method for generating adversarial samples (according to Song NPL), in order to attack the model obtained using the mixup technique (described in Zhang NPL). As the combination of cited references results in a device that does not contain all limitations of Applicant's independent claim… the similar independent claim and the dependent claims should also be reconsidered due to relevancy and dependency. respectively.
Examiner respectfully disagrees. As noted above, all limitations are now covered by Zhang/Song/Chen and within the office action below. Applicant’s arguments regarding the other independent and dependent claims rely upon the same assertions as with respect to Claim 1, and are thus likewise unpersuasive. Therefore, for the reasons given above and in the rejections below, the rejection to all Claims (including Claim 1, similar independent claims, and all dependent Claims) are maintained. More specific details are discussed below within the 35 USC § 103 Rejections.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-6, and 8 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al., “mixup: BEYOND EMPIRICAL RISK MINIMIZATION” (hereinafter referred to as Zhang), in view of Song et al., “Multi-Label Adversarial Perturbations” (hereinafter referred to as Song).
Regarding Claim 1:
Zhang teaches:
A supervised adversarial learning method for a neural network, comprising at least one iteration of an adversarial learning step, said supervised adversarial learning method comprising:
(Zhang, [Abstract], Page 1, “Large deep neural networks are powerful, but exhibit undesirable behaviors such as memorization and sensitivity to adversarial examples. In this work, we propose mixup, a simple learning principle to alleviate these issues. In essence, mixup trains a neural network on convex combinations of pairs of examples and their labels”; Page 2, Paragraph 4, “In supervised learning …” where training on “pairs of examples and their labels” denotes supervised learning; thus, mixup is a supervised adversarial learning method which comprises iterations of learning steps to learn from pairs of examples and their labels via training);
supplying, to said neural network,
(Zhang, Page 4, Fig. 2; Page 7, Table 3. Figure 2 shows the behaviors of two neural networks supplied with the CIFAR-10 dataset containing images. Table 3 shows the use of FGSM and I-FGSM to generate adversarial examples supplied to the two neural networks for testing the learning methods on adversarial attacks).
an adversarial image comprising an adversarial attack,
(Zhang, Page 2, Paragraph 1, “ … x̄ = λxi + (1- λ) xj, where xi, xj are raw input vectors … and λ ∈ [0, 1]”; Page 7, Table 3; Page 7, Paragraph 2, “… we use the model itself to generate adversarial examples, either using the Fast Gradient Sign Method (FGSM) or the Iterative FGSM (I-FGSM) methods … we test the robustness of the second ERM model and the mixup model to these examples …”. xi, xj are training examples/data (which are images (noted from Zhang, [Experiments], Page 4, Paragraph 1 (lines 1-2))) when λ ≠ [0 or 1] then the image is considered adversarial. mixup is a supervised learning method that is utilized for training and tested against adversarial images (generated by FGSM) that contain labels of the original images).
the adversarial attack being provided to orient said neural network towards a specific target result different from an expected result, the specific target result corresponding to a specific response towards which the adversarial attack is intended to orient the neural network;
(Zhang, Page 4, Fig. 2(a), “… a prediction is counted as a “miss” if it does not belong to {yi, yj}”; Page 2, Paragraph 4, “… we first define a loss function l that penalizes the differences between predictions f(x) and actual targets y, for examples (x, y) ~ P … ” Page 7, Table 3. Table 3 shows the generated adversarial test images that are used for the neural networks within the experiment to test the learning methods; where the adversarial attacks on the images are provided to orient the neural networks towards a specific target. Raw input vectors with modifications/adversarial attacks are sent to the neural network to orient the neural network to a result that is different than the expected result (y) and penalized via the loss function. A prediction is considered a “miss” when the training data is sent to the neural network model containing adversarial attacks (when λ ≠ [0 or 1]) and it does not belong to {yi, yj}. f(x) denotes the prediction from the neural network and y being the actual target (result item). The loss function is comparing the prediction based off of the adversarial image to the actual result item);
…
Zhang teaches utilizing mixup for data augmentation which blends images by combining them together (thus, handling the adversarial examples/corrupt labels). Thus, the training is based on adversarial data as they are included within the training set (as noted within Zhang, Figure 2(a): “… The model trained with mixup has fewer misses”). Even though Zhang teaches supplying a blended image and blended label containing a result and target label… Zhang does not explicitly disclose the separated labels associated with the adversarial images:
a result label associated with an object of said adversarial image and indicating a desired response associated with this adversarial image;
a target label associated with an object of said adversarial image indicating an incorrect specific result that the neural network should not return when it receives said adversarial image as input.
However, Song teaches:
a result label associated with an object of said adversarial image and indicating a desired response associated with this adversarial image;
(Song, Page 1242, Column 2, Paragraph 4, “Notations: … multi-label classification problem with l labels. Let … y ∈ {−1, 1}l×1 denote … the label vector of an instance … multi-label classification algorithms provide two types of outputs, either binary values that assign concrete relationships between instances and labels or confidence scores indicating the relevance of each instance with labels”; TABLE 1. Each image is associated with a positive label (1); thus, the ground truth label which is a desired response associated with the adversarial image) and a negative label (-1). An object of an image, is interpreted by the examiner, as any structure within an image. The positive and negative label indicate the object of said adversarial image; as the object of the image is either correctly labeled as the ground truth or incorrectly labeled (ex. from Song: Person or Sheep)).
a target label associated with the object of said adversarial image, as a same object , indicating an incorrect specific result that the neural network should not return when it receives said adversarial image as input.
(Song, Page 1242, Abstract, “Adversarial examples are delicately perturbed inputs, which aim to mislead machine learning models towards incorrect outputs”; Page 1242, Column 2, Paragraph 4, “… multi-label classification problem with l labels. Let … y ∈ {−1, 1}l×1 denote … the label vector of an instance … multi-label classification algorithms provide two types of outputs, either binary values that assign concrete relationships between instances and labels or confidence scores indicating the relevance of each instance with labels”; TABLE 1. Each image is associated with a positive label (1) and a negative label (-1); thus, the incorrect label which is an incorrect specific result that the neural network should not return as the adversarial example is aiming to mislead the neural network towards incorrect outputs. An object of an image, is interpreted by the examiner, as any structure within an image. The positive and negative label indicate the object of said adversarial image; as the object of the image is either correctly labeled as the ground truth or incorrectly labeled (ex. from Song: Person or Sheep)).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to utilize Zhang’s methodology of supervised adversarial learning where the adversarial images are supplied to a neural network to use the specific explicit multi-labeling methodology of Song. One having ordinary skill in the art would have been motivated to implement this change before the effective filing date of the claimed invention, as this leads to adversarial robustness in terms of multi-class labeling/classification, quantifying attack performance, handling uncertain labels, generating multi-label perturbations (Song, Page 1242, Column 2, Paragraph 2, “Generating targeted multi-label adversarial perturbations is still a rarely touched and challenging task. First, each instance might have an uncertain number of labels. It requires the attacking methods to be sensitive and discriminative in generating perturbations based on different targets. Second, coordinating multiple labels is difficult since labels are not mutually exclusive with each other. It is too arbitrary to target on a certain single label without considering the rests. Third, quantifying the attacking performance can be hard since multiple targeted labels may not be jointly achieved. These challenges prevent exiting attacking methods from being simply applied to generate multi-label adversarial perturbations”).
Regarding Claim 2:
Zhang and Song teach the method of Claim 1 (and thus the rejection of Claim 1 is incorporated). Song further teaches:
wherein for said adversarial image, the result label and the target label are stored together in a same adversarial label, supplied to said neural network during the adversarial learning step utilizing said adversarial image.
(Song, Page 1242, Column 2, Paragraph 4, “… multi-label classification problem with l labels. Let … y ∈ {−1, 1}l×1 denote … the label vector of an instance … multi-label classification algorithms provide two types of outputs, either binary values that assign concrete relationships between instances and labels or confidence scores indicating the relevance of each instance with labels”; TABLE 1. The label vector contains the binary values (notating if the associated label is a positive or negative example) within a vector set of length l x 1 (vector space); where the element in the label vector set stores both positive and negative labels together where each element is associated with an element in the label set. Thus, the result label and the target label are stored together in a same adversarial label (which is the label vector; ex: label vector [-1, 1, -1, -1, 1] -> [sheep, person, dog, cat, human] … which would denote the labels for the image [not sheep, person, not dog, not cat, human])).
Regarding Claim 3:
Zhang and Song teach the method of Claim 1 (and thus the rejection of Claim 1 is incorporated). Zhang further teaches:
further comprising at least one learning step that supplies as input of the neural network a non-adversarial image, wherein said non-adversarial image does not contain an adversarial attack
(Zhang, Page 4, Fig. 2. When λ = [0 or 1] then the training data is non-adversarial. Training images are adversarial as xi and xj are normal training images which are then combined/blended with FGSM generation methods).
Regarding Claim 4:
Zhang and Song teach the method of Claim 1 (and thus the rejection of Claim 1 is incorporated). Zhang and Song further teach:
further comprising utilizing a set of images
(Zhang, [Experiments], Page 4, Paragraph 1, “We evaluate mixup on the ImageNet-2012 classification dataset… This dataset contains 1.3 million training images and 50,000 validation images, from a total of 1,000 classes”).
comprising: said adversarial image comprising at least one adversarial image; for said at least one adversarial image storing said result label in association with said at least one adversarial image; for said at least one adversarial image storing said target label in association with said at least one adversarial image
(Zhang, Page 3, Paragraph 1, “x̄ = λxi + (1- λ) xj, ȳ = λyi + (1- λ) yj, where (xi, yi) and (xj, yj) are two feature-target vectors… The mixup hyper-parameter α controls the strength of interpolation between feature-target pairs” and [Experiments], Page 9, Paragraph 1, “mixup is a data augmentation method that consists of only two parts: random convex combination of raw inputs, and correspondingly, convex combination of one-hot label encodings. However, there are several design choices to make… we either use a convex combination of the two one-hot encodings as the target, or select the one-hot encoding of the closer training sample as the target”; Page 7, 2, “To assess the robustness of mixup models to adversarial examples, we use three ResNet-101 models: two of them trained using ERM on ImageNet-2012, and the third trained using mixup ... summarized in Table 3”. Song’s label vectors are storing the data together in one labeled item (vector). Zhang’s Table 3 shows adversarial examples being experimented on within the ImageNet-2012 dataset. Zhang teaches multiple experiments and combinations of training/testing to review robustness of the models with adversarial/non-adversarial data).
Regarding Claim 5:
Zhang and Song teach the method of Claim 4 (and thus the rejection of Claim 4 is incorporated). Zhang further teaches:
wherein the set of images further comprises: only adversarial images
(Zhang, [From Empirical Risk Minimization to mixup] Page 4, Fig. 2, and [Experiments], Page 7, Table 3. Fig. 2 depicts adversarial images when λ ≠ [0 or 1]. Table 3 teaches only testing on adversarial examples from the ImageNet-2012 dataset).
Regarding Claim 6:
Zhang and Song teach the method of Claim 4 (and thus the rejection of Claim 4 is incorporated). Zhang further teaches:
wherein the set of images further comprise: at least one non-adversarial image, wherein said at least one non-adversarial image does not contain an adversarial attack; and for said at least one non-adversarial image, a result label stored in association with said at least one non-adversarial image, and provided to indicate to said neural network the expected result for said at least one non-adversarial image
(Zhang, Page 3, Paragraph 1, “x̄ = λxi + (1- λ) xj, ȳ = λyi + (1- λ) yj, where (xi, yi) and (xj, yj) are two feature-target vectors… The mixup hyper-parameter α controls the strength of interpolation between feature-target pairs” and [Experiments], Page 9, Paragraph 1, “mixup is a data augmentation method that consists of only two parts: random convex combination of raw inputs, and correspondingly, convex combination of one-hot label encodings. However, there are several design choices to make… we either use a convex combination of the two one-hot encodings as the target, or select the one-hot encoding of the closer training sample as the target.” Song’s label vectors are storing the data together in one labeled item (vector). The feature-target vectors of Zhang are storing the data together in one labeled item as a convex combination and can contain both adversarial and non-adversarial images depending on λ. Zhang teaches multiple experiments and combinations of training and testing to review robustness of the models with adversarial/non-adversarial data).
Regarding Claim 8:
Claims 8 incorporate substantively all the limitations of Claim 1 in a non-transitory computer program product comprising instructions, which said instructions are executed by an electronic and/or computerized appliance (Zhang, Page 2, Paragraph 2, “mixup can be implemented in a few lines of code, and introduces minimal computation overhead … The source-code necessary to replicate our CIFAR-10 experiments is available at: https://github.com/facebookresearch/mixup-cifar10”. The machine learning application that the non-adversarial image classification tasks can be replicated with utilizing the source code from GitHub which implies a processor, memory, and non-transitory computer program product as they are inherent within a device/apparatus that utilizes source code for machine learning computations) and further recites no new limitations; thus, Claim 8 is rejected for reasons set forth in the rejection of Claim 1.
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al., “mixup: BEYOND EMPIRICAL RISK MINIMIZATION” (hereinafter referred to as Zhang), in view of Song et al., “Multi-Label Adversarial Perturbations” (hereinafter referred to as Song), in view of Xie et al., “Self-training with Noisy Student improves ImageNet classification” (hereinafter referred to as Xie).
Regarding Claim 7:
Zhang and Song teach the method of Claim 4 (and thus the rejection of Claim 4 is incorporated). Zhang further teaches:
wherein the set of images further comprises: …
(Zhang, [Experiments], Page 4, Paragraph 1, “We evaluate mixup on the ImageNet-2012 classification dataset… This dataset contains 1.3 million training images and 50,000 validation images, from a total of 1,000 classes”).
Zhang and Song fail to disclose explicitly:
… two adversarial images that contain a same adversarial attack, or that contain two different adversarial attacks.
However, Xie teaches:
… two adversarial images that contain a same adversarial attack, or that contain two different adversarial attacks
(Xie, Page 6, Fig. 3. Fig. 3 teaches images chosen from the ImageNet dataset which have been perturbed with artificial transformations for adversarial attacks. Fig. 3(c) shows three different adversarial images of the same image via artificial transformations (rotational scaling for racing car and size scaling for plate rack).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to utilize the perturbed images taught by Xie with the set of images for training in Zhang/Song/Chen. One having ordinary skill in the art would have been motivated to implement this change before the effective filing date of the claimed invention, as this leads to adversarial robustness and helps test the neural network model (Xie, [Experiments, Page 6, Paragraph 4, “Figure 3c shows images from ImageNet-P and the corresponding predictions. As can be seen, our model with Noisy Student Training makes correct and consistent predictions as images undergone different perturbations while the model without Noisy Student Training flips predictions frequently”).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to IBRAHIM RAHMAN whose telephone number is (703)756-1646. The examiner can normally be reached M-F 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kakali Chaki can be reached at (571) 272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/I.R./ Examiner, Art Unit 2122
/KAKALI CHAKI/ Supervisory Patent Examiner, Art Unit 2122
Prosecution Insights