DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Claims 1 and 11 were amended.
Claims 1-20 are pending and examined herein.
Claims 4, 5, 10, 14, 15, and 20 are rejected under 35 U.S.C. 112(b).
Claims 1-20 are rejected under 35 U.S.C. 103.
Response to Arguments
Applicant’s arguments, see pages 8 and 9, filed 03/04/2026, with respect to the rejection(s) of claim(s) 1-20 under 35 U.S.C. 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Khoreva (US 2023/0031755 A1), Wang (US 2022/0415019 A1), Wang-NPL (With Great Training Comes Great Vulnerability: Practical Attacks against Transfer Learning, August 2018), and Pervin (“Adversarial Attack Driven Data Augmentation for Accurate and Robust Medical Image Segmentation”, May 2021).
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 4, 5, 10, 14, 15, and 20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 4 and 14 state "performing an image segmentation process on an image received from the autonomous vehicle". It is unclear whether the image segmentation process in claim 4/14 is the same as the image segmentation process in claim 1/11 or if the image segmentation process is performed again. For purposes of examination, the limitation in claim 4/14 will be treated as if referring to the step in claim 1/11.
Dependent claims 5 and 15 fail to resolve the issue and are rejected with the same rationale.
Claims 10/20 state "a loss function that is combined with cross-entropy and a dissimilarity function." It is unclear whether the loss function in claim 10/20 is the same as the loss function in claim 1/11 or if it is an additional loss function. For purposes of examination, the limitation in claim 10/20 will be treated as if referring to the loss function in claim 1/11.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 4-11, and 14-20 are rejected under 35 U.S.C. 103 as being unpatentable over Khoreva (US 2023/0031755 A1), Wang (US 2022/0415019 A1), Wang-NPL (With Great Training Comes Great Vulnerability: Practical Attacks against Transfer Learning, August 2018), and Pervin (“Adversarial Attack Driven Data Augmentation for Accurate and Robust Medical Image Segmentation”, May 2021).
Regarding claim 1, Khoreva teaches
A method, comprising: ([0059] states "In another aspect, the present invention concerns a computer-implemented method for classifying an image and a corresponding label map, wherein the image and the corresponding label map are classified by the discriminator of a generative adversarial network according to an embodiment of the first aspect of this invention.")
deploying a discriminator … in an autonomous vehicle, wherein the discriminator is trained to recognize an adversarial image received by the discriminator as adversarial ([0019] states “As is common for GANs, the discriminator seeks to classify whether its input (i.e., the provided image and the provided label map) has been generated by the generator or not. In common GAN [Generative Adversarial Network] terminology, the class referring to images and label maps generated by the generator may also be called ‘fake class’ [adversarial] while the other class may be called ‘true class’ or ‘real class’.” Furthermore, [0088] states that “If it [the GAN including a discriminator] classifies the input signal (x) and output signal (o) as "fake data", this indicates that the data obtained from the sensor (30) may be critical, e.g., data which the classifier (60) was not trained for and hence a good classification result cannot be expected for or intentionally malicious data such as adversarial examples, and/or the classification as determined by the classifier (60) are inaccurate or false.” Additionally, [0074] – [0081] describe a series of steps in which the GAN is trained. Specifically, in relation to the discriminator, [0077] states “In a third step (103), the discriminator (72) is then trained to classify the first image (711) and the first label map (712) into the fake class” and [0080] states “In a sixth step (106) the discriminator (72) is trained to classify the second image and second label map into a real class.” Figure 3 shows the discriminator, located in the GAN (see [0019], “the GAN further comprises a discriminator), in a control system, which is deployed in an autonomous vehicle in Figure 4.)
receiving an original image captured by an autonomous vehicle, wherein the original image is classified as a first classification; (According to [0093], “FIG. 4 shows an embodiment in which the control system (40) is used to control an at least partially autonomous robot, e.g., an at least partially autonomous vehicle (200).” Therefore, the control system (40) depicted in Figure 3 may be understood as being in an autonomous vehicle. Additionally, the control system receives an image. Specifically, [0095] states “The image classifier (60) may be configured to detect objects in the vicinity of the at least partially autonomous robot based on the input image (x).” [0067] states "The GAN further comprises a discriminator (72), which is configured to accept a provided image (711) and a provided label map (712) and determine an output (
y
) characterizing a classification (
y
1
,
y
2
,
y
n
,
y
l
) of the provided image and the provided label map." [0069] states "In particular, the provided label map (712) may characterize a tensor of one-hot encodings of the classes of the pixels." Therefore, the image is classified as a first classification. [0049] states that “As training is conducted as a zero-sum game, improving the classification accuracy of the discriminator directly improves the accuracy of images and label maps generated by the generator.” As the training data, interpreted as the original image, is given to the discriminator, as stated in [0079], the generator is trained to create images based upon an original image. [0088] states "The generative adversarial network (70) assesses whether the input signal (x) and output signal (o) characterize "real data", i.e., data that was used for training the classifier (60). For this purpose, the generative adversarial network (70) has been trained with the same data as the classifier (60)." Therefore, as the input signals are compared to the training data to determine if it was “real”, the training data must be also from an autonomous vehicle.
[generating an adversarial image from the original image,] which causes an image segmentation model to misclassify the adversarial image ([0024] states “An advantage of the proposed GAN of the present invention is that the specific design of the generator and the discriminator allows for generating images that look like other images from a training dataset the GAN has been trained with [the original image] while also being able to generate highly accurate label maps, i.e., annotations of the generated image.” [0088] states "The input signal (x) and the output signal (o) are also provided to the generative adversarial network (70). The generative adversarial network (70) assesses whether the input signal (x) and output signal (o) characterize "real data", i.e., data that was used for training the classifier (60). For this purpose, the generative adversarial network (70) has been trained with the same data as the classifier (60). To put it in other words, the generative adversarial network (70) knows how the input signals (x) and output signals (o) should look. If it classifies the input signal (x) and output signal (o) as "fake data", this indicates that the data obtained from the sensor (30) may be critical, e.g., data which the classifier (60) was not trained for and hence a good classification result cannot be expected for or intentionally malicious data such as adversarial examples, and/or the classification as determined by the classifier (60) are inaccurate or false." Therefore, the adversarial image causes the classifier to misclassify the adversarial image. [0085] states "The input signal (x) is then passed on to a classifier (60), which is configured for semantic segmentation or object detection." Therefore, the classifier is an image segmentation model.)
generating a first training dataset based on the original image and a second training dataset based on the adversarial image; ([0079] states "In a fifth step (105), the discriminator (72) is provided a second image and a corresponding second label from a training dataset." This is the original image, and a training dataset has been generated, as it is used. [0075] states "In a first step (101), the generator (71) is provided a vector of randomly drawn values (R) as input and determines an output characterizing a first image (711) and a first label map (712)." This is the adversarial image. [0077] states "In a third step (103), the discriminator (72) is then trained to classify the first image (711) and the first label map (712) into the fake class." Therefore, as it is used to train, this is interpreted as the second training dataset, which has to have been generated as it is used.)
receiving, by the discriminator, an image from the first training dataset or the second training dataset and training the discriminator to distinguish original images from adversarial images; and ([0080] states "In a sixth step (106) the discriminator (72) is trained to classify the second image and second label map into a real class." Therefore, the discriminator has received an image from the first training dataset. [0075] states "In a first step (101), the generator (71) is provided a vector of randomly drawn values (R) as input and determines an output characterizing a first image (711) and a first label map (712)." [0077] states "In a third step (103), the discriminator (72) is then trained to classify the first image (711) and the first label map (712) into the fake class." Therefore, the discriminator has received an image from the second training dataset.)
receiving, during operation of the autonomous vehicle, a captured image; ([0082] states "At preferably evenly spaced points in time, a sensor (30) senses a condition of the actuator system. The sensor (30) may comprise several sensors. The sensor (30) is an optical sensor that takes images of the environment (20)." Fig. 4 shows that the sensor is in an autonomous vehicle. [0095] states "The image classifier (60) may be configured to detect objects in the vicinity of the at least partially autonomous robot based on the input image (x). The output signal (y) may comprise an information, which characterizes where objects are located in the vicinity of the at least partially autonomous robot. The control signal (A) may then be determined in accordance with this information, for example to avoid collisions with the detected objects." Therefore, the captured image is received by the control system during operation of the autonomous vehicle.)
passing the captured image to the discriminator… ([0088] states "The generative adversarial network classifies the input signal (x) and the output signal (o) by providing it to its discriminator (72), preprocessing the output signal ( o) if it is not a semantic segmentation map in one-hot encoding.")
performing an image segmentation process on the captured image ([0087] states "The classifier (60) determines an output signal (o) from the input signals (x), wherein the output signal ( o) characterizes a semantic segmentation or an object detection of the input signal (x).")
Khoreva does not appear to explicitly teach
[a discriminator] as an inference pre-processing module
adding to the original image a perturbation, … to generate an adversarial image, which is classified as a second classification different from the first classification, wherein the perturbation is made generated by solving an optimization problem that:
(i) minimizes a loss function comprising a cross-entropy loss between a prediction of the image segmentation model for the original image and a target segmentation corresponding to the second classification and a structural dissimilarity (DSSIM) term between the adversarial image and the original image; and
(ii) updates the original image in a direction based on a gradient of the loss of the original image and scaled up by a perturbance degree constant,
wherein the perturbation is further generated subject to a structural dissimilarity constraint such that the DSSIM term between the adversarial image and the original image is less than or equal to a predefined threshold;
[performing inference on data] only when the discriminator determines that the captured image is not adversarial;
However, Wang—directed to analogous art—teaches
as an inference pre-processing module (Fig. 6 shows the system, where the discriminative model is applied in S608, before the inference occurs in S611.)
[passing the data to the discriminator] before performing [inference] (Fig. 6 shows the system, where the image are passed to the discriminative model in S608, before the inference occurs in S611.)
[performing inference on data] only when the discriminator determines that the captured image is not adversarial; ([0099]-[0101] state "In S608 of the detection phase S602, the discriminative model of the ACGAN is applied, namely, each image 63 taken by camera 21 will be detected by the discriminator of ACGAN. In S609, if the probability (outputted from the ACGAN) for the image 63 being a real image is lower than a threshold, it is considered as anomalous sample and it should be abandoned in S610. For the normal sample, if the probability for the image 63 being head class is larger than tail, it will be classified as head class, otherwise, it will be classified as tail class in S611. The classification being head or tail will be outputted an actuator 62 (such as a robot arm) in the product line.")
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine the teachings of Khoreva with the teachings of Wang because, as stated by Wang in [0069] "The detection time will be reduced, and there is no need to collect anomalous samples, which are much harder to get than the normal samples. The efficiency may be also improved."
The combination of Khoreva and Wang does not appear to explicitly teach
adding to the original image a perturbation, … to generate an adversarial image, which is classified as a second classification different from the first classification, wherein the perturbation is made generated by solving an optimization problem that:
(i) minimizes a loss function comprising a cross-entropy loss between a prediction of the image segmentation model for the original image and a target segmentation corresponding to the second classification and a structural dissimilarity (DSSIM) term between the adversarial image and the original image; and
(ii) updates the original image in a direction based on a gradient of the loss of the original image and scaled up by a perturbance degree constant,
wherein the perturbation is further generated subject to a structural dissimilarity constraint such that the DSSIM term between the adversarial image and the original image is less than or equal to a predefined threshold;
However, Wang-NPL—directed to analogous art—teaches
adding to the original image a perturbation, … to generate an adversarial image, which is classified as a second classification different from the first classification, wherein the perturbation is made generated by solving an optimization problem that: (Page 1283 states "Attacker perturbs the source image so it could be misclassified as the same class of a specific target image. Using the Teacher model, attacker computes perturbations that mimic the internal representation of the target image at layer K." Page 1284 states "The following optimization problem is solved to craft an adversarial sample
x
'
s
.”)
min
D
T
K
x
s
'
,
T
K
x
t
(i) minimizes a loss function comprising … a structural dissimilarity (DSSIM) term between the adversarial image and the original image; and (Page 1284 states "The above optimization tries to minimize dissimilarity
D
(
⋅
)
between the two internal representations, under a constraint to limit perturbation within a budget
P
." Note that the internal representations are of the target (adversarial) image and the original (source) image.)
wherein the perturbation is further generated subject to a structural dissimilarity constraint such that the DSSIM term between the adversarial image and the original image is less than or equal to a predefined threshold; (Page 1284 states "The above optimization tries to minimize dissimilarity
D
(
⋅
)
between the two internal representations, under a constraint to limit perturbation within a budget
P
." As the constraint is within a budget (equivalent to a less than threshold), the less than or equal threshold is one increment of the smallest order of decimal smaller than P. )
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine the teachings of Khoreva and Wang with the teachings of Wang-NPL because, as Wang-NPL states on page 1284, "While a helpful way to quantify perturbation, Lp distance fails to capture what humans perceive as image distortion. Therefore, we use another metric, called DSSIM, which is an objective image quality assessment metric that closely matches with the perceived quality of an image (i.e. subjective assessment) [65, 66]. The key idea is that humans are sensitive to structural changes in an image, which strongly correlates with their subjective evaluation of image quality. To infer structural changes, DSSIM captures patterns in pixel intensities, especially among neighboring pixels. The metric also captures luminance, and contrast measures of an image, that would also impact perceived image quality."
The combination of Khoreva, Wang, and Wang-NPL does not appear to explicitly teach
[minimizes a loss function comprising] a cross-entropy loss between a prediction of the image segmentation model for the original image and a target segmentation corresponding to the second classification and
(ii) updates the original image in a direction based on a gradient of the loss of the original image and scaled up by a perturbance degree constant,
However, Pervin—directed to analogous art—teaches
[minimizes a loss function comprising] a cross-entropy loss between a prediction of the image segmentation model for the original image and a target segmentation corresponding to the second classification and (Page 3 states "Inverse FGSM (InvFGSM) adopts an inverse approach of adjusting input to a minimum loss using gradient which definitely helps to gain better performance," Eq. 2 shows that a loss function is computed between x and y, of which page 3 states "Suppose x&y are the input images and corresponding labels respectively." Page 4 states "Loss Function = Categorical Cross-Entropy". The model used is, stated on page 3, "The U-Net model, a semantic segmentation model, was developed by Olaf Ronnerberger [12]." Therefore, the labels of x are a prediction of the image segmentation for the original image, and the labels of y are the target segmentation with a different classification (different labels.) Page 3 states "Inverse FGSM (InvFGSM) adopts an inverse approach of adjusting input to a minimum loss using gradient which definitely helps to gain better performance." )
(ii) updates the original image in a direction based on a gradient of the loss of the original image and scaled up by a perturbance degree constant, (Page 3 states "As the sign of the derivatives provides adversarial noises which are added with input to generate adversarial examples, the inverse of the sign provides new type of adversarial noise that will work positively aiding in loss minimization." The sign of the derivative is interpreted as the direction based on the gradient of the loss, as shown in Eq. 2. Eq. 2 shows that the gradient is multiplied by the perturbance degree constant
ϵ
.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine the teachings of Khoreva, Wang, and Wang-NPL with the teachings of Pervin because, as stated by Pervin on page 3, "Inverse FGSM (InvFGSM) adopts an inverse approach of adjusting input to a minimum loss using gradient which definitely helps to gain better performance." Additionally, page 3 states "Data augmentation with these kind of adversarial examples with positive noises seem to be useful in enhancing model performance."
Regarding claim 4, the rejection of claim 1 is incorporated herein. Further, Khoreva teaches
performing an image segmentation process on an image received from the autonomous vehicle, and the image segmentation process results in creation of image segments. ([0093] states that “FIG. 4 shows an embodiment in which the control system (40) is used to control an at least partially autonomous robot, e.g., an at least partially autonomous vehicle.” The classifier (60) is a part of the control system (40) and in this embodiment, according to [0094] “The input signal (x) may hence be understood as an input image and the classifier as an image classifier.” [0085] states “The input signal (x) is then passed on to a classifier (60), which is configured for semantic segmentation or object detection.” Semantic segmentation results in the creation of image segments.)
Khoreva does not appear to explicitly teach
wherein when the image received from the autonomous vehicle is determined not to be adversarial,
However, Wang—directed to analogous art—teaches
wherein when the image received from the autonomous vehicle is determined not to be adversarial, performing [a classification process] on the image ([0068] – [0069] state "As shown in FIG. 1, the method 100 may comprise: S101, receiving an image to be classified; S102, inputting the image to a discriminator of a first Generative Adversarial Network, GAN; and S103, outputting a result indicating real and an index of a predetermined classification, or a result indicating fake." When the discriminator decides that the result is fake (interpreted as adversarial), it does not classify the sample.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of this application to combine the teachings of Khoreva and Wang for the reasons given above in regards to claim 1.
Regarding claim 5, the rejection of claim 4, and therefore claim 1 is incorporated herein. Further, Khoreva teaches
wherein the autonomous vehicle uses the image segments to navigate. ([0096] states, in relation to the at least partially autonomous vehicle (200) depicted in Fig. 3, "The image classifier (60) may be configured to detect objects in the vicinity of the at least partially autonomous robot based on the input image (x). The output signal (y) may comprise an information, which characterizes where objects are located in the vicinity of the at least partially autonomous robot. The control signal (A) may then be determined in accordance with this information, for example to avoid collisions with the detected objects." The autonomous vehicle uses the information received from the image segmentation model to navigate, as the actuator is controlled via the control signal (A).)
Regarding claim 6, the rejection of claim 1 is incorporated herein. Further, Khoreva teaches
wherein the image segmentation model is a deep neural network model. (See rejection of claim 1 for reasoning behind interpretation of the first unit (721) as the image segmentation model. [0068] states “The first unit (721) may especially be a neural network, in particular a convolutional neural network.” A convolutional neural network is a type of deep neural network model.)
Regarding claim 7, the rejection of claim 1 is incorporated herein. Further, Khoreva teaches
wherein an adversarial generator uses a machine learning process to create the adversarial image. ([0045] recites a step in training the Generative Adversarial Network: "Based on the first output, training the generator to generate images and corresponding label maps, which are classified into a second class, which characterizes images and label maps that have not been generated by the generator;". The generator in a GAN is an adversarial generator. Also, [0048] states "Training the GAN may be understood as teaching the GAN to learn about a probability distribution of the images and corresponding label maps of the training dataset such that it is able to generate images and label maps that "look like" data from the training dataset." A machine learning process is used in order to generate images.)
Regarding claim 8, the rejection of claim 1 is incorporated herein. Khoreva teaches
the image segmentation model ([0085] states that "The input signal (x) is then passed on to a classifier (60), which is configured for semantic segmentation or object detection.")
The combination of Khoreva and Wang does not appear to explicitly teach
wherein the image [classification] model is a student model that was generated based on a publicly available teacher model.
However, Wang-NPL—directed to analogous art—teaches
wherein the image … model is a student model that was generated based on a publicly available teacher model. (The end of section 2.1 Transfer Learning on page 1282 states "We target facial recognition, where the student task is to recognize a set of 65 faces, and uses a well-performing face recognition model called VGGFace [11] as teacher model. Using only 10 images per class to train the student model, we achieve 93.47% classification accuracy.” VGGFace is a publicly available model.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of this application to combine the teachings of Khoreva and Wang with the teacher-student model of Wang-NPL for the reasons given above in regards to claim 1. Additionally, as stated by Wang in the Introduction, "While advances in deep learning seem to arrive on a daily basis, one constraint has remained: deep learning can only build accurate models by training using large datasets. This thirst for data severely constrains the number of different models that can be independently trained. In addition, the process of training large, accurate models (often with millions of parameters) requires computational resources that can be prohibitive for individuals or small companies."
Regarding claim 9, the rejection of claim 1 is incorporated herein. Khoreva teaches
[wherein the discriminator] determines that an image received from the autonomous vehicle is another adversarial image ([0094 states, in relation to the at least partially autonomous vehicle (200) of Figure 4, that "The input signal (x) may hence be understood as an input image and the classifier (60) as an image classifier." [0085] states that "The input signal (x) is then passed on to a classifier (60), which is configured for semantic segmentation or object detection.")
Khoreva does not appear to explicitly teach
wherein when the discriminator determines that an … image is another adversarial image, no [classification] process is performed on the another adversarial image.
However, Wang—directed to analogous art—teaches
wherein when the discriminator determines that the image … is another adversarial image, no [classification] is performed on the another adversarial image. (Wang-NPL teaches a system in which when the discriminator outputs a result indicating that the image is fake, no classification is performed on the image. [0068] states that a step includes "outputting a result indicating real and an index of a predetermined classification, or a result indicating fake." By not including the fake results, according to [0069], "The detection time will be reduced, and there is no need to collect anomalous samples, which are much harder to get than the normal samples. The efficiency may be also improved.")
It would have been obvious to a person having ordinary skill in the art before the effective filing date of this application to combine the teachings of Khoreva and Wang for the reasons given above in regards to claim 1.
Regarding claim 10, the rejection of claim 1 is incorporated herein. The combination of Khoreva and Xu does not appear to explicitly teach
wherein the perturbation in the adversarial image is created using a loss function that is combined with cross-entropy and a dissimilarity function.
However, Wang-NPL—directed to analogous art—teaches
PNG
media_image1.png
200
400
media_image1.png
Greyscale
wherein the perturbation in the adversarial image is created using a loss function that is combined with cross-entropy and a dissimilarity function. (Section 6.2 Injecting Neuron Distances provides a loss function that combines cross-entropy and a dissimilarity function. “Consider a Student model, where the first
K
layers are copied from the Teacher. Let
T
k
(
.
)
, and
S
k
(
.
)
be functions that generate the internal representation at layer
K
, for the Teacher, and Student respectively. Let
I
be the set of neurons in layer
K
, and
W
s
be a vector of absolute sum of outgoing weights from each neuron
i
∈
I
. Finally, let
D
t
h
be a dissimilarity threshold between two models. Then our objective is the following, where
∘
is element-wise multiplication.”)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of this application to combine the teachings of Khoreva and Xu with the teacher-student model and the function taught by Wang because of the reasons given above in regards to claim 1.
Regarding claim 11, Khoreva teaches
A non-transitory computer readable storage medium having stored therein instructions that are executable by one or more hardware processors to perform operations comprising: (Figure 3 shows a non-transitory machine-readable storage medium (46) and [0092] states “Furthermore, the control system (40) may comprise at least one processor (45) and at least one machine-readable storage medium (46) on which instructions are stored which, if carried out, cause the control system (40) to carry out a method according to an aspect of the present invention.”)
The remainder of claim 11 is substantially similar to claim 1. Claim 11 is rejected with the same rationale, mutatis mutandis.
Claims 14-20 recite substantially similar subject matter as claims 4-10 respectively, and are rejected with the same rationale, mutatis mutandis.
Claims 2, 3, 12, and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Khoreva (US 2023/0031755 A1), Wang (US 2022/0415019 A1), Wang-NPL (With Great Training Comes Great Vulnerability: Practical Attacks against Transfer Learning, August 2018), and Pervin (“Adversarial Attack Driven Data Augmentation for Accurate and Robust Medical Image Segmentation”, May 2021) as applied to claim 1 above, and further in view of Ren (“Adversarial Attacks and Defenses in Deep Learning”, 2020).
Regarding claim 2, the rejection of claim 1 is incorporated herein. Further, the combination of Khoreva, Wang, Wang-NPL, and Pervin does not appear to explicitly teach
The method as recited in claim 1, wherein the adversarial image is generated using an optimized fast gradient sign method (FGSM) attack.
However, Ren—directed to analogous art—teaches
The method as recited in claim 1, wherein the adversarial image is generated using an optimized fast gradient sign method (FGSM) attack. (Section 3.3 describes BIM (Basic Iterative Method) to “improve the performance of FGSM” which “performs FGSM with a smaller step size and clips the updated adversarial sample into a valid range for T iterations”. Therefore, BIM can be considered an optimized FGSM attack. Further, it describes PGD (Projected Gradient Descent), “The PGD can be considered as a generalized version of BIM without the constraint
x
T
=
∈
.” As PGD is a generalized version of BIM, PGD can also be considered an optimized FGSM attack.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of this application to combine the teachings of Khoreva, Wang, Wang-NPL, and Pervin with the teachings of Ren because, as taught by Ren in Section 1. Introduction, “PGD adversarial training achieves state-of-the-art accuracy against a wide range of L1 attacks on several DL model benchmarks such as the modified National Institute of Standards and Technology (MNIST) database, the Canadian Institute for Advanced Research-10 (CIFAR-10) dataset, and ImageNet [13,14].”
Regarding claim 3, the rejection of claim 2, and therefore claim 1, is incorporated herein. Further, Khoreva does not appear to explicitly teach
The method as recited in claim 2, wherein the adversarial image exhibits less perturbation than an adversarial image generated by a non-optimized FGSM attack.
However, Ren—directed to analogous art—teaches
The method as recited in claim 2, wherein the adversarial image exhibits less perturbation than an adversarial image generated by a non-optimized FGSM attack. (See the above rejection of claim 2 for reasoning behind interpreting PGD as the optimized FGSM attack. Section 3.3 states, in relation to PGD, that “the adversarial perturbation size is smaller than
∈
” where
∈
, according to section 3.2, “is the magnitude of the perturbation” created using the FGSM attack. Because PGD creates a smaller perturbation than the perturbation of FGSM, the adversarial image that is generated by PGD would have less perturbation than an adversarial image generated by a non-optimized FGSM attack.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of this application to combine the teachings of Khoreva, Wang, Wang-NPL, and Pervin with the teachings of Ren because of the reasons given above in regards to claim 2.
Claims 12 and 13 recite substantially similar subject matter as claims 2 and 3 respectively, and are rejected with the same rationale, mutatis mutandis.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JESSICA THUY PHAM whose telephone number is (571)272-2605. The examiner can normally be reached Monday - Thursday, 7:30 A.M. - 5:30 P.M..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Li Zhen can be reached at (571) 272-3768. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.T.P./Examiner, Art Unit 2121
/Li B. Zhen/Supervisory Patent Examiner, Art Unit 2121
Prosecution Insights