DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 04/02/2026 has been entered.
Status of Claims
Claims 1-5, 7-11, 13-17, and 20-24 are pending. Claims 6, 12, and 18-19 are canceled. Claim 24 is new.
Response to Arguments
Applicant’s arguments, see p.11-14, filed 04/02/2026, with respect to the rejections of Claims 1-5, 7-11, 13-17, and 20-23 under 35 U.S.C. 103 have been fully considered but are moot because Applicant’s amendments of the independent claims has altered the scope of the claims, and therefore, necessitated new grounds of rejection which are presented below. Examiner has considered applicants arguments with respect to the new claim 24. However, arguments are moot due to new claims being presented and are therefore being analyzed as presented below.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 13, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Ma et al. (CN 116711308 A) in view of Ye et al. (US 20210390723 A1), Yuan et al. (CN 112418332 A), Chen et al. (CN 109948658 A), Han et al. (CN 109740682 B), and Yu et al. (US 20190028707 A1).
Regarding Claim 1, Ma teaches "An image processing method, comprising: obtaining a first feature map based on feature-encoding of an original image using a feature encoder of an adversarial attack network, the feature encoder comprising a convolutional layer and a residual block (ResBlock)"; (Ma, Pg. 2 Paras. 10-13 starting with "In a first aspect…", Pg. 7 Para. 8 starting with "In some embodiments…", and Pg. 10 Para. 10 starting with "In one example…", teaches the encoders and the segmentation unit being all neural networks such as a generative adversarial network wherein a current image is acquired and encoded by using a first encoder to obtain a first feature map of the current image and wherein the encoder includes convolutional layers and residual blocks, i.e., an image processing method that obtains a first feature map based on the feature-encoding of an original image using a feature encoder of an adversarial attack network wherein the encoder comprises at least a convolutional layer and a residual block).
However, Ma does not explicitly teach " obtaining, through a first feature decoder of the adversarial attack network coupled to the feature encoder, a second feature map of the original image based on the first feature map obtained from the feature encoder, the second feature map including noise information to be superimposed on the original image; obtaining, through a second feature decoder of the adversarial attack network coupled to the feature encoder, a third feature map of the original image based on the first feature map obtained from the feature encoder, the third feature map including different feature values, and each feature value representing a relative importance of an image feature at a position corresponding to the respective feature value, the first feature decoder and the second feature decoder having a same structure with (i) a same number of convolution layers having same respective kernel sizes and (ii) a same number of deconvolution layers having same respective kernel sizes; generating, by processing circuitry, a noise image by performing position-wise multiplication on the second feature map and the third feature map, the feature values of the third feature map being normalized into a predetermined range; and superimposing the original image and the noise image, to obtain a first adversarial example image by performing position-wise superimposition on the original image and the noise image".
In an analogous field of endeavor, Ye teaches "obtaining, through a first feature decoder of the adversarial attack network coupled to the feature encoder, a second feature map of the original image based on the first feature map obtained from the feature encoder"; (Ye, FIG. 1 and Para. 7 and Claim 1, teaches a network encoder-decoder structure that comprises a depth estimation sub-network and edge estimation sub-network which share the encoder and have their own decoders wherein the encoder converts the input color map into the feature map and the decoders contain deconvolution layers for up-sampling the feature map and converting the feature map into a depth map or edge map respectively, i.e., obtain a second feature map being the up-sampled feature map output from the first feature decoder being the DepthNet decoder coupled to the encoder of the input image based on the first feature map obtained from the encoder);
"
"obtaining, through a second feature decoder of the adversarial attack network coupled to the feature encoder, a third feature map of the original image based on the first feature map obtained from the feature encoder"; (Ye, FIG. 1 and Para. 7 and Claim 1, teaches a network encoder-decoder structure that comprises a depth estimation sub-network and edge estimation sub-network which share the encoder and have their own decoders wherein the encoder converts the input color map into the feature map and the decoders contain deconvolution layers for up-sampling the feature map and converting the feature map into a depth map or edge map respectively, i.e., obtain a third feature map being the up-sampled feature map output from the second feature decoder being the EdgeNet decoder coupled to the encoder of the input image based on the first feature map obtained from the encoder).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Ma wherein the network is an adversarial attack network by including the obtaining of a second and third feature map from a first and second feature decoder respectively based on the first feature map from the encoder taught by Ye. One of ordinary skill in the art would be motivated to combine the references since it enhances the ability of the network (Ye, Para. 39, teaches the motivation of combination to be to enhance the ability of the network).
However, the combination of references of Ma in view of Ye does not explicitly teach "the second feature map including noise information to be superimposed on the original image; the third feature map including different feature values, and each feature value representing a relative importance of an image feature at a position corresponding to the respective feature value; the first feature decoder and the second feature decoder having a same structure with (i) a same number of convolution layers having same respective kernel sizes and (ii) a same number of deconvolution layers having same respective kernel sizes; generating, by processing circuitry, a noise image by performing position-wise multiplication on the second feature map and the third feature map; the feature values of the third feature map being normalized into a predetermined range; and superimposing the original image and the noise image, to obtain a first adversarial example image by performing position-wise superimposition on the original image and the noise image".
In an analogous field of endeavor, Yuan teaches "the second feature map including noise information to be superimposed on the original image"; (Yuan, Pg. 4 Paras. 5-8 starting with "The disturbance feature map…" and Pg. 7 Paras. 4-10 starting with "Referring to fig. 1…", teaches respectively extracting the features of a source image and a target image to obtain a first feature map and a second feature map to obtain a disturbance feature map based on the first feature map, second feature map, and the disturbance limited item in which the disturbance feature map is fused with the source image and wherein the features of the related source image which correspond to the first feature map and features of the target image which correspond to the second feature map may be simultaneously learned based on the disturbance restriction term to obtain the disturbance feature map which is fused with the source image, i.e., the second feature map includes noise information which is later superimposed on the original image).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Ma and Ye wherein the second feature map is obtained from a decoder based on the first feature map by including the second feature map including noise information that will be superimposed on the original image taught by Yuan. One of ordinary skill in the art would be motivated to combine the references since it creates high generation efficiency (Yuan, Abstract, teaches the motivation of combination to be to create a simple operation process with high generation efficiency).
However, the combination of references of Ma in view of Ye and Yuan does not explicitly teach "the third feature map including different feature values, and each feature value representing a relative importance of an image feature at a position corresponding to the respective feature value; the first feature decoder and the second feature decoder having a same structure with (i) a same number of convolution layers having same respective kernel sizes and (ii) a same number of deconvolution layers having same respective kernel sizes; generating, by processing circuitry, a noise image by performing position-wise multiplication on the second feature map and the third feature map; the feature values of the third feature map being normalized into a predetermined range; and superimposing the original image and the noise image, to obtain a first adversarial example image by performing position-wise superimposition on the original image and the noise image".
In an analogous field of endeavor, Chen teaches "the third feature map including different feature values, and each feature value representing a relative importance of an image feature at a position corresponding to the respective feature value"; (Chen, Claim 2, teaches up-sampling a feature image to obtain a reconstructed feature image and calculating a channel space attention weight matrix according to the original image and reconstructed feature image, i.e., obtaining an additional or third feature map of original image based on a first feature map wherein the feature values would be different, and calculating a pixel space attention weight matrix according to the reconstructed channel space attention weight matrix and the original image, i.e., the feature values represent relative importance of image features at corresponding positions);
"
"generating, by processing circuitry, a noise image by performing position-wise multiplication on the second feature map and the third feature map"; (Chen, Claims 2-5 and Pg. 16 lines 7-20, teaches the added disturbance is calculated according to the number of channels of the pixel matrix of the original image and then later added to the original image to obtain the confrontation sample, i.e., the disturbance quantity functions as a noise image which is later added element-wise to the original image to form the sample image, i.e., generating a noise image, wherein the added disturbance quantity is calculated according to multiplying corresponding elements of the two matrices in which the pixel space attention weight matrix is calculated according to the reconstructed channel space attention weight matrix and the original image and a reshape function of the pixel space attention weight into the size of the attention mapping weight, i.e., position-wise multiplication on the second and third feature map being the corresponding element multiplication of the two matrices as the feature maps based on relative importance of the image feature at the position corresponding to each of the respective feature values being the space attention weight matrices);
"
"and superimposing the original image and the noise image, to obtain a first adversarial example image by performing position-wise superimposition on the original image and the noise image"; (Chen, Claims 2-5, teaches obtaining the confrontation sample through the addition of the corresponding elements of the matrices of the calculated added disturbance and the original image, i.e., superimposing the original image and the noise image to obtain an adversarial example image by performing position-wise superimposition on the original image and the noise image being the element-wise addition of the disturbance quantity image to the original image to obtain the confrontation sample image).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Ma, Ye, and Yuan wherein the third feature map is obtained through a second feature decoder based on the first feature map by including the generation of an additional feature map with different values of relative importance of image feature position and superimposing a noise image on an original image to obtain an adversarial image taught by Chen. One of ordinary skill in the art would be motivated to combine the references since it improves robustness of the model (Chen, Abstract, teaches the motivation of combination to be to improve the robustness, generalization ability of the classifier, reliability, stability, and safety of the deep learning model).
However, the combination of references of Ma in view of Ye, Yuan, and Chen does not explicitly teach "the first feature decoder and the second feature decoder having a same structure with (i) a same number of convolution layers having same respective kernel sizes and (ii) a same number of deconvolution layers having same respective kernel sizes; the feature values of the third feature map being normalized into a predetermined range".
In an analogous field of endeavor, Han teaches "the first feature decoder and the second feature decoder having a same structure with (i) a same number of convolution layers having same respective kernel sizes and (ii) a same number of deconvolution layers having same respective kernel sizes";(Han, Claim 1, teaches a first and second decoder which both include two deconvolution layers and one convolution layer wherein the kernel size for each layer is 3*3, i.e., first and second feature decoder have a same structure with same number of convolution layers, same number of deconvolution layers and same respective kernel sizes for both).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Ma, Ye, Yuan, and Chen by including the decoders having the same structure taught by Han. One of ordinary skill in the art would be motivated to combine the references since it improves the recognition accuracy (Han, Pg. 8, teaches the motivation of combination to be to greatly improve the recognition accuracy of the model).
However, the combination of references of Ma in view of Ye, Yuan, Chen, and Han does not explicitly teach "the feature values of the third feature map being normalized into a predetermined range".
In an analogous field of endeavor, Yu teaches "the feature values of the third feature map being normalized into a predetermined range";(Yu, Para. 96, teaches each feature map is normalized to have the range of [0,1], i.e., feature values of a feature map being normalized to a predetermined range).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Ma, Ye, Yuan, Chen, and Han by including the feature values of the feature map being normalized to a predetermined range taught by Yu. One of ordinary skill in the art would be motivated to combine the references since it improves visual saliency accuracy (Yu, Para. 26, teaches the motivation of combination to be to improve the visual saliency accuracy).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date.
Claim 13 recites a system or apparatus with elements corresponding to the steps recited in Claim 1. Therefore, the recited elements of this claim are mapped to the proposed combination in the same manner as the corresponding steps in its corresponding method claim. Additionally, the rationale and motivation to combine the Ma in view of Ye, Yuan, Chen, Han, and Yu references, presented in rejection of Claim 1, apply to this claim. Finally, the combination of the Ma in view of Ye, Yuan, Chen, Han, and Yu references discloses a processor configured to execute the program instructions (for example, see Ma, Page 34).
Claim 20 recites a computer-readable storage medium storing a program with instructions corresponding to the steps recited in Claim 1. Therefore, the recited programming instructions of this claim are mapped to the proposed combination in the same manner as the corresponding steps in its corresponding method claim. Additionally, the rationale and motivation to combine the Ma in view of Ye, Yuan, Chen, Han, and Yu references, presented in rejection of Claim 1, apply to this claim. Finally, the combination of the Ma in view of Ye, Yuan, Chen, Han, and Yu references discloses a computer readable storage medium (for example, see Ma, Page 34).
Claims 2, 14, and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Ma in view of Ye, Yuan, Chen, Han, Yu, Gollanapalli et al. (US 20210183020 A1), and Ren et al. (US 20190095795 A1).
Regarding Claim 2, the combination of references of Ma in view of Ye, Yuan, Chen, Han, and Yu does not explicitly teach "The method according to claim 1, wherein the obtaining the first feature map comprises: inputting the original image into the feature encoder of the adversarial attack network for the feature-encoding, to obtain the first feature map, a size of the first feature map being less than a size of the original image; the feature encoder including the convolutional layer and the ResBlock that is located after the convolutional layer in connection order; each ResBlock including an identity mapping and at least two convolutional layers, and the identity mapping of the ResBlock pointing to an output end of the ResBlock from an input end of the ResBlock”.
In an analogous field of endeavor, Gollanapalli teaches "The method according to claim 1, wherein the obtaining the first feature map comprises: inputting the original image into the feature encoder of the adversarial attack network for the feature-encoding, to obtain the first feature map, a size of the first feature map being less than a size of the original image"; (Gollanapalli, Paras. 53 and 65, teaches obtaining a feature map with lower dimensions than a feature encoded image wherein the image is encoded from an adversarial network, i.e., a size of a first feature map is less than a size of an original image encoded by an adversarial network).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Ma in view of Ye, Yuan, Chen, Han, and Yu by including the obtaining a feature map with lower dimensions than an input image taught by Gollanapalli. One of ordinary skill in the art would be motivated to combine the references since it optimizes convolutions (Gollanapalli, Paras. 4-6, teaches the motivation of combination to be to optimize convolutions in order to recover blurred features from a single image).
However, the combination of references of Ma in view of Ye, Yuan, Chen, Han, Yu, and Gollanapalli does not explicitly teach "the feature encoder including the convolutional layer and the ResBlock that is located after the convolutional layer in connection order; each ResBlock including an identity mapping and at least two convolutional layers, and the identity mapping of the ResBlock pointing to an output end of the ResBlock from an input end of the ResBlock".
In an analogous field of endeavor, Ren teaches "the feature encoder including the convolutional layer and the ResBlock that is located after the convolutional layer in connection order"; (Ren, Figs. 12-14 and Paras. 94 and 119-121, teaches inserting ResBlocks after convolutional layers wherein the convolutional layers extract features, i.e., feature encoding convolutional layer with ResBlocks located after the convolution layers in connection);
"each ResBlock including an identity mapping and at least two convolutional layers, and the identity mapping of the ResBlock pointing to an output end of the ResBlock from an input end of the ResBlock"; (Ren, Abstract and Para. 119, teaches the ResBlocks including at least two additional convolutional layers and skip connections which pass through a scale layer with the same number of feature maps, i.e., an identity mapping pointing to an output end of the ResBlock from an input end).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Ma in view of Ye, Yuan, Chen, Han, Yu, and Gollanapalli by including the residual block after convolutional layers wherein the ResBlock has identity mapping and at least two convolutional layers taught by Ren. One of ordinary skill in the art would be motivated to combine the references since it eases convergence and increases accuracy (Ren, Para. 40, teaches the motivation of combination to be to ease convergence and consistently increase accuracy).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date.
Claim 14 recites a system or apparatus with elements corresponding to the steps recited in Claim 2. Therefore, the recited elements of this claim are mapped to the proposed combination in the same manner as the corresponding steps in its corresponding method claim. Additionally, the rationale and motivation to combine the Ma in view of Ye, Yuan, Chen, Han, Yu, Gollanapalli, and Ren references, presented in rejection of Claim 2, apply to this claim. Finally, the combination of the Ma in view of Ye, Yuan, Chen, Han, Yu, Gollanapalli, and Ren references discloses a processor configured to execute the program instructions (for example, see Ma, Page 34).
Claim 21 recites a computer-readable storage medium storing a program with instructions corresponding to the steps recited in Claim 2. Therefore, the recited programming instructions of this claim are mapped to the proposed combination in the same manner as the corresponding steps in its corresponding method claim. Additionally, the rationale and motivation to combine the Ma in view of Ye, Yuan, Chen, Han, Yu, Gollanapalli, and Ren references, presented in rejection of Claim 2, apply to this claim. Finally, the combination of the Ma in view of Ye, Yuan, Chen, Han, Yu, Gollanapalli, and Ren references discloses a computer readable storage medium (for example, see Ma, Page 34).
Claims 3, 5, 15, 17, 22, and 23 are rejected under 35 U.S.C. 103 as being unpatentable over Ma in view of Ye, Yuan, Chen, Han, Yu, Ren, and Yang et al. (US 20210314474 A1).
Regarding Claim 3, the combination of references of Ma in view of Ye, Yuan, Chen, Han, and Yu does not explicitly teach The method according to claim 1, wherein the obtaining the second feature map comprises: inputting the first feature map into the first feature decoder of the adversarial attack network for feature-decoding, to obtain an original noise feature map, the first feature decoder including at least one deconvolutional layer and at least one convolutional layer that is located after the at least one deconvolutional layer in connection order; and performing suppression processing on a noise feature value at each of a plurality of positions on the original noise feature map to obtain the second feature map; a size of the second feature map being equal to the size of the original image".
In an analogous field of endeavor, Yang teaches "The method according to claim 1, wherein the obtaining the second feature map comprises: inputting the first feature map into the first feature decoder of the adversarial attack network for feature-decoding, to obtain an original noise feature map, the first feature decoder including at least one deconvolutional layer and at least one convolutional layer that is located after the at least one deconvolutional layer in connection order"; (Yang, Fig. 2 and Paras. 54 and 89, teaches a decoder network part of an adversarial network that decodes a feature map to generate a noise-reduced image wherein the decoder includes convolutional layers after the deconvolutional layers, i.e., obtain an original noise feature map from decoding a first feature map);
"and performing suppression processing on a noise feature value at each of a plurality of positions on the original noise feature map to obtain the second feature map"; (Yang, Para. 89, teaches blending features from the feature map to generate a noise-free image or a noise-reduced image, i.e., performing suppression processing on noise feature values at positions of the noise feature map).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Ma in view of Ye, Yuan, Chen, Han, and Yu by including the decoding of a feature map to obtain a noise feature map wherein noise values are suppressed taught by Yang. One of ordinary skill in the art would be motivated to combine the references since it improves the generation of sharp frames (Yang, Para. 64, teaches the motivation of combination to be to improve generation of sharp frames).
However, the combination of references of Ma in view of Ye, Yuan, Chen, Han, Yu, and Yang does not explicitly teach "a size of the second feature map being equal to the size of the original image".
In an analogous field of endeavor, Ren teaches "a size of the second feature map being equal to the size of the original image"; (Ren, Para. 60, teaches feature maps having the same size of the output, i.e., size of second feature map being equal to size of original image).
The proposed combination as well as the motivation for combining the Ma in view of Ye, Yuan, Chen, Han, Yu, and Ren references presented in the rejection of Claim 2, applies to claim 3. Thus, the method recited in claim 3 is met by Ma in view of Ye, Yuan, Chen, Han, Yu, Yang, and Ren.
Regarding Claim 5, the combination of references of Ma in view of Ye, Yuan, Chen, Han, Yu, Yang, and Ren teaches "The method according to claim 1, wherein the obtaining the third feature map comprises: inputting the first feature map into the second feature decoder of the adversarial attack network for feature-decoding, to obtain an original weighting feature map, the second feature decoder including at least one deconvolutional layer and at least one convolutional layer that is located after the at least one deconvolutional layer in connection order"; (Yang, Fig. 2 and Paras. 58-59, teaches a decoder network part of an adversarial network that decodes a feature map to generate a sharp frame wherein the decoder includes convolutional layers after the deconvolutional layers, i.e., obtain an additional feature map from inputting a feature map into a decoder of an adversarial network wherein the decoder includes convolutional layers after deconvolutional layers);
"and performing normalization processing on each feature value of the original weighting feature map to obtain the third feature map"; (Yang, FIG. 3 and Para. 63, teaches the adversarial network including a discriminator that performs a normalization function on the output being the sharp frame resulting from the decoder processing the feature maps, i.e., performing normalization processing on an image feature values of the additional feature map to obtain a processed additional feature map).
The proposed combination as well as the motivation for combining the Ma in view of Ye, Yuan, Chen, Han, Yu, Yang, and Ren references presented in the rejection of Claim 3, applies to claim 5.
In an analogous field of endeavor, Ren teaches "a size of the third feature map being equal to the size of the original image"; (Ren, Para. 60, teaches feature maps having the same size of the output, i.e., size of feature map being equal to size of original image).
The proposed combination as well as the motivation for combining the Ma in view of Ye, Yuan, Chen, Han, Yu, Gollanapalli, and Ren references presented in the rejection of Claim 2, applies to claim 5. Thus, the method recited in claim 5 is met by Ma in view of Ye, Yuan, Chen, Han, Yu, Yang, and Ren.
Claim 15 recites a system or apparatus with elements corresponding to the steps recited in Claim 3. Therefore, the recited elements of this claim are mapped to the proposed combination in the same manner as the corresponding steps in its corresponding method claim. Additionally, the rationale and motivation to combine the Ma in view of Ye, Yuan, Chen, Han, Yu, Yang, and Ren references, presented in rejection of Claim 3, apply to this claim. Finally, the combination of the Ma in view of Ye, Yuan, Chen, Han, Yu, Yang, and Ren references discloses a processor configured to execute the program instructions (for example, see Ma, Page 34).
Claim 17 recites a system or apparatus with elements corresponding to the steps recited in Claim 5. Therefore, the recited elements of this claim are mapped to the proposed combination in the same manner as the corresponding steps in its corresponding method claim. Additionally, the rationale and motivation to combine the Ma in view of Ye, Yuan, Chen, Han, Yu, Yang, and Ren references, presented in rejection of Claim 5, apply to this claim. Finally, the combination of the Ma in view of Ye, Yuan, Chen, Han, Yu, Yang, and Ren references discloses a processor configured to execute the program instructions (for example, see Ma, Page 34).
Claim 22 recites a computer-readable storage medium storing a program with instructions corresponding to the steps recited in Claim 3. Therefore, the recited programming instructions of this claim are mapped to the proposed combination in the same manner as the corresponding steps in its corresponding method claim. Additionally, the rationale and motivation to combine the Ma in view of Ye, Yuan, Chen, Han, Yu, Yang, and Ren references, presented in rejection of Claim 3, apply to this claim. Finally, the combination of the Ma in view of Ye, Yuan, Chen, Han, Yu, Yang, and Ren references discloses a computer readable storage medium (for example, see Ma, Page 34).
Claim 23 recites a computer-readable storage medium storing a program with instructions corresponding to the steps recited in Claim 5. Therefore, the recited programming instructions of this claim are mapped to the proposed combination in the same manner as the corresponding steps in its corresponding method claim. Additionally, the rationale and motivation to combine the Ma in view of Ye, Yuan, Chen, Han, Yu, Yang, and Ren references, presented in rejection of Claim 5, apply to this claim. Finally, the combination of the Ma in view of Ye, Yuan, Chen, Han, Yu, Yang, and Ren references discloses a computer readable storage medium (for example, see Ma, Page 34).
Claims 4 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Ma in view of Ye, Yuan, Chen, Han, Yu, Ren, Yang, and Watanabe et al. (US 20150103250 A1).
Regarding Claim 4, the combination of references of Ma in view of Ye, Yuan, Chen, Han, Yu, Ren, and Yang does not explicitly teach "The method according to claim 3, wherein the performing the suppression processing comprises: replacing the noise feature value of each of the plurality of positions with a target threshold when the noise feature value of the respective position is greater than the target threshold".
In an analogous field of endeavor, Watanabe teaches "The method according to claim 3, wherein the performing the suppression processing comprises: replacing the noise feature value of each of the plurality of positions with a target threshold when the noise feature value of the respective position is greater than the target threshold"; (Watanabe, Para. 75, teaches a noise reducing section wherein a noise feature value that is equal or greater than the threshold value is reassigned to the number 10, i.e., replacing a noise feature value of a plurality of pixel positions with a target threshold when the feature value exceeds a threshold).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Ma in view of Ye, Yuan, Chen, Han, Yu, Ren, and Yang by including the replacement of a noise feature value when it exceeds a threshold taught by Watanabe. One of ordinary skill in the art would be motivated to combine the references since it creates effective smoothing (Watanabe, Para. 13, teaches the motivation of combination to be to create effective smoothing on an image in accordance with its features).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date.
Claim 16 recites a system or apparatus with elements corresponding to the steps recited in Claim 4. Therefore, the recited elements of this claim are mapped to the proposed combination in the same manner as the corresponding steps in its corresponding method claim. Additionally, the rationale and motivation to combine the Ma in view of Ye, Yuan, Chen, Han, Yu, Yang, Ren, and Watanabe references, presented in rejection of Claim 4, apply to this claim. Finally, the combination of the Ma in view of Ye, Yuan, Chen, Han, Yu, Yang, Ren, and Watanabe references discloses a processor configured to execute the program instructions (for example, see Ma, Page 34).
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Ma in view of Ye, Yuan, Chen, Han, Yu, Gollanapalli, Ren, and Liu et al. (US 20200285952 A1).
Regarding Claim 7, the combination of references of Ma in view of Ye, Yuan, Chen, Han, Yu, Gollanapalli, and Ren does not explicitly teach "The method according to claim 2, further comprising: inputting the first adversarial example image into an image recognition model of the adversarial attack network; and obtaining an image recognition result of the first adversarial example image from the image recognition model".
In an analogous field of endeavor, Liu’952 teaches "The method according to claim 2, further comprising: inputting the first adversarial example image into an image recognition model of the adversarial attack network"; (Liu'952, Para. 24, teaches inputting an adversarial input, i.e., adversarial example image, into a facial recognition neural network, i.e., image recognition model of the adversarial attack network);
"and obtaining an image recognition result of the first adversarial example image from the image recognition model"; (Liu'952, Para. 24, teaches the misclassification of the input due to the adversarial attack input, i.e., obtaining an image recognition result).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Ma in view of Ye, Yuan, Chen, Han, Yu, Gollanapalli, and Ren by including the image recognition result from the adversarial image taught by Liu’952. One of ordinary skill in the art would be motivated to combine the references since it improves data processing of vulnerabilities (Liu'952, Para. 1, teaches the motivation of combination to be to improve the data processing of quantifying vulnerabilities of adversarial perturbations).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date.
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Ma in view of Ye, Yuan, Chen, Han, Yu, Gollanapalli, Ren, Liu’952, and Wang (US 20210406582 A1).
Regarding Claim 8, the combination of references of Ma in view of Ye, Yuan, Chen, Gollanapalli, Ren, and Liu’952 teaches "The method according to claim 7, further comprising: training the adversarial attack network, the training including: obtaining a second adversarial example image of a sample image included in a training dataset"; (Liu'952, Para. 83, teaches obtaining a second adversarial image of the natural image by perturbing one or more pixels of a second grid of the natural image, i.e., obtaining a second adversarial example image of a sample image included in the training data);
"inputting the sample image and the second adversarial example image into the image recognition model for feature-encoding, to obtain feature data of the sample image and feature data of the second adversarial example image"; (Liu'952, Para. 83, teaches the second adversarial image and natural image is processed by the computer model to obtain a feature map, i.e., inputting sample image and adversarial image into the recognition computer model for feature-encoding and obtaining feature data);
"
"obtaining another feature map of the sample image, the other feature map of the sample image including different feature values, and each feature value of the other feature map representing the relative importance of an image feature at the position corresponding to the respective feature value"; (Chen, Claim 2, teaches up-sampling a feature image to obtain a reconstructed feature image and calculating a channel space attention weight matrix according to the original image and reconstructed feature image, i.e., obtaining an additional or third feature map of sample image based on a first feature map wherein the feature values would be different, and calculating a pixel space attention weight matrix according to the reconstructed channel space attention weight matrix and the original image, i.e., the feature values represent relative importance of image features at corresponding positions);
The proposed combination as well as the motivation for combining the Ma in view of Ye, Yuan, Chen, Han, Yu, Gollanapalli, Ren, and Liu’952 references presented in the rejection of Claim 7, applies to claim 8.
However, the combination of references of Ma in view of Ye, Yuan, Chen, Han, Yu, Gollanapalli, Ren, and Liu’952 does not explicitly teach "obtaining a first loss function value and a second loss function value based on the feature data of the sample image and the feature data of the second adversarial example image; obtaining a third loss function value based on the other feature map of the sample image; and performing end-to-end training on an initial adversarial attack network to obtain the adversarial attack network based on the first loss function value, the second loss function value, and the third loss function value".
In an analogous field of endeavor, Wang teaches "obtaining a first loss function value and a second loss function value based on the feature data of the sample image and the feature data of the second adversarial example image"; (Wang, Para. 24, teaches computing first and second losses corresponding to the plurality of subpixels of the training image and input image and training predictions wherein the predictions are determined based on first and second feature maps respectively, i.e., a first and second loss function value based on feature data of the sample image and example image);
"obtaining a third loss function value based on the other feature map of the sample image"; (Wang, Para. 149, teaches computing third losses using a third feature analyzing branch, i.e., third loss function value based on third feature map);
"and performing end-to-end training on an initial adversarial attack network to obtain the adversarial attack network based on the first loss function value, the second loss function value, and the third loss function value"; (Wang, Para. 29 and 151-153, teaches training a neural network based on first losses, second losses, and third losses, i.e., performing end-to-end training a network to obtain a network based on first, second, and third loss function values).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Ma in view of Ye, Yuan, Chen, Han, Yu, Gollanapalli, Ren, and Liu’952 wherein the example image is the adversarial image and the network is an adversarial network by including the determination of three loss function based on feature data to train the network taught by Wang. One of ordinary skill in the art would be motivated to combine the references since it improves the accuracy of image segmentation and classification (Wang, Para. 103, teaches the motivation of combination to be to improve the accuracy of image segmentation and classification).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date.
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Ma in view of Ye, Yuan, Chen, Han, Yu, Gollanapalli, Ren, Liu’952, Wang, and Park et al. (US 20210166071 A1).
Regarding Claim 9, the combination of references of Ma in view of Ye, Yuan, Chen, Han, Yu, Gollanapalli, Ren, Liu’952, and Wang does not explicitly teach "The method according to claim 8, wherein the obtaining the first loss function value comprises: isolating, from the feature data of the sample image, a feature angle of the sample image; isolating, from the feature data of the second adversarial example image, a feature angle of the second adversarial example image; and obtaining the first loss function value based on the feature angle of the sample image and the feature angle of the second adversarial example image, an optimization objective of the first loss function value being to increase a feature angle between the sample image and the second adversarial example image".
In an analogous field of endeavor, Park teaches "The method according to claim 8, wherein the obtaining the first loss function value comprises: isolating, from the feature data of the sample image, a feature angle of the sample image"; (Park, Para. 17, teaches a loss function including feature angles from an input image and training image, i.e., obtaining first loss function comprising isolating a feature angle from sample image);
"isolating, from the feature data of the second adversarial example image, a feature angle of the second adversarial example image"; (Park, Para. 14, teaches including a plurality of feature angles from input feature vectors based on an input image, enrolled feature vectors of an enrolled image, and a training image, i.e., isolating a feature angle from an additional image);
"and obtaining the first loss function value based on the feature angle of the sample image and the feature angle of the second adversarial example image, an optimization objective of the first loss function value being to increase a feature angle between the sample image and the second adversarial example image"; (Park, Paras. 5 and 11, teaches a loss function determined from a plurality of feature vectors and class vectors among a plurality of classes with a plurality of feature angles, i.e., obtaining a first loss function value based on feature angles between images of different classes, and updated the neural network such that the feature angle is reduced and the class angle is increased, i.e., optimizing the network using the loss function by increasing feature angles between images from different classes).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Ma in view of Ye, Yuan, Chen, Han, Yu, Gollanapalli, Ren, Liu’952, and Wang wherein the additional image is a second adversarial example and images of different classes are the sample image and adversarial image by including the determination of feature angles between images for a loss function and optimizing the function by increasing the angles between images of different classes taught by Park. One of ordinary skill in the art would be motivated to combine the references since it increases performance of the network and reduces loss (Park, Para. 10, teaches the motivation of combination to be to update the parameters of the neural network for increased performance and reduce loss).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Ma in view of Ye, Yuan, Chen, Han, Yu, Gollanapalli, Ren, Liu’952, Wang, Yokoyama et al. (US 20230024820 A1), and Cai et al. (US 11409304 B1).
Regarding Claim 10, the combination of references of Ma in view of Ye, Yuan, Chen, Han, Yu, Gollanapalli, Ren, Liu’952, and Wang does not explicitly teach "The method according to claim 8, wherein the obtaining the second loss function value comprises: isolating, from the feature data of the sample image, a feature modulus value of the sample image; isolating, from the feature data of the second adversarial example image, a feature modulus value of the second adversarial example image; and obtaining the second loss function value based on the feature modulus value of the sample image and the feature modulus value of the second adversarial example image, an optimization objective of the second loss function value being to reduce a difference between the feature modulus value of the sample image and the feature modulus value of the second adversarial example image".
In an analogous field of endeavor, Yokoyama teaches "The method according to claim 8, wherein the obtaining the second loss function value comprises: isolating, from the feature data of the sample image, a feature modulus value of the sample image"; (Yokoyama, Paras. 22 and 33, teach determining magnitudes of predetermined feature quantities of a non-defective product image, i.e., isolating a feature modulus value from a sample image);
"isolating, from the feature data of the second adversarial example image, a feature modulus value of the second adversarial example image"; (Yokoyama, Paras. 22 and 33, teach determining magnitudes of predetermined feature quantities of a defective product image, i.e., isolating a feature modulus value of a second image of a pair of images).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Ma in view of Ye, Yuan, Chen, Gollanapalli, Ren, Liu’952, and Wang wherein image pairs are the sample image and adversarial example image by including the determination of a feature modulus of the images from the feature data taught by Yokoyama. One of ordinary skill in the art would be motivated to combine the references since it visualizes the accuracy of the determination (Yokoyama, Paras. 6-7, teach the motivation of combination to be to visualize the accuracy of the determine device).
However, the combination of references of Ma in view of Ye, Yuan, Chen, Han, Yu, Gollanapalli, Ren, Liu’952, Wang, and Yokoyama does not explicitly teach "and obtaining the second loss function value based on the feature modulus value of the sample image and the feature modulus value of the second adversarial example image, an optimization objective of the second loss function value being to reduce a difference between the feature modulus value of the sample image and the feature modulus value of the second adversarial example image".
In an analogous field of endeavor, Cai teaches "and obtaining the second loss function value based on the feature modulus value of the sample image and the feature modulus value of the second adversarial example image, an optimization objective of the second loss function value being to reduce a difference between the feature modulus value of the sample image and the feature modulus value of the second adversarial example image"; (Cai, Col. 12 lines 9-19 and Col. 26 lines 36-67, teaches a loss function based on minimizing or optimizing the differences of parameters of the machine-learned model wherein the parameters of the model include feature magnitude, i.e., obtaining a loss function value based on feature modulus wherein the loss function reduces a difference between model parameters).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Ma in view of Ye, Yuan, Chen, Han, Yu, Gollanapalli, Ren, Liu’952, Wang, and Yokoyama wherein the model parameters include feature modulus values of the images by including the reducing of a difference between model parameters using the loss function taught by Cai. One of ordinary skill in the art would be motivated to combine the references since it improves the accuracy of predictions (Cai, Col. 26 lines 36-67, teach the motivation of combination to be to teach the model to more accurately predict object behavior from image features as well as refining top-down predictions over time).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Ma in view of Ye, Yuan, Chen, Han, Yu, Gollanapalli, Ren, Liu’952, Wang, Chen et al. (US 20220005200 A1), and Chaudhury et al. (US 20220172080 A1).
Regarding Claim 11, the combination of references of Ma in view of Ye, Yuan, Chen, Han, Yu, Gollanapalli, Ren, Liu’952, and Wang teaches "
"
"
"and performing the end-to-end training on the initial adversarial attack network to obtain the adversarial attack network"; (Wang, Para. 29 and 151-153, teaches training a neural network based on first losses, second losses, and third losses, i.e., performing end-to-end training on a network to obtain a network based on first, second, and third loss function values).
The proposed combination as well as the motivation for combining the Ma in view of Ye, Yuan, Chen, Han, Yu, Gollanapalli, Ren, Liu’952, and Wang references presented in the rejection of Claim 8, applies to claim 11.
However, the combination of references of Ma in view of Ye, Yuan, Chen, Han, Yu, Gollanapalli, Ren, Liu’952, and Wang does not explicitly teach "The method according to claim 8, wherein the performing the end-to-end training comprises: obtaining a first sum value of the second loss function value and the third loss function value; obtaining a product value of a target constant and the first sum value; determining a second sum value of the first loss function value and the product value as a final loss function value”.
In an analogous field of endeavor, Chen’200 teaches "The method according to claim 8, wherein the performing the end-to-end training comprises: obtaining a first sum value of the second loss function value and the third loss function value";(Chen'200, Para. 37, teaches adding a contour accuracy loss to the segmentation loss, i.e., obtaining a first sum value of two loss functions);
"obtaining a product value of a target constant and the first sum value";(Chen'200, Para. 6, teaches multiplying the added loss functions by a constant, i.e., obtaining a product value of a target constant and first sum).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Ma in view of Ye, Yuan, Chen, Han, Yu, Gollanapalli, Ren, Liu’952, and Wang wherein the loss functions are the second and third loss functions by including the sum of loss functions multiplied by a constant taught by Chen’200. One of ordinary skill in the art would be motivated to combine the references since it improves accuracy of predicted masks (Chen'200, Para. 5, teaches the motivation of combination to be to improve the accuracy of the predicted masks).
However, the combination of references of Ma in view of Ye, Yuan, Chen, Han, Yu, Gollanapalli, Ren, Liu’952, Wang, and Chen’200 does not explicitly teach "determining a second sum value of the first loss function value and the product value as a final loss function value".
In an analogous field of endeavor, Chaudhury teaches "determining a second sum value of the first loss function value and the product value as a final loss function value"; (Chaudhury, Claim 6, teaches determining a total loss as a sum of a loss of a common classifier and a result of multiplying a hyperparameter by a loss, i.e., sum of first loss function and product value).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Ma in view of Ye, Yuan, Chen, Han, Yu, Gollanapalli, Ren, Liu’952, Wang, and Chen’200 wherein the common classifier loss is the first loss function and the multiplying is the product value of a sum of losses and a constant by including the final loss determination as a sum of the loss and a product of a loss and a constant taught by Chaudhury. One of ordinary skill in the art would be motivated to combine the references since it increases robustness of the model (Chaudhury, Para. 29, teaches the motivation of combination to be to increase robustness of shared embeddings between modalities with less training data).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date.
Allowable Subject Matter
Claims 24 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. The following is the examiner’s stated reason for indication of allowable subject matter: none of the cited prior art references, alone or in combination, provides a motivation to teach the ordered combination of limitations recited in Claim 24. Regarding Claim 24, Liu’952 teaches "The method according to claim 1, further comprising: obtaining an adversarial example image of a sample image included in a training dataset";(Liu'952, Para. 82 and Claim 14, teaches the training dataset comprises the collection of digital images in which the adversarial attack input dataset for evaluation comprises a collection of digital images having one or more perturbations in accordance with one or more adversarial attack methodologies and wherein at least one adversarial version of one or more natural input datasets in the training dataset by introducing one or more perturbations into the natural input dataset, i.e., obtain an adversarial example image of a sample image in the training dataset).
In an analogous field of endeavor, Watanabe et al. (US 20220284695 A1) teaches "obtaining a feature vector of the sample image and a feature vector of the adversarial example image";(Watanabe'695, Para. 47, teaches reading the reference images, performs convolution by the parameter of the CNN as an input to the CNN, and extracts and acquires a feature vector wherein the obtained first similarity is a first similarity between the feature vector of the basic image and the feature vector of a similar image, i.e., obtain feature vectors of two images);
"determining a first loss function value corresponding to a cosine value of an angle between the feature vector of the sample image and the feature vector of the adversarial example image";(Watanabe'695, Para. 48 and Claim 11, teaches the loss function includes a first similarity between the feature vector of the basic image and the feature vector of a similar image, a first similarity between the feature vector of the basic image and the feature vector of a dissimilar image, and a margin wherein the first similarity is based on a cosine similarity between a first feature vector associated with the basic image and a second feature vector associated with a reference image, i.e., first loss function value corresponds to a cosine value of an angle between the feature vectors of the images being the cosine similarity between the vectors).
In an analogous field of endeavor, Risser (US 20180068463 A1), Para. 116, teaches the magnitude of feature vectors produced from the content image deep in the network can also be used as a scaling factor for the content loss itself as opposed to explicitly evaluating a second loss function corresponding to a difference between a feature modulus value of the sample image and a feature modulus value of the adversarial image. Therefore, Risser does not explicitly teach "determining a second loss function value corresponding to a difference between a feature modulus value of the sample image and a feature modulus value of the adversarial example image".
In an analogous field of endeavor, Wang teaches "and training the adversarial attack network based on the first loss function value and the second loss function value"; (Wang, Para. 29 and 151-153, teaches training a neural network based on first losses, second losses, and third losses, i.e., train network based on a first and second loss function value).
Therefore, none of the cited prior art references alone or in combination teach the ordered combination of limitations of "determining a second loss function value corresponding to a difference between a feature modulus value of the sample image and a feature modulus value of the adversarial example image" with the rest of the claim limitations.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANDREW STEVEN BUDISALICH whose telephone number is (703)756-5568. The examiner can normally be reached Monday - Friday 8:30am-5:00pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached on (571) 272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANDREW S BUDISALICH/Examiner, Art Unit 2662
/AMANDEEP SAINI/Supervisory Patent Examiner, Art Unit 2662
Prosecution Insights