DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) filed on 2/14/2024 was considered and placed on the file of record by the examiner.
Claim Objections
Claims 1, 11, and 16 are objected to because of the following informalities: “at downstream model.” Appropriate correction “at a downstream model” is required.
Drawings
The drawings are objected to because elements are not described in figures 1 and 2. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claim 7 is rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the enablement requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to enable one skilled in the art to which it pertains, or with which it is most nearly connected, to make and/or use the invention.
The specification fails to enable one skilled in the art to perform the calculation in claim 7.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 7 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The equation and each variable in claim 7 are not defined with a clear meaning.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-6, 8-18, 20 are rejected under 35 U.S.C. 103 as being unpatentable over Geng et al., “One-Step Diffusion Distillation via Deep Equilibrium Models” in view of Nie et al. “Diffusion Models for Adversarial Purification.”
Regarding claim 1, Geng teaches a computer-implemented method for attacking a neural network, comprising:
receiving an input data, wherein the input data includes at least an image and a corresponding ground truth label (see figure 1, section 2, section 4.1, Geng discusses receiving image data and ground-truth image);
adding a pre-determined amount of noise to the image to create a noisy image (see figure 1, section 2, Geng discusses injecting Gaussian noise into the image data to create a noise injected image);
denoising the noisy image utilizing a diffusion model that includes a deep equilibrium root solver configured to generate a denoised image (see figure 1, section 2, Geng discusses diffusion model for denoising the noise injected image by implementing a Deep equilibrium (DEQ) model that solves for this fixed point z⋆, using black-box root finding algorithms like Broyden’s method, or Anderson acceleration in the forward pass of the diffusion model);
determining a first gradient of the denoised image with respect to the input data including at least the image, wherein the first gradient is associated with the diffusion model (see figure 1, section 2, Geng discusses diffusion model calculating gradient values to create a denoised image),
utilizing the denoised image at downstream model of the neural network (see figure 1, section 3, Geng discusses a downstream decoder model for decoding the denoised image).
Geng does not expressly disclose utilizing the denoised image at downstream model of the neural network, outputting a predicated label associated with the denoised image; determining a loss utilizing with the predicted label and the ground truth label; determining a second gradient associated with the downstream model utilizing at least the loss; and outputting an aggregate gradient that represents an error of the neural network output utilizing the predicted label, wherein the aggregate gradient is calculated utilizing at least the first gradient and the second gradient.
However, Nie teaches utilizing the denoised image at downstream model of the neural network, outputting a predicated label associated with the denoised image (see figure 1, section 1, section 3, Nie discusses a downstream classifier that outputs an image label associated with a denoised image),
determining a loss utilizing with the predicted label and the ground truth label (see figure 1, section 3.1, section 3.2, section A.1., section B.5, Nie discusses the KL divergence loss and matching ground-truth value if the SDE solver has a small numerical error);
determining a second gradient associated with the downstream model utilizing at least the loss (see figure 1, section 3.1, section 3.2, section A.3, section B.5, Nie discusses gradient of backpropagating after the classifier model); and
outputting an aggregate gradient that represents an error of the neural network output utilizing the predicted label, wherein the aggregate gradient is calculated utilizing at least the first gradient and the second gradient (see figure 1, section 3.1, section 3.2, section A.1, section A.3, section B.5, Nie discusses calculating the gradients of the loss function based on the diffusion model and the classifier model).
Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Geng with Nie to derive at the invention of claim 1. The result would have been expected, routine, and predictable in order to perform diffusion network optimization.
The determination of obviousness is predicated upon the following: One skilled in the art would have been motivated to modify Geng in this manner in order to improve network optimization for a diffusion network by taking into account the diffusion gradient and classifier gradient to recover a proper clean image from noisy images. Furthermore, the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in this manner explained using known engineering design, interface and/or programming techniques, without changing a fundamental operating principle of Geng, while the teaching of Nie continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result of calculating backpropagation gradients from a downstream classifier model that cleans noise injected image from a diffusion model process. The Geng and Nie systems are diffusion networks with image denoise processes, therefore one of ordinary skill in the art would have reasonable expectation of success in the combination. It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Regarding claim 2, Geng teaches wherein the diffusion model includes a denoising diffusion implicit model (see figure 1, section 2, Geng discusses diffusion model calculating gradient values for the denoised image).
The same motivation of claim 1 is applied to claim 2. Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Geng with Nie to derive at the invention of claim 2. The result would have been expected, routine, and predictable in order to perform diffusion network optimization.
Regarding claim 3, Nie teaches wherein the first gradient is determined not utilizing back propagation through an iterative denoising process (see figure 1, Nie discusses the first gradient from the Diffusion model is not through back propagation).
The same motivation of claim 1 is applied to claim 3. Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Geng with Nie to derive at the invention of claim 3. The result would have been expected, routine, and predictable in order to perform diffusion network optimization.
Regarding claim 4, Nie teaches wherein denoising is not done sequentially via the diffusion model (see section 3, Nie discusses image purification via a diffusion model).
The same motivation of claim 1 is applied to claim 4. Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Geng with Nie to derive at the invention of claim 4. The result would have been expected, routine, and predictable in order to perform diffusion network optimization.
Regarding claim 5, Nie teaches wherein a damping factor is utilized in calculating the first gradient (see section B.4, Nie discusses a damping coefficient in the diffusion model).
The same motivation of claim 1 is applied to claim 5. Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Geng with Nie to derive at the invention of claim 5. The result would have been expected, routine, and predictable in order to perform diffusion network optimization.
Regarding claim 6, Nie teaches wherein convergence is improved in response to increasing the damping factor (see section B.4, Nie discusses a damping coefficient in the diffusion model).
The same motivation of claim 1 is applied to claim 6. Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Geng with Nie to derive at the invention of claim 6. The result would have been expected, routine, and predictable in order to perform diffusion network optimization.
Regarding claim 8, Nie teaches wherein the aggregate gradient is determined utilizing backpropagation (see figure 1, section 3.2, Nie discusses gradient calculated by backpropagation).
The same motivation of claim 1 is applied to claim 8. Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Geng with Nie to derive at the invention of claim 8. The result would have been expected, routine, and predictable in order to perform diffusion network optimization.
Regarding claim 9, Nie teaches wherein the downstream model of the neural network is a pretrained model (see figure 1, section 4, section 5.1, Nie discusses the downstream model is a trained classifier).
The same motivation of claim 1 is applied to claim 9. Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Geng with Nie to derive at the invention of claim 9. The result would have been expected, routine, and predictable in order to perform diffusion network optimization.
Regarding claim 10, Nie teaches wherein the input data includes at least an image and corresponding label (see figure 1, Nie discusses image labels).
The same motivation of claim 1 is applied to claim 10. Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Geng with Nie to derive at the invention of claim 10. The result would have been expected, routine, and predictable in order to perform diffusion network optimization.
Claim 11 is rejected as applied to claim 1 as pertaining to a corresponding system.
Regarding claim 12, Nie teaches wherein the image includes at least video data, picture data, or sound data (see figure 1, Nie discusses image data).
The same motivation of claim 1 is applied to claim 12. Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Geng with Nie to derive at the invention of claim 12. The result would have been expected, routine, and predictable in order to perform diffusion network optimization.
Regarding claim 13, Geng teaches wherein the deep equilibrium root solver utilizes Anderson acceleration to generate the denoised image (see figure 1, section 2, Geng discusses diffusion model for denoising the noise injected image by implementing a Deep equilibrium (DEQ) models with a root finding algorithm, Anderson acceleration, in the forward pass of the diffusion model).
The same motivation of claim 1 is applied to claim 13. Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Geng with Nie to derive at the invention of claim 13. The result would have been expected, routine, and predictable in order to perform diffusion network optimization.
Regarding claim 14, Nie teaches wherein the aggregate gradient is utilized to generate one or more adversarial examples at the neural network that maximize a loss function of the neural network (see figure 1, section 3.2, Nie discusses computing full gradients of a loss function regarding the adversarial images to optimize the network).
The same motivation of claim 1 is applied to claim 14. Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Geng with Nie to derive at the invention of claim 14. The result would have been expected, routine, and predictable in order to perform diffusion network optimization.
Regarding claim 15, Nie teaches wherein a damping factor is utilized in determining the first gradient (see section B.4, Nie discusses a damping coefficient in the diffusion model).
The same motivation of claim 1 is applied to claim 15. Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Geng with Nie to derive at the invention of claim 15. The result would have been expected, routine, and predictable in order to perform diffusion network optimization.
Claim 16 is rejected as applied to claim 1 as pertaining to a corresponding broader method.
Claim 17 is rejected as applied to claim 13 as pertaining to a corresponding method.
Claim 18 is rejected as applied to claim 14 as pertaining to a corresponding method.
Regarding claim 20, Nie teaches wherein denoising the noisy image includes utilizing a stochastic approach that adds extra noise at each denoising step (see section 3.1, section 4, Nie discusses inject Gaussian noise to each network layer for better robustness via a stochastic process).
The same motivation of claim 1 is applied to claim 20. Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Geng with Nie to derive at the invention of claim 20. The result would have been expected, routine, and predictable in order to perform diffusion network optimization.
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Geng et al., “One-Step Diffusion Distillation via Deep Equilibrium Models” in view of Nie et al. “Diffusion Models for Adversarial Purification” in view of Deutsch et al. (US 2025/0139057).
Regarding claim 19, Geng and Nie do not expressly disclose wherein the downstream model of the neural network is an untrained model. However, Deutsch teaches wherein the downstream model of the neural network is an untrained model (see figure 5, para. 0101-0102, Deutsch discusses an untrained model to provide feedback to a deep Adversarial Network model or Generative Network).
Motivation to combine may be gleaned from the prior art considered. It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the invention of Geng and Nie with Deutsch to derive at the invention of claim 19. The result would have been expected, routine, and predictable in order to perform diffusion network optimization.
The determination of obviousness is predicated upon the following: One skilled in the art would have been motivated to modify Geng and Nie this manner in order to improve network optimization for a diffusion network taking into account the diffusion gradient and classifier gradient to recover a proper clean image from noisy images. Furthermore, the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in this manner explained using known engineering design, interface and/or programming techniques, without changing a fundamental operating principle of Geng and Nie, while the teaching of Deutsch continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result of backpropagation gradients from a downstream classifier model that cleans noise injected image from a diffusion model process. The Geng, Nie, and Deutsch systems are diffusion network with image denoise processes, therefore one of ordinary skill in the art would have reasonable expectation of success in the combination. It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Cho et al. (US 2025/0078349) discusses reverse or the generative process is modeled as a denoising neural network trained to remove noise gradually at each step.
Ni et al., “Deep Equilibrium Multimodal Fusion” discusses computing gradients based on a loss function between a RootSolver and target, and using the Anderson acceleration as a solver.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KENNY A CESE whose telephone number is (571) 270-1896. The examiner can normally be reached on Monday – Friday, 9am – 4pm.
If attempts to reach the primary examiner by telephone are unsuccessful, the examiner’s supervisor, Gregory Morse can be reached on (571) 272-3838. The fax phone number for the organization where this application or proceeding is assigned is (571) 273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Kenny A Cese/
Primary Examiner, Art Unit 2663