DETAILED ACTION
This action is in response to the application filed on September 26, 2024. Claims 1-20 are pending and have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claims 1, 14, and 19 these claims recite the following limitations which are found to be abstract ideas not reciting a practical application or significantly more, with claim 1 being exemplary:
generating one or more output images using the trained diffusion model (abstract idea as a mental process as a human is capable of generating output images);
computing a gradient matrix for each image in the set of training images (mathematical concepts, mathematical relationships, mathematical formulas or equations, mathematical calculations);
calculating an importance score for the each image in the set of training images based on the computed gradient matrix, wherein the importance score indicates an importance of the set of training images to the one or more output image (mathematical concepts, mathematical relationships, mathematical formulas or equations, mathematical calculations);
This judicial exception is not integrated into a practical application for the following reasons. Claims 1, 14, and 19 all recite the additional element of “and identifying one or more of the training images as the influential training image when the importance score exceeds a predetermined threshold,” however, this limitation also recites an abstract idea as a mental process as a human is capable identifying one or more training images as the influential training image when the importance score exceeds a predetermined threshold.
Claim 14 further recites the additional element of “a processor; a memory in electronic communication with the processor; and instructions stored in the memory and executable by the processor to cause the system to,” and Claim 19 further recites the additional element of, “a computer-readable storage medium, on which a computer program is stored, wherein the computer program, when executed in a computer, causes the computer to perform operations for.” While these limitations include an additional elements of a processor; a memory; a computer-readable storage medium they are not sufficient to recite a practical application of the abstract ideas recited in claims 14 and 19 as they amount to mere generic computer elements and thus amount to no more than a recitation of the words “apply it” (or an equivalent) or are no more than the mere instructions to implement an abstract idea or other exception on a computer. See MPEP 2106.05(f).
Further, the claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because when considered separately and in combination, the above recited additional elements from claims 14 and 19 do not add significantly more (also known as an “inventive concept”) to the exception. Rather, the additional elements disclosed above perform well-understood, routine, conventional computer functions.
Therefore, independent claims 1, 14, and 19 are directed towards an abstract idea without a practical application or significantly more.
Regarding claims 4-13 the limitations are merely directed towards insignificant pre/post-solution extra activity that nonetheless do not integrate the abstract idea recited from claim 1 into a practical application.
Regarding claims 2-3, 15-18, and 20 the limitations are merely directed towards mathematical concepts, mathematical relationships, mathematical formulas or equations, mathematical calculations that nonetheless do not integrate the abstract idea recited from claim 1 into a practical application.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 4-8, 10-11, 14, and 18-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al, US 20250104399 in view of Dai et al, (Training Data Attribution for Diffusion Models, 2023) in view of Ungureanu et al, US 20240386707.
Regarding claim 1, Wang teaches a method of
identifying an influential training image in a diffusion model for image generation, comprising the steps of (see Wang, Paragraph [0021], “The present disclosure describes systems and methods for data processing. Embodiments of the disclosure include a data processing apparatus configured to analyze data attribution for text-to-image diffusion models by generating attribution scores for samples of a training dataset. The data processing apparatus initially tunes a pre-trained generative model toward an exemplar image (e.g., exemplar object or exemplar style) to generate synthesized images that are computationally influenced by the exemplar image by construction”):
training the diffusion model using a set of training images (see Wang, Paragraph [0021], “the data processing apparatus generates a custom “attribution dataset”, with pairs of training images and their influenced synthesized images”);
generating one or more output images using the trained diffusion model (see Wang, Paragraph [0021], “their influenced synthesized images,” synthesized images are considered to be one or more output images);
calculating an importance score for the each image in the set of training images (see Wang, Paragraph [0021], “an attribution component of the data processing apparatus computes attribution scores over the set of training images. An attribution score indicates a degree of influence for the training image on generating the synthesized image”);
and identifying one or more of the training images as the influential training image (see Wang, Paragraph [0050], “In some examples, the training image that has the highest attribution score is an exemplar image used to fine-tune the image generation model”).
Wang does not expressively teach
computing a gradient matrix for each image in the set of training images
calculating an importance score for the each image in the set of training images based on the computed gradient matrix,
However, Dai in a similar invention in the same field of endeavor teaches
computing a gradient matrix for each image in the set of training images (see Dai, pg. 7, 3.3 Jacobian approximation of influence correlates with true influence, “Next, we take our MNIST, CIFAR-10, and CelebA encoded ensembles and generate samples along with their counterfactuals and Jacobians,” and pgs. 5-6, “2.4 We can approximate the counterfactual by computing a Jacobian, “∂y(x,ε) ∂x is the Jacobian, which is an m-by-n matrix,” gradient and Jacobian matrix are both used to describe derivatives; for scalar functions, the vector of derivatives is a gradient vector, while for vector valued functions, the vector of derivatives is called a Jacobian matrix, therefore the gradient is just a special case of the Jacobian when the function is scalar-valued);
calculating an importance score for the each image in the set of training images based on the computed gradient matrix(see Dai, pg. 12, 3.3 Jacobian approximation of influence correlates with true influence, “As noted in Section 2.4, while the evaluation of the Jacobian can be computationally intensive, once computed we will be able to evaluate the approximate counterfactual with a matrix vector product. We therefore compute for each image in the training set their approximate influence on each generated sample (measured as the Euclidean distance from the approximate counterfactual to the original sample). We present the top 10 most influential training images on a random selection of generated samples in Figure 6.”),
The combination of Wang and Dai are analogous art because they are both in the same field of endeavor of [field of endeavor]. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to [limitations of claim 1 and 7] as taught in the system of Dai in the system of Wang to [rationale] (Dai, Paragraph []).
Wang in view of Dai does not expressively teach
and identifying one or more of the training images as the influential training image when the importance score exceeds a predetermined threshold
However, Ungureanu in a similar invention in the same field of endeavor teaches
and identifying one or more of the training images as the influential training image when the importance score exceeds a predetermined threshold (see Ungureanu, Paragraph [0022], “the bias system determines a global bias score for the generative machine learning model by determining bias scores for each of the additional visual attributes included in the dictionary. The bias system then compares the determined bias scores to a bias threshold. In an example, the bias threshold is an empirical threshold. For example, the bias system determines the global bias score based on a number of the determined bias scores which exceed the bias threshold”).
The combination of Wang, Dai, and Ungureanu are analogous art because they are all in the same field of endeavor of [field of endeavor]. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, to [limitations] as taught in the system of Ungureanu in the system of Wang in view of Dai to [rationale] (Ungureanu, Paragraph []).
Regarding claim 4, Wang in view of Dai in view of Ungureanu teaches the method of claim 1,
wherein the gradient matrix computation is performed at different training data checkpoints (see Dai, pg. 7, 3.3 Jacobian approximation of influence correlates with true influence, “Next, we take our MNIST, CIFAR-10, and CelebA encoded ensembles and generate samples along with their counterfactuals and Jacobians,” and pgs. 5-6, “2.4 We can approximate the counterfactual by computing a Jacobian, “∂y(x,ε) ∂x is the Jacobian, which is an m-by-n matrix,” gradient and Jacobian matrix are both used to describe derivatives; for scalar functions, the vector of derivatives is a gradient vector, while for vector valued functions, the vector of derivatives is called a Jacobian matrix, therefore the gradient is just a special case of the Jacobian when the function is scalar-valued; the gradient matrix can be performed for each image which is considered to be a different training data checkpoint).
The rationale of claim 1 has been applied herein.
Regarding claim 5, Wang in view of Dai in view of Ungureanu teaches the method of claim 1, further comprising:
identifying training data checkpoints that contributed most to the generated one or more output images (see Dai, pg. 12, 3.3 Jacobian approximation of influence correlates with true influence, “As noted in Section 2.4, while the evaluation of the Jacobian can be computationally intensive, once computed we will be able to evaluate the approximate counterfactual with a matrix vector product. We therefore compute for each image in the training set their approximate influence on each generated sample (measured as the Euclidean distance from the approximate counterfactual to the original sample). We present the top 10 most influential training images on a random selection of generated samples in Figure 6,” all training images are contributing to the output images and are evaluated based on their influence on each generated).
The rationale of claim 1 has been applied herein.
Regarding claim 6, Wang in view of Dai in view of Ungureanu teaches the method of claim 1,
wherein the gradient matrix computation is performed at a final checkpoint (see Dai, pg. 10, Fig. 4: Ensembles of diffusion models converge as ensemble size increases, “a) the final generated image gets increasingly closer to the image generated by the full ensemble. Distributions of Euclidean distances from the final image are given as violin plots, each violin representing 2752 samples generated from the encoded ensemble trained on MNIST,” the final image is considered to be the final checkpoint).
The rationale of claim 1 has been applied herein.
Regarding claim 7, Wang in view of Dai in view of Ungureanu teaches the method of claim 1,
wherein the diffusion model is a Denoising Diffusion Probabilistic Model (DDPM) or a Latent Diffusion Model (LDM) (see Dai, pg. 20, B.2. Model training, “We used the same model architecture as was used by Ho et al. [6]. The PyTorch implementation of the model was sourced from https://github.com/w86763777/pytorch-ddpm/blob/7abb4c0358db4f3f2b0a4609a9d726185b815e65/model.py,” the model used is a DDPM).
The rationale of claim 1 has been applied herein.
Regarding claim 8, Wang in view of Dai in view of Ungureanu teaches the method of claim 1, further comprising
ranking the set of training images based on their importance scores to identify most important images (see Wang, Paragraph [0028], “The attribution component generates a set of soft attribution scores and ranks the scores, apportioning “influence” over candidate training images”).
The rationale of claim 1 has been applied herein.
Regarding claim 10, Wang in view of Dai in view of Ungureanu teaches the method of claim 1,
wherein positive importance scores identify proponent images, and negative scores identify opponent images among the set of training images (see Wang, Paragraph [0148], “Contrastive learning refers to a type of machine learning in which a model is trained using the selection of positive and negative sample pairs. Contrastive learning can be used in either a supervised or unsupervised (e.g., self-supervised) training context. A loss function for a contrastive learning model can encourage a model to generate similar results for positive sample pairs, and dissimilar results for negative sample pair”).
The rationale of claim 1 has been applied herein.
Regarding claim 11, Wang in view of Dai in view of Ungureanu teaches the method of claim 1, further comprising:
displaying the identified influential image and/or its corresponding importance score (see Wang, Paragraph [0040], “Data processing apparatus 110 returns the attribution score associated with the selected training image to user 100,” and Paragraph [0042], “A user interface may enable user 100 to interact with user device 105. In some embodiments, the user interface may include an audio device, such as an external speaker system, an external display device such as a display screen, or an input device (e.g., a remote control device interfaced with the user interface directly or through an I/O controller module)”).
The rationale of claim 1 has been applied herein.
As per claim 14, Claim 14 claims a system of identifying an influential training image in a diffusion model for image generation, comprising: a processor; a memory in electronic communication with the processor; and instructions stored in the memory and executable by the processor to cause the system to: complete the same limitations as Claim 1. Therefore, the rejection and rationale are analogous to that made in Claim 1.
Wang further teaches a processor; a memory in electronic communication with the processor; and instructions stored in the memory and executable by the processor to cause the system to: (see Wang, Paragraph [0043], “Data processing apparatus 110 may also include a processor unit, a memory unit, an I/O module, and a training component”).
As per claim 18, Claim 18 claims the same limitations as Claim 11 and is dependent on a similarly rejected independent claim. Therefore, the rejection and rationale is analogous to that made in Claim 11.
As per claim 19, Claim 19 claims a computer-readable storage medium, on which a computer program is stored, wherein the computer program, when executed in a computer, causes the computer to perform operations for: completing the same limitations as Claim 1. Therefore, the rejection and rationale are analogous to that made in Claim 1.
Wang further teaches a computer-readable storage medium, on which a computer program is stored, wherein the computer program, when executed in a computer, causes the computer to perform operations for: (see Wang, Paragraph [0097], “Examples of memory unit 920 include solid state memory and a hard disk drive. In some examples, memory unit 920 is used to store computer-readable, computer-executable software including instructions that, when executed, cause a processor to perform various functions described herein”).
Allowable Subject Matter
Claim(s) 2-3, 9, 12-13, 15-17, and 20 are rejected under 35 U.S.C. 101 and objected to as being dependent upon a rejected base claim, but would be allowable the 35 U.S.C. 101 rejection was overcome and if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DOMINIQUE JAMES whose telephone number is (703)756-1655. The examiner can normally be reached 9:00 am - 6:00 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emily Terrell can be reached at (571)270-3717. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DOMINIQUE JAMES/Examiner, Art Unit 2666
/EMILY C TERRELL/Supervisory Patent Examiner, Art Unit 2666
Prosecution Insights