Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s argument filed 02/19/2026 have been fully considered but they are not persuasive.
Applicant’s Argument: On pages 7-10 of Applicant’s response, applicant states that the claims do not recite limitations that incorporate mathematical concepts or constitute mental processes. Applicant argues that the claim limitation “computing one or more losses based on the first data point and the second data point” is merely based on a mathematical concept. Further, the amended claims do not recite mental process for the same reason given by the USPTO in Example 39. Applicant also notes that the limitation “generating a trained generative model based on the one or more losses, wherein the trained generative model includes the score-based generative model” is not address in the previous Office Action.
Examiner’s Response: Applicant’s argument is not persuasive. Under Subject Matter Eligibility Analysis Step 2A Prong 1, independent claim 4 recites an abstract idea as detailed in the analysis below. The claim limitation “computing one or more losses based on the first data point and the second data point” recites an abstract idea of a mathematical calculation because it is directed to calculating an output based on the input. The Specification (par. 82-83) describes one embodiment of training the model based on one or more loss. The direct calculation of the objective function is used to update the parameters of the model. Therefore, the claim limitation “computing one or more losses based on the first data point and the second data point “ recites a mathematical calculation.
In Example 39, the claim does not recite any of the judicial exceptions under Subject Matter Eligibility Analysis Step 2A Prong 1. Thus, Example 39 is not applicable to the claims of the invention.
In the previous Office Action (sent 11/28/2025; pg. 5), the claim limitation “generating a trained generative model based on the one or more losses, wherein the trained generative model includes the score-based generative model” is identified as an additional element rather than an abstract idea. Training a machine learning model is a generic computer process and the use of a computer or other machinery in its ordinary capacity does not integrate a judicial exception into a practical application or provide significantly more.
Applicant’s Argument: On pages 11-14 of Applicant’s response, applicant states that the claims are directed to a technological improvement of the training of a generative model using a score-based generative model to generate new data. Specifically, the improvement is recited as “converting a first data point included in a training dataset into a first set of values” and “performing one or more denoising operations” in claim 4.
Examiner’s Response: Applicant’s argument is not persuasive. During examination, the examiner should analyze the "improvements" consideration by evaluating the specification and the claims to ensure that a technical explanation of the asserted improvement is present in the specification, and that the claim reflects the asserted improvement (see MPEP §2106.05(a)). The MPEP (§2106.05(a)(II)) also warns, “it is important to keep in mind that an improvement in the abstract idea itself (e.g. a recited fundamental economic concept) is not an improvement in technology.” Here, the alleged improvement in the form of “converting a first data point included in a training dataset into a first set of values” and “performing one or more denoising operations” is an improvement to the abstract idea of a mental process that can be performed in the human mind. Additionally, the claim limitation “generating a trained generative model based on the one or more losses, wherein the trained generative model includes the score-based generative model” is a generic computer process and the use of a computer or other machinery in its ordinary capacity does not integrate a judicial exception into a practical application or provide significantly more.
An important consideration in determining whether a claim improves technology is the extent to which the claim covers a particular solution to a problem or a particular way to achieve a desired outcome, as opposed to merely claiming the idea of a solution or outcome (see MPEP 2106.05(a)). The amended claims do not provide sufficient details to describe any technological improvement. If the specifications explicitly set forth an improvement but in a conclusory manner (see MPEP 2106.04(d)(1): a bare assertion of an improvement without the detail necessary to be apparent to a person of ordinary skill in the art), the examiner should not determine the claim improves technology.
Applicant’s Argument: On pages 16-18 of Applicant’s response, applicant states that the combination of the cited references cannot teach or suggest each and every limitation of the amended claim 1.
Examiner’s Response: Applicant’s argument is not persuasive. Applicant’s arguments with respect to claim 1 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. The amendments to claim 1 change the scope of the invention. The combination of the cited references still teaches the claim as a whole.
In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986).
The claimed invention is rejected under 35 U.S.C. 103 as being unpatentable over Karatsiolis in view of Song. The claimed invention discloses an autoencoder with an embedded score-based generative model. Karatsiolis teaches the structure of the claimed invention because Karatsiolis discloses an autoencoder with an embedded classifier. Karatsiolis in combination of Song is used to teach the claim invention because the classifier in Karatsiolis can be replaces by the score-based generative model taught in Song.
The claim limitation in claim 4, “converting a training image included in a training dataset into a first set of values associated with a base distribution for a score-based generative model” is a multi-step process as detailed in dependent claim 5. The input pass is processed by the encoder neural network to generate a second set of latent variable values and the score-based generative model performs diffusion operations on the second set of latent variable values. Claim 4 also recites “performing one or more denoising operations via the score-based generative model on the first set of values to output, from the score-based generative model, a first set of latent variable values associated with a continuous latent space learned by an encoder separate from the score-based generative model”.
Karatsiolis in view of Song teaches this process. Karatsiolis (pg. 3, Fig, 2) discloses the input data corrupted with noise is encoded into a latent representation Z1 and the latent representation is passed onto the classifier. Karatsiolis (pg. 5, Section V, par. 1) discloses the classifier consists of multiple hidden layers that performs various operations such as activation functions and feature mapping. The score-based generative model from Song can be incorporated into the structure disclose by Karatsiolis to replace the classifier and the score-based generative model also performs multiple operations. The score-based generative model from Song (pg 3-4, Section 3.1 & 3.2) performs a diffusion process and a reverse diffusion process. Under the broadest reasonable interpretation, the term “denoising operations” constitutes a wide range of operations that can fall within the scope of the definition. The feature mapping performed by the classifier can be a denoising operation because feature mapping transforms noisy input data into representations that prioritize structural information while suppressing noise.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 4-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding Claim 4:
Subject Matter Eligibility Analysis Step 1:
Claim 4 recites “A computer-implemented method for training a generative model, the method comprising” and is thus a process, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
“converting a first data point included in a training dataset into a first set of values associated with a base distribution for a score-based generative model” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“performing one or more denoising operations based on the first set of values to output, a first set of latent variable values associated with a continuous latent space ” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“performing one or more additional operations to convert the first set of latent variable values into a second data point” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“computing one or more losses based on the first data point and the second data point” (a mathematical calculation; see par. 135 in Specification, computing a reconstruction loss)
Claim 4 therefore recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
“performing one or more denoising operations via the score-based generative model to output, from the score-based generative model,” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“generating a trained generative model based on the one or more losses, wherein the trained generative model includes the score-based generative model” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
The additional elements as disclosed above alone or in combination do not integrate the judicial exception into practical application as they are mere insignificant extra solution activity in combination of generic computer functions being implemented with generic computer elements in a high level of generality to perform the disclosed abstract idea above. Therefore, Claim 4 is directed to the abstract idea.
Subject Matter Eligibility Analysis Step 2B:
“performing one or more denoising operations via the score-based generative model to output, from the score-based generative model,” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“generating a trained generative model based on the one or more losses, wherein the trained generative model includes the score-based generative model” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
The additional elements as disclosed above alone or in combination do not recite significantly more than the abstract idea itself as they are mere insignificant extra solution activity in combination of generic computer functions being implemented with generic computer elements in a high level of generality to perform the disclosed abstract idea above. Therefore, Claim 4 is subject-matter ineligible.
Regarding Claim 14:
The claim recites an article of manufacture that performs the method as described in claim 4. Therefore, claim 14 is rejected for the same reasons as disclosed for claim 4. The limitations for additional elements of claim 14 are analyzed below.
Subject Matter Eligibility Analysis Step 2A Prong 1:
Please see Step 2A Prong 1 analysis of claim 4
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“One or more non-transitory computer readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claim 5:
Subject Matter Eligibility Analysis Step 2A Prong 1:
“” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. evaluation)
“performing one or more diffusion operations to convert the second set of latent variable values into the first set of values” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. evaluation)
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“performing one or more encoding operations via an encoder neural network to ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claim 6:
Subject Matter Eligibility Analysis Step 2A Prong 1: None
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“wherein performing the one or more additional operations comprises applying a decoder neural network to the first set of latent variable values to produce the second data point” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claims 7 and 17:
Subject Matter Eligibility Analysis Step 2A Prong 1:
“wherein computing the one or more losses comprises computing a cross-entropy loss associated with a first distribution of the first set of latent variable values ” (a mathematical calculation)
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claim 8:
Subject Matter Eligibility Analysis Step 2A Prong 1:
“wherein computing the cross-entropy loss comprises sampling from a proposal distribution associated with a loss weighting included in the cross-entropy loss” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. evaluation)
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B: None
Regarding Claim 9:
Subject Matter Eligibility Analysis Step 2A Prong 1: None
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“wherein the loss weighting comprises a diffusion coefficient associated with a diffusion process between the continuous latent space and the base distribution” (merely specifies a particular technological environment in which the abstract idea is to take place, ie. a field of use, and thus does not integrate the abstract idea into a practical application nor cannot provide significantly more than the abstract idea itself - see MPEP 2106.05(h))
Regarding Claim 10:
Subject Matter Eligibility Analysis Step 2A Prong 1: None
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“wherein the cross- entropy loss comprises at least one of a first loss weighting associated with the encoder neural network and a second loss weighting associated with the score-based generative model” (merely specifies a particular technological environment in which the abstract idea is to take place, ie. a field of use, and thus does not integrate the abstract idea into a practical application nor cannot provide significantly more than the abstract idea itself - see MPEP 2106.05(h))
Regarding Claim 11:
Subject Matter Eligibility Analysis Step 2A Prong 1:
“” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. evaluation)
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“wherein generating the trained generative model comprises” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claims 12 and 19:
Subject Matter Eligibility Analysis Step 2A Prong 1:
“computing a reconstruction loss associated with the first data point and the second data point” (a mathematical calculation)
“computing a negative encoder entropy loss associated with a second set of latent variable values generated by an encoder neural network based on the training dataset” (a mathematical calculation)
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“training dataset” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claim 13:
Subject Matter Eligibility Analysis Step 2A Prong 1:
“” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. evaluation)
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“wherein, in operation, the trained generative model converts” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claim 15:
Subject Matter Eligibility Analysis Step 2A Prong 1:
“” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. evaluation)
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“wherein the instructions further cause the one or more processors to perform the step of generating a pre-trained encoder neural network and a pre-trained decoder neural network included in the score-based generative model based on a standard Normal prior, wherein the pre-trained encoder neural network converts ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claim 16:
Subject Matter Eligibility Analysis Step 2A Prong 1: None
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“wherein generating the trained generative model comprises performing end-to-end training of the pre-trained encoder neural network, the pre-trained decoder neural network, and the score-based generative model based on the one or more losses” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claim 18:
Subject Matter Eligibility Analysis Step 2A Prong 1:
“wherein computing the cross-entropy loss comprises computing the cross-entropy loss based on a geometric variance associated with the one or more denoising operations” (a mathematical calculation)
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B: None
Regarding Claim 20:
Subject Matter Eligibility Analysis Step 2A Prong 1: None
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“wherein the score-based generative model comprises a set of residual network blocks” (merely specifies a particular technological environment in which the abstract idea is to take place, ie. a field of use, and thus does not integrate the abstract idea into a practical application nor cannot provide significantly more than the abstract idea itself - see MPEP 2106.05(h))
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Karatsiolis, "Conditional Generative Denoising Autoencoder" in view of Song, "Score-Based Generative Modeling Through Stochastic Differential Equations".
Regarding claim 1, Karatsiolis teaches:
“A computer-implemented method for training a generative model, the method comprising” (abstract, A generative denoising autoencoder model (generative model) is trained to produce better data samples from a set of images.)
“converting a training image included in a training dataset into a first set of values associated with a base distribution for a score-based” ([pg. 3, section III, par. 1-2; pg. 5, section V, par. 1; pg. 3, Figure 2], x denotes the training data of images and
x
~
denotes a data point from the training dataset that has been corrupted with noise.
x
~
is inputted into the encoder to produce the latent representation, which is sent to the classifier (scored-based model) to generate Z2. Figure 2 shows the conditional generative denoising autoencoder (generative model) and Z1 and Z2 are combined together to form the joint distribution for the model. The training dataset goes into the encoder and the output of the encoder is passed into the classifier. A classifier may consist of multiple hidden layers and each layer comprises of a plurality of neurons. The neurons apply an activation function on the input data. The intermediate features generated by the activation functions can represent the first set of values.)
“performing one or more denoising operations via the score-based on the first set of values to output, from the score-based generative model, a first set of latent variable values associated with a continuous latent space learned by an encoder separate from the score-based ” ([pg. 3, section III, par. 1-2; pg. 5, section IV, par. 4; pg. 3, Figure 2], The classifier (scored-based model) performs feature mapping (denoising operations) to generate Z2 (first set of latent variable values) that encapsulate discriminative information. Z2 is combined with Z1 to form the joint distribution Z. The input of the classifier is the output of the encoder, which converts the training data into latent space. The output of the classifier is latent representation. The classifier component and the encoder component are separate modules in the proposed framework.)
“performing one or more additional operations to convert the first set of latent variable values into an output image” ([pg. 5, section IV, par. 4; pg. 3, Figure 2], Z (first set of latent variable values) is input to the decoder and the output is X’, a generated image by the model that resembles the input image.)
“computing one or more losses based on the training image and the output image” ([pg. 4, col. 1, par. 2-4], The two objective functions are shown in page 4 of the reference. The unsupervised objective function calculates a loss between the input data, x1 and the output image from the decoder.)
“generating a trained generative model based on the one or more losses, wherein the trained generative model includes the score-based ” ([pg. 4, col. 1, par. 2-4, pg. 3, Figure 1], The unsupervised objective function is used to train the generative denoising autoencoder model. The generative denoising autoencoder model contains an autoencoder and an embedded classifier, which has the score-based learning.)
Karatsiolis does not explicitly disclose an implementation of a score-based generative model. However, Song discloses in the same field of endeavor:
“converting a training image included in a training dataset into a first set of values associated with a base distribution for a score-based generative model” ([pg. 3-4, section 3.1, par. 1-2; pg. 8-9, section 5, par. 3; pg. 4, Figure 2], The data distribution of the image training data is converted into the prior distribution which is the base distribution of a score-based generative model. In one application, the score-based generative model may be a time-dependent classifier.)
“performing one or more denoising operations via the score-based generative model on the first set of values to output, from the score-based generative model, a first set of latent variable values associated with a continuous latent space learned by ” ([pg. 3-4, section 3.1, par. 1-2; pg. 4, section 3.2, par. 1; pg. 4, Figure 2], The generative model performs a reverse- time SDE (denoising operations) on the data distribution. Data is mapped to an unstructured prior distribution with forward SDE and intermediate latent variables can be sampled from the prior distribution by reversing the SDE process. The score-based generative model performs the reverse diffusion process to generate an output.)
“generating a trained generative model based on the one or more losses, wherein the trained generative model includes the score-based generative model” ([pg. 4-5, section 3.3, par. 1-3], The score (one or more losses) is calculated and used to train the score-based generative model.)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of a score-based generative model from Song into the teaching of Karatsiolis. The score-based generative model from Song can replace the classifier from Karatsiolis. Doing so can allow for new sampling procedures and new modeling capabilities (Song, abstract).
Regarding claim 2, Karatsiolis teaches:
“wherein the trained generative model further includes a decoder neural network that converts the first set of latent variable values into the output image.” ([pg. 5, section IV, par. 4; pg. 3, Figure 2], Z (first set of latent variable values) is input to the decoder and the output is X’, a generated image by the model that resembles the input image.)
Regarding claim 3, Karatsiolis teaches:
“wherein, in operation, the trained generative model converts a second set of values associated with the base distribution into a second set of latent variable values in order to generate a new image that is not included in the training dataset.” ([pg. 5, section IV, par. 4; pg. 3, Figure 2], The flow of the input data through the structural components shown in Figure 2 is an iterative process. The generated output is modified and used as input to the generative model. The new input allows for a new Z2 (second set of values), Z (second set of latent variables), and next output (new image) to be generated.)
Regarding claim 4, Karatsiolis teaches:
“A computer-implemented method for training a generative model, the method comprising” (abstract, A generative denoising autoencoder model (generative model) is trained to produce better data samples from a set of images.)
“converting a first data point included in a training dataset into a first set of values associated with a base distribution for a score-based” ([pg. 3, section III, par. 1-2; pg. 5, section V, par. 1; pg. 3, Figure 2], x denotes the training data of images and
x
~
denotes a data point from the training dataset that has been corrupted with noise.
x
~
(first data point) is inputted into the encoder to produce the latent representation, which is sent to the classifier (scored-based model) to generate Z2. Figure 2 shows the conditional generative denoising autoencoder (generative model) and Z1 and Z2 are combined together to form the joint distribution for the model. The training dataset goes into the encoder and the output of the encoder is passed into the classifier. A classifier may consist of multiple hidden layers and each layer comprises of a plurality of neurons. The neurons apply an activation function on the input data. The intermediate features generated by the activation functions can represent the first set of values.)
“performing one or more denoising operations via the score-based based on the first set of values to output, from the score-based generative model, a first set of latent variable values associated with a continuous latent space learned by an encoder separate from the score-based ” ([pg. 3, section III, par. 1-2; pg. 5, section IV, par. 4; pg. 3, Figure 2], The classifier (scored-based model) performs feature mapping (denoising operations) to generate Z2 (first set of latent variable values) that encapsulate discriminative information. Z2 is combined with Z1 to form the joint distribution Z. The input of the classifier is the output of the encoder, which converts the training data into latent space. The output of the classifier is latent representation. The classifier component and the encoder component are separate modules in the proposed framework.)
“performing one or more additional operations to convert the first set of latent variable values into a second data point” ([pg. 5, section IV, par. 1-4; pg. 3, Figure 2], Z (first set of latent variable values) is input to the decoder and the output is X’t, a reconstructed outcome (second data point).)
“computing one or more losses based on the first data point and the second data point” ([pg. 4, col. 1, par. 2-4], The two objective functions are shown in page 4 of the reference. The unsupervised objective function calculates a loss between the input data, x1 and the output data from the decoder.)
“generating a trained generative model based on the one or more losses, wherein the trained generative model includes the score-based ” ([pg. 4, col. 1, par. 2-4], The unsupervised objective function is used to train the generative denoising autoencoder model. The generative denoising autoencoder model contains an autoencoder and an embedded classifier, which has the score-based learning.)
Karatsiolis does not explicitly disclose an implementation of a score-based generative model. However, Song discloses in the same field of endeavor:
“converting a first data point included in a training dataset into a first set of values associated with a base distribution for a score-based generative model” ([pg. 3-4, section 3.1, par. 1-2; pg. 8-9, section 5, par. 3; pg. 4, Figure 2], The data distribution of the image training data is converted into the prior distribution which is the base distribution of a score-based generative model. In one application, the score-based generative model may be a time-dependent classifier.)
“performing one or more denoising operations via the score-based generative model based on the first set of values to output, from the score-based generative model, a first set of latent variable values associated with a continuous latent space learned by ” ([pg. 3-4, section 3.1, par. 1-2; pg. 4, section 3.2, par. 1; pg. 4, Figure 2], The generative model performs a reverse- time SDE (denoising operations) on the data distribution. Data is mapped to an unstructured prior distribution with forward SDE and intermediate latent variables can be sampled from the prior distribution by reversing the SDE process. The score-based generative model performs the reverse diffusion process to generate an output.)
“generating a trained generative model based on the one or more losses, wherein the trained generative model includes the score-based generative model” ([pg. 4-5, section 3.3, par. 1-3], The score (one or more losses) is calculated and used to train the score-based generative model.)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of a score-based generative model from Song into the teaching of Karatsiolis. The score-based generative model from Song can replace the classifier from Karatsiolis. Doing so can allow for new sampling procedures and new modeling capabilities (Song, abstract).
Regarding claim 14:
Claim 14 recites an article of manuracture (“One or more non-transitory computer readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of”) that performs the same process as described in Claim 4. Therefore claim 14 is rejected under the same reasons mention for claim 4. The additional elements of claim 14 is addressed below by Karatsiolis:
“One or more non-transitory computer readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of” ([pg. 3, section III, par. 1; Figure 1], The generative model is shown in Figure 1. It is implied that the architecture is created on a computer system that consists of processors and memory for running the process described by the reference.)
Regarding claim 5, Karatsiolis teaches:
“performing one or more encoding operations via an encoder neural network to convert the first data point into a second set of latent variable values.” ([pg. 3, section III, par. 1-2; pg. 5, section V, par. 1; Figure 2], Z1 (second set of latent variable values) is generated by the encoder from the input data (first data point).)
“performing one or more ” ([pg. 3, section III, par. 1-2; pg. 5, section V, par. 1; Figure 2], The classifier receives the latent representation Z1 (second set of latent variable values) and outputs Z2 (first set of values).)
Karatsiolis does not explicitly disclose an implementation of “performing one or more diffusion operations to convert the second set of latent variable values into the first set of values”. However, Song discloses in the same field of endeavor:
“performing one or more diffusion operations to convert the second set of latent variable values into the first set of values” ([pg. 4, section 3.2, par. 1; Figure 2], The generative model performs a diffusion process on the latent representation x(T) to generate x(0) (first set of values).)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of “performing one or more diffusion operations to convert the second set of latent variable values into the first set of values” from Song into the teaching of Karatsiolis. Doing so can allow for new sampling procedures and new modeling capabilities (Song, abstract).
Regarding claim 6, Karatsiolis teaches:
“wherein performing the one or more additional operations comprises applying a decoder neural network to the first set of latent variable values to produce the second data point” ([pg. 5, section IV, par. 1-4; Figure 2], Z (first set of latent variable values) is input to the decoder and the output is X’t, a reconstructed outcome (second data point).)
Regarding claims 7 and 17, Karatsiolis teaches:
“wherein computing the one or more losses comprises computing a cross-entropy loss associated with a first distribution of the first set of latent variable values generated by the score-based The two objective functions are shown in page 4 of the reference. The supervised objective function computes a cross-entropy loss that is related to the classifier and the encoder. In the equation, Z1 and QΘ are related to the encoder and Pw is related to the classifier. The supervised objective function is a calculation of the loss that is associated with the classifier and encoder.)
Karatsiolis does not explicitly disclose an implementation of a score-based generative model. However, Song discloses in the same field of endeavor:
“wherein computing the one or more losses comprises computing a cross-entropy loss associated with a first distribution of the first set of latent variable values generated by the score-based generative model The score (one or more losses) is calculated using Equation 7. The score-based generative model can be substitute for the classifier)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of a score-based generative model from Song into the teaching of Karatsiolis. Doing so can allow for new sampling procedures and new modeling capabilities (Song, abstract).
Regarding claim 8, Karatsiolis teaches:
“wherein computing the cross-entropy loss comprises sampling from a proposal distribution associated with In determining the learning gradient information, the cross-entropy loss is calculated with respect to the sampling of an input data from a uniform distribution.)
Karatsiolis does not explicitly disclose an implementation of “sampling from a proposal distribution associated with a loss weighting included in the cross-entropy loss”. However, Song discloses in the same field of endeavor:
“wherein computing the cross-entropy loss comprises sampling from a proposal distribution associated with a loss weighting included in the cross-entropy loss” ([pg. 4-5, section 3.3, par. 1-3, Equation 7], Equation 7 shows the calculation of the cross-entropy loss function and λ is a loss weighting parameter.)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of a score-based generative model from Song into the teaching of Karatsiolis. Doing so can improve the performance of the generative model by incorporating architectural changes (Song, abstract).
Regarding claim 9, Karatsiolis does not explicitly disclose an implementation of “wherein the loss weighting comprises a diffusion coefficient associated with a diffusion process between the continuous latent space and the base distribution”. However, Song discloses in the same field of endeavor:
“wherein the loss weighting comprises a diffusion coefficient associated with a diffusion process between the continuous latent space and the base distribution” ([pg. 3-4, section 3.1, par. 1, Equation 5-7], Equation 5 shows the calculation relating the data distribution and the prior distribution. The function g is the diffusion coefficient of the data.)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of a score-based generative model from Song into the teaching of Karatsiolis. Doing so can improve the performance of the generative model by incorporating architectural changes (Song, abstract).
Regarding claim 10, Karatsiolis teaches:
“wherein the cross-entropy loss comprises at least one of a first loss weighting associated with the encoder neural network and a second loss weighting associated with the score-based The two objective functions are shown in page 4 of the reference. The supervised and unsupervised objective functions compute a cross-entropy loss that is related to the classifier and the encoder. The loss is based on backpropagating the gradient information back to the encoder and classifier to update their weights.)
Karatsiolis does not explicitly disclose an implementation of a score-based generative model. However, Song discloses in the same field of endeavor:
“wherein the cross-entropy loss comprises at least Equation 7 shows the calculation of the cross-entropy loss function and λ is a loss weighting parameter. A score-based generative model can be substitute for the classifier)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of a score-based generative model from Song into the teaching of Karatsiolis. Doing so can allow for new sampling procedures to generate image samples of higher quality (Song, abstract)
Regarding claim 11, Karatsiolis teaches:
“wherein generating the trained generative model comprises updating a plurality of parameters associated with the score-based The parameters of the objective function can be updated during the training procedure of the generative model. These parameters may include sample size and noise corruption.)
Karatsiolis does not explicitly disclose an implementation of a score-based generative model. However, Song discloses in the same field of endeavor:
“wherein generating the trained generative model comprises updating a plurality of parameters associated with the score-based generative model and the encoder neural network based on the cross- entropy loss” ([pg. 4-5, section 3.3, par. 1-3, Equation 7], The score of the score-based generative model is calculated using Equation 7. A score-based generative model can be substitute for the classifier.)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of a score-based generative model from Song into the teaching of Karatsiolis. Doing so can allow for new sampling procedures to generate image samples of higher quality (Song, abstract)
Regarding claims 12 and 19, Karatsiolis teaches:
“computing a reconstruction loss associated with the first data point and the second data point” ([pg. 4, col. 1, par. 2-4], The unsupervised objective function calculates a loss between the input data, x1 (first data point) and the output data (second data point) from the decoder.)
“computing a negative encoder entropy loss associated with a second set of latent variable values generated by an encoder neural network based on the training dataset” ([pg. 7, col. 1, par. 2], During a specific instance of the iterative process of the generation model, negative gradient information (negative encoder entropy loss) may be provided back into the model to generate a second set of Z1 latent representations to specifically remove certain image properties in the generated image.)
Regarding claim 13, Karatsiolis teaches:
“wherein, in operation, the trained generative model converts a second set of values associated with the base distribution into a second set of latent variable values in order to generate a new data point that is not included in the training dataset” ([pg. 5, section IV, par. 4; Figure 2 and 7], The flow of the input data through the structural components shown in Figure 2 is an iterative process. The generated output is modified and used as input to the generative model. The new input allows for a new Z2 (second set of values), Z (second set of latent variables), and next output (new image) to be generated. Figure 7 shows the iterative process of the generative model to start with an initial image and generate a series of new images until a target image is produced.)
Regarding claim 15, Karatsiolis teaches:
“wherein the instructions further cause the one or more processors to perform the step of generating a pre-trained encoder neural network and a pre-trained decoder neural network included in the Table I shows the process of training the generative denoising autoencoder. Step 4e uses the gradient information to update the encoder parameters and results in a pre-trained encoder. Step 4d uses the gradient information to update the decoder parameters and results in a pre-trained decoder. Step 4a and step 5a sends sample data to the encoder to convert to a latent representation and the procedure is repeated until convergence. Therefore, future iterations of the encoding process will generate a new set of latent variables. Step 4c passes the latent representation to the decoder to output new image data that differs from the initial data.)
Karatsiolis does not explicitly disclose an implementation of “score-based generative model based on a standard Normal prior”. However, Song discloses in the same field of endeavor:
“The data distribution of the image training data is converted into the prior distribution which is the base distribution of a score-based generative model.)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of “score-based generative model based on a standard Normal prior” from Song into the teaching of Karatsiolis. Doing so can allow for new sampling procedures to generate image samples of higher quality (Song, abstract)
Regarding claim 16, Karatsiolis teaches:
“wherein generating the trained generative model comprises performing end-to-end training of the pre-trained encoder neural network, the pre-trained decoder neural network, and the score-based Table I shows the process of training the encoder, decoder, and classifier. A gradient descent method (one or more losses) is used to train the 3 components of the system.)
Karatsiolis does not explicitly disclose an implementation of “score-based generative model”. However, Song discloses in the same field of endeavor:
“wherein generating the trained generative model comprises performing end-to-end training of The score based generative model generates new data from initial data by injecting noise into the data. The loss from the output and input is used to train the model to produce more accurate representations. The score-based generative model can replace the classifier in the Karatsiolis reference.)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of “score-based generative model” from Song into the teaching of Karatsiolis. Doing so can allow for new sampling procedures to generate image samples of higher quality (Song, abstract).
Regarding claim 18, Karatsiolis does not explicitly disclose an implementation of “wherein computing the cross-entropy loss comprises computing the cross-entropy loss based on a geometric variance associated with the one or more denoising operations”. However, Song discloses in the same field of endeavor:
“wherein computing the cross-entropy loss comprises computing the cross-entropy loss based on a geometric variance associated with the one or more denoising operations” ([pg. 15, Appendix C, par. 1], Appendix A-C provides details on how general SDEs can be applied to the framework discussed in the reference. An implementation where a geometric sequence relating the injecting noise into the data can be applied to the diffusion process.)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of “wherein computing the cross-entropy loss comprises computing the cross-entropy loss based on a geometric variance associated with the one or more denoising operations” from Song into the teaching of Karatsiolis. Doing so can improve the performance of the generative model by incorporating architectural changes (Song, abstract).
Regarding claim 20, Karatsiolis does not explicitly disclose an implementation of “wherein the score-based generative model comprises a set of residual network blocks”. However, Song discloses in the same field of endeavor:
“wherein the score-based generative model comprises a set of residual network blocks” ([pg. 26, Appendix H.2, par. 1-4], Appendix provides details on how improvements can be applied to the framework discussed in the reference. The score-based generative model has the residual blocks modified for performance improvements. The residual blocks are used for feature mapping in the process.)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of “wherein the score-based generative model comprises a set of residual network blocks” from Song into the teaching of Karatsiolis. Doing so can improve the performance of the generative model by incorporating architectural changes (Song, abstract).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GARY MAC whose telephone number is (703)756-1517. The examiner can normally be reached Monday - Friday 8:00 AM - 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abdullah Kawsar can be reached on (571) 270-3169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GARY MAC/Examiner, Art Unit 2127
/ABDULLAH AL KAWSAR/Supervisory Patent Examiner, Art Unit 2127
Prosecution Insights