Detailed Action
This action is responsive to the original filing on 09/20/2021 and the remarks and
amendment filed on 04/02/2025. Claims 14-26 are pending. Claims 14,25 and 26 are independent claims.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 14-26 rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding Claim 14:
Subject Matter Eligibility Analysis Step 1:
Claim 14 recites “A computer-implemented method for training a normalizing flow” and is thus a process, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 14 recites steps:
[D]etermining at least one training input signal (a mental process that can be done by choosing an input).
[D]etermining a training output signal for each of the at least one training input signal (a mental process that can be done by creating an output after receiving instructions from an input).
[D]etermining a first loss value, wherein the first loss value is based on a likelihood or a log-likelihood of the at least one determined training output signal with respect to a predefined probability distribution (a mathematical concept that determines a value based on a probability distribution).
[D]etermining an approximation of a gradient of the trainable parameters of the first layer with respect to the first loss value, wherein the gradient is dependent on an inverse of a matrix of the trainable parameters and determining the approximation of the gradient is achieved by optimizing an approximation of the inverse (a mathematical concept of calculating a gradient approximation using the loss value and numeric parameters).
[U]pdating the trainable parameters of the first layer based on the approximation of the gradient (a mathematical process of recalculating the parameters based on the gradient approximation that was calculated).
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 14 further recites additional elements:
[T]he normalizing flow is configured to determine a first output signal characterizing a likelihood or a log-likelihood of an input signal. This limitation merely specifies a particular technological environment (normalizing flow) which the abstract idea is to take place, i.e., a field of use (See MPEP 2106.05(h)).
[T]he normalizing flow includes at least one first layer. This limitation merely further specifies a particular technological environment which the abstract idea is to take place, i.e., a field of use (See MPEP 2106.05(h)).
[T]he first layer includes trainable parameters and a layer input to the first layer is based on the input signal and the first output signal is based on a layer output of the first layer. This limitation merely further specifies a particular technological environment which the abstract idea is to take place, i.e., a field of use (See MPEP 2106.05(h)).
[U]sing the normalizing flow. This limitation merely further specifies a particular technological environment which the abstract idea is to take place, i.e., a field of use (See MPEP 2106.05(h)).
Subject Matter Eligibility Analysis Step 2B:
The claim does not include additional elements considered individually and in combination that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract ideas into practical application, the additional elements only specify technological environment to perform the method. Limiting the abstract idea to a particular field of use or technological environment cannot provide an inventive concept. The claim is not patent eligible.
Regarding Claim 15:
Subject Matter Eligibility Analysis Step 1:
A process as claimed by Claim 14.
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 15 recites:
[T]he approximation of the inverse is optimized (a mathematical concept of updating a parameter matrix contain numeric values)
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 15 further recites additional elements:
[B]ased on the at least one training output signal. This limitation merely specifies a particular technological environment (at least one training output) which the abstract idea is to take place, i.e., a field of use (See MPEP 2106.05(h)).
Subject Matter Eligibility Analysis Step 2B:
The claim does not include additional elements considered individually and in combination that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into practical application, the additional element only specifies technological environment to perform the method. Limiting the abstract idea to a particular field of use or technological environment cannot provide an inventive concept. The claim is not patent eligible.
Regarding Claim 16:
Subject Matter Eligibility Analysis Step 1:
A process as claimed by claim 14.
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 16 recites:
[T]he layer output is determined according to the formula
PNG
media_image1.png
29
191
media_image1.png
Greyscale
wherein z is the layer output of the first layer, σ is an invertible activation function of the first layer and hl is a result of a matrix multiplication of a matrix Wi comprising the trainable parameters of the first layer and the layer input zl-1 (a mathematical concept that recites a formula for the output of the normalizing flow).
[T]he approximation of the gradient of the first loss value with respect to the trainable parameters is determined according to the formula
PNG
media_image2.png
40
184
media_image2.png
Greyscale
wherein
PNG
media_image3.png
31
46
media_image3.png
Greyscale
is a partial derivative of the first loss value with respect to the result of the matrix multiplication, a superscript T denotes transposing a matrix or a vector, xi the training input signal and Rl is the approximation of the inverse of the matrix Wl (a mathematical concept that recites a formula for the gradient of the loss value of the normalizing flow).
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 16 further recites additional element:
[T]he first layer is a fully connected layer. This limitation merely specifies a particular technological environment (first layer) which the abstract idea is to take place i.e., a field of use (See MPEP 2106.05(h)).
Subject Matter Eligibility Analysis Step 2B:
The claim does not include additional elements considered individually and in combination that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into practical application, the additional element only specifies technological environment to perform the method. Limiting the abstract idea to a particular field of use or technological environment cannot provide an inventive concept. The Claim is not patent eligible.
Regarding Claim 17:
Subject Matter Eligibility Analysis Step 1:
A process as claimed by Claim 14.
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 17 recites:
R, is determined based on a second loss function
PNG
media_image4.png
29
203
media_image4.png
Greyscale
wherein ||∙|| is a norm (a mathematical concept that recites the loss formula as well as the concept of norm)
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 17 does not recite any further limitations that could be an additional element.
Subject Matter Eligibility Analysis Step 2B:
The claim does not include additional elements considered individually and in combination that are sufficient to amount to significantly more than the judicial exception.
Regarding Claim 18:
Subject Matter Eligibility Analysis Step 1:
A process as claimed by claim 14.
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 18 recites:
Rl is determined using an iterative optimization algorithm (a mathematical concept that recites how to calculate the variable).
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 18 further recites additional element:
[T]he iterative optimization algorithm being a gradient descent algorithm wherein only one optimization step is performed for determining Rl. This limitation merely specifies a particular technological environment (algorithm) which the abstract idea is to take place i.e., a field of use (See MPEP 2106.05(h)).
Subject Matter Eligibility Analysis Step 2B:
The claim does not include additional elements considered individually and in combination that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into practical application, the additional element only specifies technological environment to perform the method. Limiting the abstract idea to a particular field of use or technological environment cannot provide an inventive concept. The claim is not patent eligible.
Regarding Claim 19
Subject Matter Eligibility Analysis Step 1:
A process as claimed by claim 14.
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 19 recites:
[T]he layer output is determined according to the formula
PNG
media_image5.png
31
202
media_image5.png
Greyscale
wherein Zl is the layer output of the first layer, σ is an invertible activation function of the first layer, Hl is a result of a discrete convolution of a tensor Wl comprising the trainable parameters of the first layer and the layer input Zl-1 and * denotes a discrete convolution operation (a mathematical concept that recites a formula for the output of the normalizing flow).
[T]he gradient of the first loss value with respect to the trainable parameters is determined according to the formula
PNG
media_image6.png
81
284
media_image6.png
Greyscale
wherein
PNG
media_image3.png
31
46
media_image3.png
Greyscale
is a partial derivative of the first loss value with respect to the result of the
aH1 discrete convolution, xi is the training input signal, ʘ denotes an element-wise multiplication operation, ones_like is a function that takes a first tensor as input and returns a second tensor of the same shape as the first tensor, wherein the second tensor is filled with all ones, and Rl is a tensor characterizing an approximation of a third tensor, wherein convolving the third tensor with Hl yields Zl-1 , and flip is a function that determines a tensor for a transpose convolution (a mathematical concept that recites a formula for the gradient of the loss value of the normalizing flow).
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 19 further recites additional element:
[T]he first layer is a convolutional layer. This limitation merely specifies a particular technological environment (first layer) which the abstract idea is to take place i.e., a field of use (See MPEP 2106.05(h)).
Subject Matter Eligibility Analysis Step 2B:
The claim does not include additional elements considered individually and in combination that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into practical application, the additional elements only specify technological environment to perform the method. Limiting the abstract idea to a particular field of use or technological environment cannot provide an inventive concept. The claim is not patent eligible.
Regarding Claim 20:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 20 recites:
R, is determined based on a second loss function
PNG
media_image7.png
31
246
media_image7.png
Greyscale
wherein ||∙|| is a norm (a mathematical concept that recites the loss formula as well as the concept of norm)
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 20 does not recite any further limitations that could be an additional element.
Subject Matter Eligibility Analysis Step 2B:
The claim does not include additional elements considered individually and in combination that are sufficient to amount to significantly more than the judicial exception
Regarding Claim 21:
Subject Matter Eligibility Analysis Step 1:
A process as claimed by claim 14.
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 21 recites:
Rl is determined using an iterative optimization algorithm (a mathematical concept that recites how to calculate the variable).
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 21 further recites additional element:
[T]he iterative optimization algorithm being a gradient descent algorithm wherein only one optimization step is performed for determining Rl. This limitation merely specifies a particular technological environment (algorithm) which the abstract idea is to take place i.e., a field of use (See MPEP 2106.05(h)).
Subject Matter Eligibility Analysis Step 2B:
The claim does not include additional elements considered individually and in combination that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into practical application, the additional element only specifies technological environment to perform the method. Limiting the abstract idea to a particular field of use or technological environment cannot provide an inventive concept. The claim is not patent eligible.
Regarding Claim 22:
Subject Matter Eligibility Analysis Step 1:
A process as claimed by claim 14.
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 22 does not recite any new abstract ideas outside of the ideas of claim 14.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 22 further recites:
[A] device is operated in accordance with the output signal of the normalizing flow. This limitation recites using a device to perform the abstract ideas of Claim 14 e.g., “apply it on a device” (See MPEP 2106.05(f)).
Subject Matter Eligibility Analysis Step 2B:
The claim does not include additional elements considered individually and in combination that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into practical application, the additional element only amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is not patent eligible.
Regarding Claim 23:
Subject Matter Eligibility Analysis Step 1:
A process as claimed by claim 14.
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 23 does not recite any new abstract ideas outside of the ideas of claim 14.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 23 further recites additional elements:
[T]he normalizing flow is comprised in a classifier. This limitation merely specifies a particular technological environment (classifier) which the abstract idea is to take place i.e., a field of use (See MPEP 2106.05(h)).
[T]he classifier is configured to determine a second output signal characterizing a classification of the input signal. This limitation merely specifies a particular technological environment (classifier) which the abstract idea is to take place i.e., a field of use (See MPEP 2106.05(h)).
[T]he second output signal is determined based on the first output signal. This limitation merely specifies a particular technological environment (second output) which the abstract idea is to take place i.e., a field of use (See MPEP 2106.05(h)).
Subject Matter Eligibility Analysis Step 2B:
The claim does not include additional elements considered individually and in combination that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into practical application, the additional elements only specify technological environment to perform the method. Limiting the abstract idea to a particular field of use or technological environment cannot provide an inventive concept. The claim is not patent eligible.
Regarding Claim 24:
Subject Matter Eligibility Analysis Step 1:
A process as claimed by claim 14.
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 24 does not recite any new abstract ideas outside of the ideas of claim 14.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 24 further recites additional elements:
[T]he input signal characterizes an internal state of a device and/or an operation status of the device and/or a state of an environment of the device. This limitation merely specifies a particular technological environment (classifier) which the abstract idea is to take place i.e., a field of use (See MPEP 2106.05(h)).
[I]nformation comprised in the first output signal of the normalizing flow is
made available to a user of the device by means of a displaying device. This limitation recites insignificant extra-solution activity of data gathering (See MPEP 2106.05(g)).
Subject Matter Eligibility Analysis Step 2B:
The claim does not include additional elements considered individually and in combination that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into practical application, the additional elements only specify technological environment to perform the method. The additional element of information comprised in the first output signal of the normalizing flow is made available to a user of the device by means of a displaying device is well understood, routine, and conventional activity of “transmitting or receiving data over a network” (See MPEP 2106.05(d)). Limiting the abstract idea to a particular field of use or technological environment and well understood, routine and conventional activity cannot provide an inventive concept. The claim is not patent eligible.
Regarding Claims 25-26:
Claim 25 recites “A training system configured to train a normalizing flow” and Claim 26 recites “A non-transitory machine-readable storage medium on which is stored a computer program for training a normalizing flow” which fall under a machine and manufacture respectively and are both a category of the four statutory categories of patentable subject matter. However, Claims 25 and 26 both have the same limitations and additional elements as Claim 14 and follow the same analysis for steps 2A and 2B. Following the same analysis, Claims 25 and 26 are subject-matter ineligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 14, 15, 25, and 26 are rejected under 35 U.S.C. 103 as being unpatentable over Grover et al., "Flow-GAN: Combining Maximum Likelihood and Adversarial Learning in Generative Models" in view of Gresele et al., "Relative gradient optimization of the Jacobian term in unsupervised deep learning"
Regarding Claim 14, Grover et al. teaches
[A] computer-implemented method for training a normalizing flow (Grover et al., section 3.1 paragraph 1, “a Flow-GAN can be trained via maximum likelihood estimation”, Grover et al., section 3 paragraph 2, “A Flow-GAN consists of a pair of generator-discriminator networks with the generator specified as a normalizing flow”);
wherein the normalizing flow is configured to determine a first output signal characterizing a likelihood or a log-likelihood of an input signal (Grover et al., pg. 3, col. 2, par. 1 “Hence, we can evaluate and optimize for the log-likelihood assigned by the model to a data point” signifies an output characterizing a likelihood.);
wherein the normalizing flow includes at least one first layer, wherein the first layer includes (Grover et al., pg. 3, col. 2, par. 2, “Thereafter, we require the transformations between the various layers of the generator…” signifies the layer structure);
wherein the first layer includes trainable parameters (Grover et al., pg. 3, col. 1, par. 2, “The generator Gθ : Rk → Rd is a deterministic function differentiable with respect to the parameters θ” signifies trainable parameters through parameters θ);
a layer input to the first layer (Grover et al., pg. 3, col. 2, par. 1, “A normalizing flow model specifies a parametric transformation from a prior density p(z)”)
is based on the input signal (Grover et al., Appendix A, specifies inputs used);
the first output signal is based on a layer output of the first layer (Grover et al., pg. 3, col. 2, par. 1 “Hence, we can evaluate and optimize for the log-likelihood assigned by the model to a data point” signifies an output characterizing a likelihood. Grover et al., pg. 3, col. 2, par. 2, “Thereafter, we require the transformations between the various layers of the generator…” signifies the layer structure.);
the method comprising the following steps: determining at least one training input signal (Grover et al., Appendix A, specifies inputs and training datasets used);
determining a training output signal for each of the at least one training input signal using the normalizing flow (Grover et al., figure 2, shows output signal data and the negative log-likelihoods they characterize);
and determining a first loss value, wherein the first loss value is based on a likelihood or a log-likelihood of the at least one determined training output signal with respect to a predefined probability distribution (Grover et al., eq 7, shows an objective function that uses min-max and is objective can loss when it is minimized).
Grover et al. does not explicitly teach the following limitations however Gresele et al. does.
determining an approximation of a gradient of the trainable parameters of the first layer with respect to the first loss value (Gresele et al., section 2.2 paragraph 3 “The parameters of the network are randomly initialized and then learned by gradient based optimization with an objective function £, which is a scalar function of the final output of the network. At each learning step, updates for the weights are proportional to the partial derivative of the loss with respect to each weight” describes gradient is determined from objective function which could be a loss function);
wherein the gradient is dependent on an inverse of a matrix of the trainable parameters (Gresele et al., section 3 paragraph 4 and eq 10 “Therefore, the computation of each of the gradient relative to such term involves a matrix inversion” the equation shows calculations with respect to trainable parameters/weights noted as W and its inverse);
…determining the approximation of the gradient is achieved by optimizing an approximation of the inverse (Gresele et al., Appendix A section A1, describes a requirement for a matrix inversion but a strategy to employ to remove the complications of matrix inversion through relative gradients);
updating the trainable parameters of the first layer based on the approximation of the gradient (Gresele et al., Appendix A section A1, describes parameter updates that can be performed based on the optimized method using relative gradients).
Grover et al. and Gresele et al. are both analogous art to the present invention because both references are reasonably pertinent to the problem faced by the inventor to train a normalizing flow efficiently yet still providing accurate data transformations and mappings. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine training a normalizing flow using a training input to produce a training output and a loss value as taught by Grover et al. with matrix and gradient approximation and optimization of Gresele et al. The motivation to do so is to reduce the computational cost as suggested by Gresele et al. (Abstract, “Based on relative gradients, we exploit the matrix structure of neural network parameters to compute updates efficiently even in high-dimensional spaces; the computational cost of the training is quadratic in the input size, in contrast with the cubic scaling of the naive approaches. This allows fast training with objective functions involving the log determinant of the Jacobian without imposing constraints on its structure, in stark contrast to normalizing flows.”).
Regarding Claim 15, the claim recites the limitations of Claim 14 taught by Grover et al. in view of Gresele et al. and further teaches a limitation taught by Gresele et al.:
[T]he approximation of the inverse is optimized based on the at least one training input signal. (Gresele et al., Section 2.2 paragraph 1 “When an input pattern x is presented to the network, it produces a final output ZL and a series of intermediate outputs. By defining z 0 = x and Z£ = ge(x)” signifies the input signal, Gresele et al., eq 6, shows the gradient is proportional to Z which takes in x (the input), and Gresele et al., eq 14 and 15, shows relative gradients proportional to Z and with respect to weight matrix inversions).
Regarding Claims 25 and 26, the claims recite an analogous system and an analogous storage medium teaching the same limitations taught by Grover et al. in view of Gresele et al. and is thus rejected for reasons set forth in the rejection of Claim 14.
Claims 22 and 24 are rejected under 35 U.S.C. 103 as being unpatentable over Grover et al. in view of Gresele et al. in view of Djelouah et al., US PG PUB 2022/0005161.
Regarding Claim 22, the claim recites the limitations of Claim 14 taught by Grover et al. in view of Gresele et al. and further teaches a limitation taught by Djelouah et al.:
[W]herein a device is operated in accordance with the output signal of the normalizing flow. (Djelouah et al., Figure 4, shows a diagram of a process that a device goes through that renders enhanced images based on the output from a normalizing flow).
Grover et al., Gresele et al. and Djelouah et al. are all analogous art to the present invention because the references are all within the same field of normalizing flows. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine training a normalizing flow using a training input to produce a training output and a loss value with matrix and gradient approximation and optimization as taught by Grover et al. and Gresele et al. with a device operating based on the output of the normalizing flow of Djelouah et al. The motivation to do so is to apply a normalizing flow model to a more commonly used application or device and optimize computational cost of the device by optimizing the cost to run the normalizing flow model as taught by Djelouah (Paragraph 50, “Optimization in image space is difficult. Nevertheless, thanks to the normalizing flow based generative model provided by software code 110/210/310 a bijective mapping f from the image space to latent space has been learned. As a result, the optimization problem to be solved may be expressed” signifies that the image enhancing device previously used with a neural network like this had difficulty with optimization and can be mitigated using a normalizing flow).
Regarding Claim 24, the claim recites the limitations of Claim 14 taught by Grover et al. in view of Gresele et al. and further teaches a limitation taught by Djelouah et al.
[W]herein the input signal characterizes an internal state of a device and/or an operation status of the device and/or a state of an environment of the device, and wherein information comprised in the first output signal of the normalizing flow is made available to a user of the device by means of a displaying device (Djelouah et al., Figure 1, shows and input image [130] sent from the device to a network [108] to the neural network [140] to enhance and sent back to the network to the display).
Claims 23 is rejected under 35 U.S.C. 103 as being unpatentable over Grover et al. in view of Gresele et al. in view of Ardizonne et al. ("Training Normalizing Flows with the Information Bottleneck for Competitive Generative Classification")
Regard Claim 23, the claim recites the limitations of Claim 14 taught by Grover et al. in view of Gresele et al. and further teaches a limitation taught by Ardizzone et al.:
[W]herein the normalizing flow is comprised in a classifier (Ardizzone et al, Figure 1, shows an information bottleneck invertible neural network as a generative classifier, Section 4 Paragraph 1, “We construct our IB-INN by combining the design efforts of various works on INNs and normalizing flow”);
wherein the classifier is configured to determine a second output signal characterizing a classification of the input signal, wherein the second output signal is determined based on the first output signal (Ardizzone et al, Figure 4, shows the classification from sample inputs)
Grover et al., Gresele et al. and Ardizzone et al. are all analogous art to the present invention because the references are all within the same field of normalizing flows. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine training a normalizing flow using a training input to produce a training output and a loss value with matrix and gradient approximation and optimization as taught by Grover et al. and Gresele et al. with a classifier of Ardizzone et al. The motivation to do so is to still be able to generate data and provide accurate information and classifications of said data that still characterizes likelihoods in a semi-supervised machine learning environment as taught by Ardizzone (Abstract, “This model class offers advantages such as improved uncertainty quantification and out-of-distribution detection, but traditional generative classifier solutions suffer considerably in classification accuracy. We find the trade-off parameter in the IB controls a mix of generative capabilities and accuracy close to standard classifiers”).
Response to Arguments
Applicant's arguments filed 04/02/2025 have been fully considered but they are not persuasive.
The drawings objections have been withdrawn in light of the instant amendments.
The specification objections have been withdrawn in light of the instant amendments.
Applicant’s arguments, see remarks, pages 12-14, filed 04/02/2025, with respect to Claims 14-26 under 35 U.S.C 112(b) have been fully considered and are persuasive. The rejections of Claims 14-26 under 32 U.S.C 112(b) has been withdrawn.
Applicant’s arguments, see remarks, page 14 and terminal disclaimer, filed 04/02/2025, and see terminal disclaimer review decision, filed 04/06/2025, with respect to Claims 14, 22, 23, 25 , and 26 under non-statutory double patenting have been fully considered and are persuasive. The rejections of Claims 14, 22, 23, 25 , and 26 under non-statutory double patenting has been withdrawn.
Applicant's arguments regarding the rejections of Claims 14-26 under 35 U.S.C. 101 have been fully considered but they are not persuasive.
Applicant argues:
Page 14, “the Patent Office has misapplied Step 2A, Prong Two, of this framework by ignoring the various statements in the specification linking the claimed invention to improvements in the technology of neural networks.”
And Page 16, “This analysis fails to conform to the guidance of Section 2106.04(d)(1), as applied by the PTAB in its Park decision, because it does not assess whether the claims integrate the alleged judicial exception into a practical application by addressing the statements in the specification linking the claimed invention to the achievement of technological improvements in the field of machine leaning. For instance, [0013] of the published version of the specification explains that in "training an unrestricted normalizing flow...the computational complexity of training the normalizing flow is quadratic" and [t]he method advantageously achieves this by efficiently approximating the matrix inversions necessary during training of the normalizing flow." More to the point, according to [0014], "training the normalizing flow this way leads to not having to restrict the normalizing flow with respect to the weight layers, which leads to a more powerful mapping function and hence improves the ability of the normalizing flow to accurately determine a likelihood or log-likelihood." The claimed invention mirrors this improvement because it recites "a likelihood or a log-likelihood" as well as "an inverse of a matrix of the trainable parameters," and as the above blurbs quoted from the specification explain, the matrix inversion improves the ability to determine a likelihood or log-likelihood.
The Examiner respectfully disagrees:
MPEP 2106.04 describes whether a Claim is directed to a judicial exception and describes Subject Matter Eligibility Step 2A which is used to determine an exception. Step 2A is a 2-prong analysis with prong 1 determining abstract ideas and prong 2 determining if additional elements integrate the judicial exception. The applicant merely recites improvements of the abstract ideas of the claims from the specification in their remarks but does not describe how the remaining, additional elements integrate the judicial exception and does not show how the additional elements shows the improvements. The additional elements of the claims merely recites the technological environment in which the abstract ideas are to take place which is not significantly more to be a judicial exception.
Applicant's arguments regarding the rejections of Claims 14-15, and 25-26 under 35 U.S.C. 103 have been fully considered but they are not persuasive.
Applicant argues:
Page 17 “Grover does not disclose the features "determining an approximation of a gradient of the trainable parameters of the first layer with respect to the first loss value...determining the approximation of the gradient is achieved by optimizing an approximation of the inverse." However, contrary to the Office Action, a POSITA would not have agreed that Gresele overcomes this admitted deficiencies in Grover. In particular, as stated in Gresele, Sec. 4, par. 4, determining a gradient for an unrestricted normalizing flow involves computing the inverse of a matrix which leads to cubical computational complexity. Gresele circumvents the problem by using the relative gradient instead of the normal gradient (where normal gradient refers to the partial derivatives of the parameters with respect to the loss function). This leads to the inverse cancelling out from the computation of the relative gradient.”
Examiner respectfully disagrees:
Applicant seems to be arguing that Gresele et al. does not teach the deficiencies of Grover et al. as it teaches a method to use relative gradients to not use the inverse at all. Gresele et al. also teaches a method of determining a gradient approximation, in section 2.2 par. 3, using an inverse matrix in section 3 par. 4 and optimizes the matrix inversion shown in Appendix A. Gresele et al. teaches a method that does use inverse matrices to determine gradients using relative gradients to optimize the matrix inversion. Gresele et al. teaches the methods and proceeds to teach methods not using an matrix inversion./A.T.N./
Applicant's arguments regarding the rejections of Claims 22-23 under 35 U.S.C. 103 have been fully considered but they are not persuasive.
Applicant argues:
Gresele et al. obviate the rejections
Examiner respectfully disagrees.
Gresele does not obviate the rejections as it is analogous art as described above.
Therefore, Examiner respectfully asserts that the cited arts teach sufficiently the limitations recited in the amended claims.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANDREW THOMAS NGUYEN whose telephone number is (571)272-2378. The examiner can normally be reached Monday - Thursday 7:45am-4:45pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Cesar Paula can be reached at (571)272-4128. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/A.T.N./Examiner, Art Unit 2145
/CESAR B PAULA/Supervisory Patent Examiner, Art Unit 2145
Prosecution Insights