DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
computer readable storage medium note:
With respect to claim 19, ¶ 115 of the instant application recites, " A computer readable storage medium, as that term is used in the present disclosure, is not to be construed as storage in the form of transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide, light pulses passing through a fiber optic cable, electrical signals communicated through a wire, and/or other transmission media." Thus, claims 19 is an eligible subject matter.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The analysis of the claims’ subject matter eligibility will follow the 2019 Revised Patent Subject Matter Eligibility Guidance, 84 Fed. Reg. 50-57 (January 7, 2019) (“2019 PEG”).
With respect to claim 1.
Claim 1 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Is the claim to a process, machine, manufacture, or composition of matter? Yes—claim 1 recites a system, which is a machine.
Step 2A, prong one: Does the claim recite an abstract idea, law of nature or natural phenomenon? Yes—the limitations identified below each, under its broadest reasonable interpretation, covers mental processes abstract idea grouping (concepts performed in the human mind (including an observation, evaluation, judgment, opinion)), see MPEP 2106.04(a)(2), subsection III and the 2019 PEG or mathematical concept, but for the recitation of generic computer components:
“a generative component that generates an intermediate dataset comprising an inverse copula network, and a result component that utilizes the intermediate dataset as input for an inverse marginal cumulative distribution function network, resulting in a result dataset of objects, wherein the inverse marginal cumulative distribution function network was generated based on a model dataset of objects.”: (mathematical concept of modeling distribution and/ Mental processes- concept of observation and evaluation of modeling distribution).
Step 2A, prong two: Does the claim recite additional elements that integrate the judicial exception into a practical application? No—the judicial exception is not integrated into a practical application.
“a memory that stores computer executable components; and a processor, operably coupled to the memory, and that executes the computer executable components stored in the memory”: mere instructions to “apply it” because it only includes high level of generality description to apply the abstract idea. See MPEP § 2106.05(f). This judicial exception is not integrated into a practical application. In particular, the claim recites additional elements that amount to recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer, which do not integrate a judicial exception into a practical application. See MPEP 2106.05(f).
The generic computer components in these steps are recited at a high-level of generality (i.e., as a generic computer component performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception? No—there are no additional limitations beyond the mental processes identified above. The limitation treated above, are directed to the well-understood, routine, and conventional activity of storing and retrieving information in memory. See MPEP § 2106.05(d)(II); Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015). It also includes limitations that Merely reciting the words “apply it” (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f). The additional element is insignificant application, which is similar to examples of activities that the courts have found to be insignificant extra-solution activity, in accordance with MPEP 2106.05(g), Insignificant Extra-Solution Activity. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 2.
Step 1: A system, as above.
Step 2A Prong 1: The claim recites that “wherein generating the intermediate dataset comprises transforming independent uniform noise into correlated uniform noise”: This limitation merely recites additional mathematical concepts.
Step 2A Prong 2, Step 2B: This judicial exception is not integrated into a practical application. Mere recitation of generic computer components neither integrates the judicial exception into a practical application nor provides an inventive concept.
Claim 3.
Step 1: A system, as above.
Step 2A Prong 1: The claim recites that “wherein the inverse marginal cumulative distribution function network was generated based on applying a marginal cumulative distribution function to the model dataset, and wherein the result dataset was generated by applying an inverse of the marginal cumulative distribution function to the intermediate dataset.”: This limitation merely recites additional mathematical concepts.
Step 2A Prong 2, Step 2B: This judicial exception is not integrated into a practical application. Mere recitation of generic computer components neither integrates the judicial exception into a practical application nor provides an inventive concept.
Claim 4.
Step 1: A system, as above.
Step 2A Prong 1: The claim recites that “wherein the intermediate dataset corresponds to captured correlations between attributes of the model dataset.”: This limitation merely recites additional mathematical concepts.
Step 2A Prong 2, Step 2B: This judicial exception is not integrated into a practical application. Mere recitation of generic computer components neither integrates the judicial exception into a practical application nor provides an inventive concept.
Claim 5.
Step 1: A system, as above.
Step 2A Prong 1: The claim recites that “wherein the result dataset results from applying the inverse marginal cumulative distribution function network to the captured correlations to yield the result dataset that comprises a synthetic dataset that is similar to the model dataset.”: This limitation merely recites additional mathematical concepts.
Step 2A Prong 2, Step 2B: This judicial exception is not integrated into a practical application. Mere recitation of generic computer components neither integrates the judicial exception into a practical application nor provides an inventive concept.
Claim 6.
Step 1: A system, as above.
Step 2A Prong 1: The claim recites that “wherein the inverse marginal cumulative distribution function network corresponds to a uniform distribution of captured respective marginal distributions of the model dataset.”: This limitation merely recites additional mathematical concepts.
Step 2A Prong 2, Step 2B: This judicial exception is not integrated into a practical application. Mere recitation of generic computer components neither integrates the judicial exception into a practical application nor provides an inventive concept.
Claim 7.
Step 1: A system, as above.
Step 2A Prong 1: The claim recites that “wherein the result dataset comprises a synthetic dataset having a dependence structure and marginal distributions that are similar to the model dataset with a degree of similarity that exceeds a threshold.”: This limitation merely recites additional mathematical concepts.
Step 2A Prong 2, Step 2B: This judicial exception is not integrated into a practical application. Mere recitation of generic computer components neither integrates the judicial exception into a practical application nor provides an inventive concept.
Claim 8.
Step 1: A system, as above.
Step 2A Prong 1: The claim recites that “wherein the computer executable components further comprise a discriminator component that performs operations comprising: generating a discriminator network that classifies the result dataset in relation to the model dataset; and
backpropagating, depending on the classification of the result dataset, changes to the intermediate dataset and the discriminator network, to improve the classifying.”: This limitation merely recites additional mathematical optimization of GAN training.
Step 2A Prong 2, Step 2B: This judicial exception is not integrated into a practical application. Mere recitation of generic computer components neither integrates the judicial exception into a practical application nor provides an inventive concept.
Claim 9.
Step 1: A system, as above.
Step 2A Prong 1: The claim recites that “wherein backpropagating a change to the intermediate dataset comprises accessing the intermediate dataset via the inverse marginal cumulative distribution function network.”: This limitation merely recites additional mathematical optimization of GAN training.
Step 2A Prong 2, Step 2B: This judicial exception is not integrated into a practical application. Mere recitation of generic computer components neither integrates the judicial exception into a practical application nor provides an inventive concept.
Claim 10.
Step 1: A system, as above.
Step 2A Prong 1: The claim recites that “wherein the generative component and the discriminator component are comprised in a generative adversarial network.”: This limitation merely recites additional mathematical optimization of GAN training.
Step 2A Prong 2, Step 2B: This judicial exception is not integrated into a practical application. Mere recitation of generic computer components neither integrates the judicial exception into a practical application nor provides an inventive concept.
Claim 11.
Step 1: A system, as above.
Step 2A Prong 1: The claim recites that “wherein the generative adversarial network comprises a Wasserstein generative adversarial network.”: This limitation merely recites additional mathematical optimization of GAN training.
Step 2A Prong 2, Step 2B: This judicial exception is not integrated into a practical application. Mere recitation of generic computer components neither integrates the judicial exception into a practical application nor provides an inventive concept.
Claim 12.
Step 1: A system, as above.
Step 2A Prong 1: The claim recites that “wherein the generative adversarial network operates by a process that comprises utilizing gradient penalties.”: This limitation merely recites additional mathematical optimization of GAN training.
Step 2A Prong 2, Step 2B: This judicial exception is not integrated into a practical application. Mere recitation of generic computer components neither integrates the judicial exception into a practical application nor provides an inventive concept.
Claim 13.
Step 1: A system, as above.
Step 2A Prong 1: The claim recites that “an extrapolation component that modifies the inverse marginal cumulative distribution function network, resulting in a modified inverse marginal cumulative distribution function network, wherein the result component utilizes the intermediate dataset as input for the modified inverse marginal cumulative distribution function network, and wherein, based on the result dataset being generated by the modified inverse marginal cumulative distribution network, the result dataset comprises synthetic data generated by extrapolation beyond the model dataset of objects.”: This limitation merely recites additional mathematical concepts of extrapolation.
Step 2A Prong 2, Step 2B: This judicial exception is not integrated into a practical application. Mere recitation of generic computer components neither integrates the judicial exception into a practical application nor provides an inventive concept.
Claim 14.
Step 1: A system, as above.
Step 2A Prong 1: The claim recites that “wherein the extrapolation component modifies the inverse marginal cumulative distribution function network based on an association between a different dataset of the objects, and wherein the synthetic data results from extrapolation from the model dataset to the different dataset.”: This limitation merely recites additional mathematical concepts of extrapolation.
Step 2A Prong 2, Step 2B: This judicial exception is not integrated into a practical application. Mere recitation of generic computer components neither integrates the judicial exception into a practical application nor provides an inventive concept.
Claim 15.
Step 1: A system, as above.
Step 2A Prong 1: The claim recites that “herein the different dataset comprises a non-tail portion of the model dataset and the synthetic data comprises extrapolated data that describes a tail portion of the model dataset.”: This limitation merely recites additional mathematical concepts of extrapolation.
Step 2A Prong 2, Step 2B: This judicial exception is not integrated into a practical application. Mere recitation of generic computer components neither integrates the judicial exception into a practical application nor provides an inventive concept.
Claims 16-18
Step 1: The claims recite an method; therefore, they fall into the statutory category of machines.
Step 2A Prong 1: The claims recite the same mental processes as claims 1 and 4-5, respectively.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. As before, the mere recitation that the method is to be performed on a generic computer amounts to a mere instruction to apply the exception on the computer. See MPEP § 2106.05(f). With that exception, the analysis mirrors that of claims 1, and 4-5, respectively.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The analysis, with the one exception noted above, mirrors that of claims 1, 4-5, respectively.
With respect to claim 19.
Claim 19 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Is the claim to a process, machine, manufacture, or composition of matter? Yes—claim 19 recites a computer program product.
Step 2A, prong one: Does the claim recite an abstract idea, law of nature or natural phenomenon? Yes—the limitations identified below each, under its broadest reasonable interpretation, covers mental processes abstract idea grouping (concepts performed in the human mind (including an observation, evaluation, judgment, opinion)), see MPEP 2106.04(a)(2), subsection III and the 2019 PEG or mathematical concept, but for the recitation of generic computer components:
“generate an inverse copula network; utilize the inverse copula network as input for an inverse marginal cumulative distribution function network, resulting in the synthetic dataset of objects, wherein the inverse marginal cumulative distribution function network was generated based on a model dataset of objects; generate a discriminator network that classifies the synthetic dataset in relation to the model dataset; and to improve the classifying, backpropagating, depending on the classification of the result dataset, changes to the inverse copula network and the discriminator network;”: (mathematical concept of modeling distribution and/ Mental processes- concept of observation and evaluation of modeling distribution)
Step 2A, prong two: Does the claim recite additional elements that integrate the judicial exception into a practical application? No—the judicial exception is not integrated into a practical application.
“the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor”: mere instructions to “apply it” because it only includes high level of generality description to apply the abstract idea. See MPEP § 2106.05(f). This judicial exception is not integrated into a practical application. In particular, the claim recites additional elements that amount to recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer, which do not integrate a judicial exception into a practical application. See MPEP 2106.05(f).
The generic computer components in these steps are recited at a high-level of generality (i.e., as a generic computer component performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception? No—there are no additional limitations beyond the mental processes identified above. The limitation treated above, are directed to the well-understood, routine, and conventional activity of storing and retrieving information in memory. See MPEP § 2106.05(d)(II); Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015). It also includes limitations that Merely reciting the words “apply it” (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f). The additional element is insignificant application, which is similar to examples of activities that the courts have found to be insignificant extra-solution activity, in accordance with MPEP 2106.05(g), Insignificant Extra-Solution Activity. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 20.
Step 1: A system, as above.
Step 2A Prong 1: The claim recites that “wherein the inverse copula network corresponds to captured correlations between attributes of the model dataset”: This limitation merely recites additional mathematical concepts.
Step 2A Prong 2, Step 2B: This judicial exception is not integrated into a practical application. Mere recitation of generic computer components neither integrates the judicial exception into a practical application nor provides an inventive concept.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-7 and 16-18 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Wiese et al. (“Copula & Marginal Flows: Disentangling the Marginal from its Joint”, 7 Jul 2019, arXiv:1907.03361).
Regarding claim 1.
Wiese teaches a computer-implemented system comprising: a memory that stores computer executable components; and a processor, operably coupled to the memory, and that executes the computer executable components stored in the memory, wherein the computer executable components comprise: a generative component that generates an intermediate dataset comprising an inverse copula network (see abstract, “copula and marginal generative flows (CM flows) which allow for an exact modeling of the tail and any prior assumption on the CDF up to an approximation of the uniform distribution”, also see page 3, section 2.2, “the use of copula and marginal generative flows (CM f lows) in order to model exact tail beliefs whilst having a tractable log-likelihood. CM flows are a new model developed in this paper and are inspired by representing the joint distribution of (X1,X2) as a copula plus its marginals, also known as a pair-copula construction (PCC)… The marginal flow approximates the inverse CDFs F−1 g−1 θ,η m−1 X1 , F−1 X2 , whereas the copula flow approximates the generating function of C := (FX1 (X1),FX2 (X2)). Thus, a CM flow is given by gθ,η(u) = mθ ◦hη(u) for u ∈ [0,1]2 and the used transformations are depicted in Figure 3. Although we restrict ourselves to introducing the bivariate CM flow, the proposed flow can be generalized by using Vine copulas”, also see page 7, section 5.2, Copula Flows: Modeling the Joint Distribution, i.e. wherein copula generative flow == the inverse copula network),
and a result component that utilizes the intermediate dataset as input for an inverse marginal cumulative distribution function network, resulting in a result dataset of objects, wherein the inverse marginal cumulative distribution function network was generated based on a model dataset of objects (see page 3, section 2.2, “the use of copula and marginal generative flows (CM f lows) in order to model exact tail beliefs whilst having a tractable log-likelihood. CM flows are a new model developed in this paper and are inspired by representing the joint distribution of (X1,X2) as a copula plus its marginals, also known as a pair-copula construction (PCC)… The marginal flow approximates the inverse CDFs F−1 g−1 θ,η m−1 X1 , F−1 X2 , whereas the copula flow approximates the generating function of C := (FX1 (X1),FX2 (X2)). Thus, a CM flow is given by gθ,η(u) = mθ ◦hη(u) for u ∈ [0,1]2 and the used transformations are depicted in Figure 3. Although we restrict ourselves to introducing the bivariate CM flow, the proposed flow can be generalized by using Vine copulas”, also see page 7, section 5.2, Copula Flows: Modeling the Joint Distribution, i.e. wherein CDF == cumulative distribution function and also wherein The marginal flow approximates the inverse CDFs teaches the results component inverse marginal CDF cumulative distribution function, also wherein the generated samples transformed via marginal flow are data/objects).
Regarding claim 2.
Wiese teaches the computer-implemented system of claim 1,
Wiese further teaches wherein generating the intermediate dataset comprises transforming independent uniform noise into correlated uniform noise (see page 3, section 2.2, “the use of copula and marginal generative flows (CM f lows) in order to model exact tail beliefs whilst having a tractable log-likelihood. CM flows are a new model developed in this paper and are inspired by representing the joint distribution of (X1,X2) as a copula plus its marginals, also known as a pair-copula construction (PCC)… The marginal flow approximates the inverse CDFs F−1 g−1 θ,η m−1 X1 , F−1 X2 , whereas the copula flow approximates the generating function of C := (FX1 (X1),FX2 (X2)). Thus, a CM flow is given by gθ,η(u) = mθ ◦hη(u) for u ∈ [0,1]2 and the used transformations are depicted in Figure 3. Although we restrict ourselves to introducing the bivariate CM flow, the proposed flow can be generalized by using Vine copulas”, also see page 6-7 section 5).
Regarding claim 3.
Wiese teaches the computer-implemented system of claim 1,
Wiese further teaches wherein the inverse marginal cumulative distribution function network was generated based on applying a marginal cumulative distribution function to the model dataset, and wherein the result dataset was generated by applying an inverse of the marginal cumulative distribution function to the intermediate dataset (see page 3, section 2.2, “the use of copula and marginal generative flows (CM f lows) in order to model exact tail beliefs whilst having a tractable log-likelihood. CM flows are a new model developed in this paper and are inspired by representing the joint distribution of (X1,X2) as a copula plus its marginals, also known as a pair-copula construction (PCC)… The marginal flow approximates the inverse CDFs F−1 g−1 θ,η m−1 X1 , F−1 X2 , whereas the copula flow approximates the generating function of C := (FX1 (X1),FX2 (X2)). Thus, a CM flow is given by gθ,η(u) = mθ ◦hη(u) for u ∈ [0,1]2 and the used transformations are depicted in Figure 3. Although we restrict ourselves to introducing the bivariate CM flow, the proposed flow can be generalized by using Vine copulas”, also see page 6-7 section 5).
Regarding claim 4.
Wiese teaches the computer-implemented system of claim 1,
Wiese further teaches wherein the intermediate dataset corresponds to captured correlations between attributes of the model dataset (see page 3, section 2.2, “the use of copula and marginal generative flows (CM f lows) in order to model exact tail beliefs whilst having a tractable log-likelihood. CM flows are a new model developed in this paper and are inspired by representing the joint distribution of (X1,X2) as a copula plus its marginals, also known as a pair-copula construction (PCC)… The marginal flow approximates the inverse CDFs F−1 g−1 θ,η m−1 X1 , F−1 X2 , whereas the copula flow approximates the generating function of C := (FX1 (X1),FX2 (X2)). Thus, a CM flow is given by gθ,η(u) = mθ ◦hη(u) for u ∈ [0,1]2 and the used transformations are depicted in Figure 3. Although we restrict ourselves to introducing the bivariate CM flow, the proposed flow can be generalized by using Vine copulas”, i.e. joint dependencies are correlations, also see page 6-7 section 5).
Regarding claim 5.
Wiese teaches the computer-implemented system of claim 2,
Wiese further teaches wherein the result dataset results from applying the inverse marginal cumulative distribution function network to the captured correlations to yield the result dataset that comprises a synthetic dataset that is similar to the model dataset (see page 3, section 2.2, “the use of copula and marginal generative flows (CM f lows) in order to model exact tail beliefs whilst having a tractable log-likelihood. CM flows are a new model developed in this paper and are inspired by representing the joint distribution of (X1,X2) as a copula plus its marginals, also known as a pair-copula construction (PCC)… The marginal flow approximates the inverse CDFs F−1 g−1 θ,η m−1 X1 , F−1 X2 , whereas the copula flow approximates the generating function of C := (FX1 (X1),FX2 (X2)). Thus, a CM flow is given by gθ,η(u) = mθ ◦hη(u) for u ∈ [0,1]2 and the used transformations are depicted in Figure 3. Although we restrict ourselves to introducing the bivariate CM flow, the proposed flow can be generalized by using Vine copulas”, also see page 6-7 section 5).
Regarding claim 6.
Wiese teaches the computer-implemented system of claim 5,
Wiese further teaches wherein the inverse marginal cumulative distribution function network corresponds to a uniform distribution of captured respective marginal distributions of the model dataset (see page 3, section 2.2, “the use of copula and marginal generative flows (CM f lows) in order to model exact tail beliefs whilst having a tractable log-likelihood. CM flows are a new model developed in this paper and are inspired by representing the joint distribution of (X1,X2) as a copula plus its marginals, also known as a pair-copula construction (PCC)… The marginal flow approximates the inverse CDFs F−1 g−1 θ,η m−1 X1 , F−1 X2 , whereas the copula flow approximates the generating function of C := (FX1 (X1),FX2 (X2)). Thus, a CM flow is given by gθ,η(u) = mθ ◦hη(u) for u ∈ [0,1]2 and the used transformations are depicted in Figure 3. Although we restrict ourselves to introducing the bivariate CM flow, the proposed flow can be generalized by using Vine copulas”, also see page 6-7 section 5).
Regarding claim 7.
Wiese teaches the computer-implemented system of claim 1,
Wiese further teaches wherein the result dataset comprises a synthetic dataset having a dependence structure and marginal distributions that are similar to the model dataset with a degree of similarity that exceeds a threshold (see page 3, section 2.2, “the use of copula and marginal generative flows (CM f lows) in order to model exact tail beliefs whilst having a tractable log-likelihood. CM flows are a new model developed in this paper and are inspired by representing the joint distribution of (X1,X2) as a copula plus its marginals, also known as a pair-copula construction (PCC)… The marginal flow approximates the inverse CDFs F−1 g−1 θ,η m−1 X1 , F−1 X2 , whereas the copula flow approximates the generating function of C := (FX1 (X1),FX2 (X2)). Thus, a CM flow is given by gθ,η(u) = mθ ◦hη(u) for u ∈ [0,1]2 and the used transformations are depicted in Figure 3. Although we restrict ourselves to introducing the bivariate CM flow, the proposed flow can be generalized by using Vine copulas”, also see page 6-7 section 5).
Claims 16-18 recites a method to perform the system recited in claims 1 and 4-5. Therefore the rejection of claims 1 and 4-5 above applies equally here.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 8-10 and 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wiese et al. (“Copula & Marginal Flows: Disentangling the Marginal from its Joint”, 7 Jul 2019, arXiv:1907.03361) in view of Goodfellow et al. (“Generative Adversarial Nets”, Advances in Neural Information Processing Systems 27 (NIPS 2014)).
Regarding claim 8.
Wiese teaches the computer-implemented system of claim 1,
Wiese do not teach the limitations of claim 8.
Goodfellow teaches wherein the computer executable components further comprise a discriminator component that performs operations comprising: generating a discriminator network that classifies the result dataset in relation to the model dataset (see page 4,
PNG
media_image1.png
740
1068
media_image1.png
Greyscale
”); and backpropagating, depending on the classification of the result dataset, changes to the intermediate dataset and the discriminator network, to improve the classifying (see page 2, “This framework can yield specific training algorithms for many kinds of model and optimization algorithm. In this article, we explore the special case when the generative model generates samples by passing random noise through a multilayer perceptron, and the discriminative model is also a multilayer perceptron. We refer to this special case as adversarial nets. In this case, we can train both models using only the highly successful backpropagation and dropout algorithms [16] and sample from the generative model using only forward propagation.”, also see page 4, figure 1 description above and algorithm 1, showing updating gradients which is known that GAN uses backpropagation as described in page 2).
Both Wiese and Goodfellow pertain to the problem of generative model data transformation, thus being analogous. It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine Wiese and Goodfellow to teach the above limitations. The motivation for doing so would be “a generative model G that captures the data distribution, and a discriminative model D that estimates the probability that a sample came from the training data rather than G. The training procedure for G is to maximize the probability of D making a mistake. This framework corresponds to a minimax two-player game. In the space of arbitrary functions G and D, a unique solution exists, with G recovering the training data distribution and D equal to 1 2 everywhere. In the case where G and D are defined by multilayer perceptron, the entire system can be trained with backpropagation. There is no need for any Markov chains or unrolled approximate inference networks during either training or generation of samples. Experiments demonstrate the potential of the framework through qualitative and quantitative evaluation of the generated samples” (see Goodfellow abstract).
Regarding claim 9.
Wiese and Goodfellow teaches the computer-implemented system of claim 8,
Wiese further teaches see page 3, section 2.2, “the use of copula and marginal generative flows (CM f lows) in order to model exact tail beliefs whilst having a tractable log-likelihood. CM flows are a new model developed in this paper and are inspired by representing the joint distribution of (X1,X2) as a copula plus its marginals, also known as a pair-copula construction (PCC)… The marginal flow approximates the inverse CDFs F−1 g−1 θ,η m−1 X1 , F−1 X2 , whereas the copula flow approximates the generating function of C := (FX1 (X1),FX2 (X2)). Thus, a CM flow is given by gθ,η(u) = mθ ◦hη(u) for u ∈ [0,1]2 and the used transformations are depicted in Figure 3. Although we restrict ourselves to introducing the bivariate CM flow, the proposed flow can be generalized by using Vine copulas”, also see page 7, section 5.2, Copula Flows: Modeling the Joint Distribution, i.e. wherein CDF == cumulative distribution function and also wherein The marginal flow approximates the inverse CDFs teaches the results component inverse marginal CDF cumulative distribution function, also wherein the generated samples transformed via marginal flow are data/objects),
Goodfellow teaches the backpropagating a change to the intermediate dataset (see page 2, “This framework can yield specific training algorithms for many kinds of model and optimization algorithm. In this article, we explore the special case when the generative model generates samples by passing random noise through a multilayer perceptron, and the discriminative model is also a multilayer perceptron. We refer to this special case as adversarial nets. In this case, we can train both models using only the highly successful backpropagation and dropout algorithms [16] and sample from the generative model using only forward propagation.”, also see page 4, figure 1 description above and algorithm 1, showing updating gradients which is known that GAN uses backpropagation as described in page 2).
The motivation utilized in the combination of claim 8, super, applies equally as well to claim 9.
Regarding claim 10.
Wiese and Goodfellow teaches the computer-implemented system of claim 8,
Goodfellow further teaches wherein the generative component and the discriminator component are comprised in a generative adversarial network (see page 2, “This framework can yield specific training algorithms for many kinds of model and optimization algorithm. In this article, we explore the special case when the generative model generates samples by passing random noise through a multilayer perceptron, and the discriminative model is also a multilayer perceptron. We refer to this special case as adversarial nets. In this case, we can train both models using only the highly successful backpropagation and dropout algorithms [16] and sample from the generative model using only forward propagation.”, also see page 4, figure 1 description above and algorithm 1, showing updating gradients which is known that GAN uses backpropagation as described in page 2).
The motivation utilized in the combination of claim 8, super, applies equally as well to claim 10.
Regarding claim 19.
Wiese teaches a computer program product that generates a synthetic dataset of objects, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to: generate an inverse copula network (see abstract, “copula and marginal generative flows (CM flows) which allow for an exact modeling of the tail and any prior assumption on the CDF up to an approximation of the uniform distribution”, also see page 3, section 2.2, “the use of copula and marginal generative flows (CM f lows) in order to model exact tail beliefs whilst having a tractable log-likelihood. CM flows are a new model developed in this paper and are inspired by representing the joint distribution of (X1,X2) as a copula plus its marginals, also known as a pair-copula construction (PCC)… The marginal flow approximates the inverse CDFs F−1 g−1 θ,η m−1 X1 , F−1 X2 , whereas the copula flow approximates the generating function of C := (FX1 (X1),FX2 (X2)). Thus, a CM flow is given by gθ,η(u) = mθ ◦hη(u) for u ∈ [0,1]2 and the used transformations are depicted in Figure 3. Although we restrict ourselves to introducing the bivariate CM flow, the proposed flow can be generalized by using Vine copulas”, also see page 7, section 5.2, Copula Flows: Modeling the Joint Distribution, i.e. wherein copula generative flow == the inverse copula network);
utilize the inverse copula network as input for an inverse marginal cumulative distribution function network, resulting in the synthetic dataset of objects, wherein the inverse marginal cumulative distribution function network was generated based on a model dataset of objects (see page 3, section 2.2, “the use of copula and marginal generative flows (CM f lows) in order to model exact tail beliefs whilst having a tractable log-likelihood. CM flows are a new model developed in this paper and are inspired by representing the joint distribution of (X1,X2) as a copula plus its marginals, also known as a pair-copula construction (PCC)… The marginal flow approximates the inverse CDFs F−1 g−1 θ,η m−1 X1 , F−1 X2 , whereas the copula flow approximates the generating function of C := (FX1 (X1),FX2 (X2)). Thus, a CM flow is given by gθ,η(u) = mθ ◦hη(u) for u ∈ [0,1]2 and the used transformations are depicted in Figure 3. Although we restrict ourselves to introducing the bivariate CM flow, the proposed flow can be generalized by using Vine copulas”, also see page 7, section 5.2, Copula Flows: Modeling the Joint Distribution, i.e. wherein CDF == cumulative distribution function and also wherein The marginal flow approximates the inverse CDFs teaches the results component inverse marginal CDF cumulative distribution function, also wherein the generated samples transformed via marginal flow are data/objects);
Wiese do not teach generate a discriminator network that classifies the synthetic dataset in relation to the model dataset; and to improve the classifying, backpropagating, depending on the classification of the result dataset, changes to the inverse copula network and the discriminator network.
Goodfellow teaches generate a discriminator network that classifies the synthetic dataset in relation to the model dataset (see page 4,
PNG
media_image1.png
740
1068
media_image1.png
Greyscale
”); and to improve the classifying, backpropagating, depending on the classification of the result dataset, changes to the inverse copula network and the discriminator network (see page 2, “This framework can yield specific training algorithms for many kinds of model and optimization algorithm. In this article, we explore the special case when the generative model generates samples by passing random noise through a multilayer perceptron, and the discriminative model is also a multilayer perceptron. We refer to this special case as adversarial nets. In this case, we can train both models using only the highly successful backpropagation and dropout algorithms [16] and sample from the generative model using only forward propagation.”, also see page 4, figure 1 description above and algorithm 1, showing updating gradients which is known that GAN uses backpropagation as described in page 2).
Both Wiese and Goodfellow pertain to the problem of generative model data transformation, thus being analogous. It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine Wiese and Goodfellow to teach the above limitations. The motivation for doing so would be “a generative model G that captures the data distribution, and a discriminative model D that estimates the probability that a sample came from the training data rather than G. The training procedure for G is to maximize the probability of D making a mistake. This framework corresponds to a minimax two-player game. In the space of arbitrary functions G and D, a unique solution exists, with G recovering the training data distribution and D equal to 1 2 everywhere. In the case where G and D are defined by multilayer perceptron, the entire system can be trained with backpropagation. There is no need for any Markov chains or unrolled approximate inference networks during either training or generation of samples. Experiments demonstrate the potential of the framework through qualitative and quantitative evaluation of the generated samples” (see Goodfellow abstract).
Regarding claim 20.
Wiese and Goodfellow teaches the computer program product of claim 19,
Wiese further teaches wherein the inverse copula network corresponds to captured correlations between attributes of the model dataset (see page 3, section 2.2, “the use of copula and marginal generative flows (CM f lows) in order to model exact tail beliefs whilst having a tractable log-likelihood. CM flows are a new model developed in this paper and are inspired by representing the joint distribution of (X1,X2) as a copula plus its marginals, also known as a pair-copula construction (PCC)… The marginal flow approximates the inverse CDFs F−1 g−1 θ,η m−1 X1 , F−1 X2 , whereas the copula flow approximates the generating function of C := (FX1 (X1),FX2 (X2)). Thus, a CM flow is given by gθ,η(u) = mθ ◦hη(u) for u ∈ [0,1]2 and the used transformations are depicted in Figure 3. Although we restrict ourselves to introducing the bivariate CM flow, the proposed flow can be generalized by using Vine copulas”, i.e. joint dependencies are correlations, also see page 6-7 section 5).
Claim(s) 11-12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wiese et al. (“Copula & Marginal Flows: Disentangling the Marginal from its Joint”, 7 Jul 2019, arXiv:1907.03361) in view of Goodfellow et al. (“Generative Adversarial Nets”, Advances in Neural Information Processing Systems 27 (NIPS 2014)) in further in view of Gulrajani et al. (“Improved Training of Wasserstein GANs”, Advances in Neural Information Processing Systems 30 (NIPS 2017)).
Regarding claim 11.
Wiese and Goodfellow teaches the computer-implemented system of claim 10,
Wiese and Goodfellow do not teach limitation of claim 11.
Gulrajani further teaches wherein the generative adversarial network comprises a Wasserstein generative adversarial network (see page 1, “In particular, [1] provides an analysis of the convergence properties of the value function being optimized by GANs. Their proposed alternative, named Wasserstein GAN (WGAN) [2], leverages the Wasserstein distance to produce a value function which has better theoretical properties than the original. WGAN requires that the discriminator (called the critic in that work) must lie within the space of 1-Lipschitz functions, which the authors enforce through weight clipping.”).
Wiese, Goodfellow and Gulrajani pertain to the problem of generative model data transformation, thus being analogous. It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine Wiese, Goodfellow and Gulrajani to teach the above limitations. The motivation for doing so would be “The recently proposed Wasserstein GAN (WGAN) makes progress toward stable training of GANs, but sometimes can still generate only poor samples or fail to converge. We find that these problems are often due to the use of weight clipping in WGAN to enforce a Lipschitz constraint on the critic, which can lead to undesired behavior. We propose an alternative to clipping weights: penalize the norm of gradient of the critic with respect to its input. Our proposed method performs better than standard WGAN and enables stable training of a wide variety of GAN architectures with almost no hyperparameter tuning, including 101-layer ResNets and language models with continuous generators. We also achieve high quality generations on CIFAR-10 and LSUN bedrooms.” (see Gulrajani abstract).
Regarding claim 12.
Wiese, Goodfellow and Gulrajani teaches the computer-implemented system of claim 11,
Gulrajani further teaches wherein the generative adversarial network operates by a process that comprises utilizing gradient penalties (In particular, [1] provides an analysis of the convergence properties of the value function being optimized by GANs. Their proposed alternative, named Wasserstein GAN (WGAN) [2], leverages the Wasserstein distance to produce a value function which has better theoretical properties than the original. WGAN requires that the discriminator (called the critic in that work) must lie within the space of 1-Lipschitz functions, which the authors enforce through weight clipping. Our contributions are as follows: 1. On toy datasets, we demonstrate how critic weight clipping can lead to undesired behavior. 2. We propose gradient penalty (WGAN-GP), which does not suffer from the same problems. 3. We demonstrate stable training of varied GAN architectures, performance improvements over weight clipping, high-quality image generation, and a character-level GAN language model without any discrete sampling.).
The motivation utilized in the combination of claim 11, super, applies equally as well to claim 12.
Claim(s) 13-15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wiese et al. (“Copula & Marginal Flows: Disentangling the Marginal from its Joint”, 7 Jul 2019, arXiv:1907.03361) in view of Goodfellow et al. (“Generative Adversarial Nets”, Advances in Neural Information Processing Systems 27 (NIPS 2014)) in view of Krueger et al. (“Out-of-Distribution Generalization via Risk Extrapolation”, Proceedings of the 38th International Conference on Machine Learning, PMLR 139, 2021).
Regarding claim 13.
Wiese and Goodfellow teaches the computer-implemented system of claim 8,
Wiese teaches wherein the computer executable components further comprise: see page 3, section 2.2, “the use of copula and marginal generative flows (CM f lows) in order to model exact tail beliefs whilst having a tractable log-likelihood. CM flows are a new model developed in this paper and are inspired by representing the joint distribution of (X1,X2) as a copula plus its marginals, also known as a pair-copula construction (PCC)… The marginal flow approximates the inverse CDFs F−1 g−1 θ,η m−1 X1 , F−1 X2 , whereas the copula flow approximates the generating function of C := (FX1 (X1),FX2 (X2)). Thus, a CM flow is given by gθ,η(u) = mθ ◦hη(u) for u ∈ [0,1]2 and the used transformations are depicted in Figure 3. Although we restrict ourselves to introducing the bivariate CM flow, the proposed flow can be generalized by using Vine copulas”, also see page 6-7 section 5).
Wiese and Goodfellow do not teach an extrapolation component… the result dataset comprises synthetic data generated by extrapolation beyond the model dataset of objects.
Krueger teaches an extrapolation component… the result dataset comprises synthetic data generated by extrapolation beyond the model dataset of objects (see page 2, section 1, “REx optimizes for robustness to the forms of distributional shift that have been observed to have the largest impact on performance in training domains.”, also see section 2, “We consider multi-source domain generalization, where our goal is to find parameters θ that perform well on unseen do mains, given a set of m training domains, E = {e1,..,em}, sometimes also called environments. We assume the loss function, is fixed, and domains only differ in terms of their data distribution Pe(X,Y ) and dataset De.”, also page 5, section 3, “The principle of Risk Extrapolation (REx) has two aims: 1. Reducing training risks 2. Increasing similarity of training risks In general, these goals can be at odds with each other; decreasing the risk in the domain with the lowest risk also de creases the overall similarity of training risks.”).
Wiese, Goodfellow and Krueger pertain to the problem of neural network training, thus being analogous. It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine Wiese, Goodfellow and Krueger to teach the above limitations. The motivation for doing so would be “In particular, we show that reducing differences in risk across training domains can reduce a model’s sensitivity to a wide range of extreme distributional shifts, including the challenging setting where the input contains both causal and anti-causal elements. We motivate this approach, risk extrapolation (REx), as a form of robust optimization over a perturbation set of extrapolated domains (MM-REx), and propose a penalty on the variance of training risks (V-REx) as a simpler variant. We prove that variants of Rex can recover the causal mechanisms of the tar gets, while also providing robustness to changes in the input distribution (“covariate shift”). By trading-off robustness to causally induced distributional shifts and covariate shift, REx is able to outperform alternative methods such as Invariant Risk Minimization in situations where these types of shift co-occur.” (see Krueger abstract).
Regarding claim 14.
Wiese, Goodfellow and Krueger teaches the computer-implemented system of claim 13,
Krueger further teaches wherein the extrapolation component modifies the inverse marginal cumulative distribution function network based on an association between a different dataset of the objects, and wherein the synthetic data results from extrapolation from the model dataset to the different dataset (see page 2, “REx optimizes for robustness to the forms of distributional shift that have been observed to have the largest impact on performance in training domains…We consider multi-source domain generalization, where our goal is to find parameters θ that perform well on unseen do mains, given a set of m training domains, E = {e1,..,em}, sometimes also called environments. We assume the loss function, is fixed, and domains only differ in terms of their data distribution Pe(X,Y ) and dataset De. The risk function for a given domain/distribution e”, also page 5, “In the CMNIST training domains, the color of a digit is more predictive of the label than the shape is. But because the correlation between color and label is not invariant, predictors that use the color feature achieve different risk on different domains. By enforcing equality of risks, REx prevents the model from using the color feature enabling successful generalization to the test domain where the correlation between color and label is reversed.” ).
The motivation utilized in the combination of claim 13, super, applies equally as well to claim 14.
Regarding claim 15.
Wiese, Goodfellow and Krueger teaches the computer-implemented system of claim 14,
Wiese further teaches wherein the different dataset comprises a non-tail portion of the model dataset and the synthetic data comprises extrapolated data that describes a tail portion of the model dataset (see page 5,
PNG
media_image2.png
158
874
media_image2.png
Greyscale
).
Krueger teaches the extrapolation data (see page 2, section 1, “REx optimizes for robustness to the forms of distributional shift that have been observed to have the largest impact on performance in training domains.”, also see section 2, “We consider multi-source domain generalization, where our goal is to find parameters θ that perform well on unseen do mains, given a set of m training domains, E = {e1,..,em}, sometimes also called environments. We assume the loss function, is fixed, and domains only differ in terms of their data distribution Pe(X,Y ) and dataset De.”, also page 5, section 3, “The principle of Risk Extrapolation (REx) has two aims: 1. Reducing training risks 2. Increasing similarity of training risks In general, these goals can be at odds with each other; decreasing the risk in the domain with the lowest risk also de creases the overall similarity of training risks.”)
The motivation utilized in the combination of claim 13, super, applies equally as well to claim 15.
Related prior arts:
BESENBRUCH et al. (US 20230379469 A1) teaches If we know the marginal (factorised) distributions of the joint distribution and the correlation that we want, the task is possible. We simulate random correlated variables given the joint multivariate distribution. We transform them to correlated variables in [0,1] using the joint normal distribution's marginals' cumulative distributions. We re-transform these values in a joint hyperbolic distribution by using the inverse marginal cumulative distributions of the joint hyperbolic. FIG. 24 illustrates an example of this process.
Oktay et al. (US 20200364603 A1) teaches The model parameters (e.g., weights and biases) are represented in a “latent” or “reparameterization” space, amounting to a reparameterization. In some implementations, this space can be equipped with a learned probability model, which is used first to impose an entropy penalty on the parameter representation during training, and second to compress the representation using arithmetic coding after training. The proposed approach can thus maximize accuracy and model compressibility jointly, in an end-to-end fashion, with the rate-error trade-off specified by a hyperparameter.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to IMAD M KASSIM whose telephone number is (571)272-2958. The examiner can normally be reached 10:30AM-5:30PM, M-F (E.S.T.).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael J. Huntley can be reached at (303) 297 - 4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/IMAD KASSIM/Primary Examiner, Art Unit 2129
Prosecution Insights