Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 06/22/2026 has been entered.
Status of Claims
This action is responsive to the application filed on 06/22/2026.
Claims 1-8, 11-12, and 17-24 are pending.
Claims 1-3, 17-19, and 21-23 have been amended.
Claims 9-10 and 13-16 have been canceled.
Response to Arguments
Applicant’s arguments, with respect to the rejection(s) of claim(s) 1, 17, and 21 under 35 U.S.C. 103, have been considered but they are not persuasive. Applicant argues that no reference teaches the amended claim limitations now stating “wherein the mutual information and the reconstruction loss are regularization functions in the first and second objective functions”, since Chen does not consider “reconstruction loss”. The examiner respectfully disagrees.
Due to the broadness of the claim language, Chen has been found to teach the amended limitations. Chen, sections 3-4, section 5 last paragraph, 7.1, and equation 3 teach “information-regularized minmax game” (regularization function) computation with “variational regularization of mutual information” determination for training the generator to a sufficient level of generation and discrimination, and then iteratively training on the dataset and latent codes (spliced vectors).
See 35 U.S.C 103 section for full mapping of claim limitations necessitated by applicant amendments.
Applicant’s arguments, with respect to the rejection(s) of claim(s) 1, 17, and 21 under 35 U.S.C. 103, limitations stating “calculating a reconstruction loss based on differences between the dummy samples and the real samples, the reconstruction loss indicating a combined loss of the classifier, the feature decomposition, and the intermediate sample generation network”, have been considered but are moot because the arguments do not apply to the current combination of references being used in the current rejection.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-8, 11-12 and 17-24 are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al (“InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets”, 2016) hereinafter Chen, in view of Kamath et al (US Pub 20190213705) hereinafter Kamath, in view of Frid-Adar et al (“GAN-based synthetic medical image augmentation for increased CNN performance in liver lesion classification”, 2018) hereinafter Frid-Adar.
Regarding claims 1, 17, and 21, Chen teaches a sample generation method comprising; sample generation apparatus comprising: processing circuitry configured to; non-transitory computer-readable storage medium, storing instructions which when executed by at least one processor cause the at least one processor to perform a sample generation method comprising (section 8 teaches a computational cost of executing a GAN relating to computer system computation cost, known to include one or more processors communicatively coupled to one or memories for executing code to perform the embodiments of the disclosure):
obtaining real category feature vectors extracted from real samples respectively through a classifier and a feature decomposition (section 4 teaches vectors from vector decomposition for source and latent code of semantic features categorization comprising variable attributes of data extracted from “images from the MNIST dataset (real samples)” and );
combining the real category feature vectors with real category label vectors corresponding to the real samples to obtain spliced vectors (sections 4 and 7.1-7.2 teach the vectors of attributes are allocated to numerical identity as digits);
inputting the spliced vectors to an intermediate sample generation network, to obtain dummy samples through mapping, the dummy samples being a reconstruction of the real samples based on the spliced vectors (sections 3-4 and 7.1-7.2 teach “we provide the generator network with both the incompressible noise z and the latent code c, so the form of the generator becomes G(z, c)” for the input vector of noise and from the converted MNIST dataset vectors (real samples based on the spliced vectors); and outputs “samples from the generator distribution PG by transforming a noise variable z ~ Pnoise(z) into a sample G(z) (dummy samples through mapping… being a reconstruction of the real samples)”);
determining mutual information between the dummy samples and the corresponding spliced vectors (section 4 and equation 3 teach “minmax game” computation for “mutual information” determination);
calculating a reconstruction loss based on differences between the dummy samples and the real samples, (Chen, sections 5, 7.2, and Appendix B teach calculating an error value via a loss function between the generated sample and the real data sample values);
by processing circuitry of a computer device, inputting the real samples and the dummy samples to an intermediate sample discrimination network to construct a first objective function of the intermediate sample generation network and a second objective function of the intermediate sample discrimination network, performing iterative adversarial training of the intermediate sample generation network and the intermediate sample discrimination network with reference to the mutual information and the reconstruction loss, and iteratively maximizing the mutual information and minimizing the reconstruction loss to search for a minimum value of the first objective function and a maximum value of the second objective function during the adversarial training until an iteration stop condition is met (Chen, sections 3, 5, and Appendix B teach training “an adversarial discriminator network D that aims to distinguish between samples from the true data distribution Pdata (second objective) and the generator’s distribution PG” for fooling the discriminator (first objective) in order to finish the “minimax game with a variational regularization of mutual information and a hyperparameter” as an error function through the “min G” and “max D”, and attaining “maximal mutual information”. Section 7.1 and Fig. 1 teach the training procedure over multiple training iterations.); and
inputting the spliced vectors to a trained sample generation network in response to a determination that the iteration stop condition is met, to output a dummy sample set, wherein the mutual information and the reconstruction loss are regularization functions in the first and second objective functions (sections 3-4, section 5 last paragraph, 7.1, and equation 3 teach “information-regularized minmax game” (regularization function) computation with “variational regularization of mutual information” determination for training the generator to a sufficient level of generation and discrimination, and then iteratively training on the dataset and latent codes (spliced vectors)).
However, Chen does not explicitly teach the reconstruction loss indicating a combined loss of the classifier, the feature decomposition, and the intermediate sample generation network.
Kamath teaches the reconstruction loss indicating a combined loss of the classifier, the feature decomposition, and the intermediate sample generation network (paragraphs 0151-0155 teach “feature reconstruction loss” from a pretrained network (classifier) based on processed feature representation distances (feature decomposition) between the generated outputs and original images (intermediate sample generation network)).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to implement Kamath’s teachings of machine learning image transformation network reconstruction loss computations into Chen’s teaching of adversarial training utilizing image vectors and mutual information in order to encourage output image similarity to a target image with greater accuracy (Kamath, paragraphs 0151-0155).
Further, Chen at least implies sample generation method comprising; sample generation apparatus comprising: processing circuitry configured to; non-transitory computer-readable storage medium, storing instructions which when executed by at least one processor cause the at least one processor to perform a sample generation method comprising, and by processing circuitry of a computer device, inputting the real samples and the dummy samples to an intermediate sample discrimination network, performing iterative adversarial training of the intermediate sample generation network and the intermediate sample discrimination network with reference to the mutual information and the reconstruction loss, and iteratively maximizing the mutual information and minimizing the reconstruction loss during the adversarial training until an iteration stop condition is met (see mappings above); however Frid-Adar teaches sample generation method comprising; sample generation apparatus comprising: processing circuitry configured to; non-transitory computer-readable storage medium, storing instructions which when executed by at least one processor cause the at least one processor to perform a sample generation method comprising, and by processing circuitry of a computer device, inputting the real samples and the dummy samples to an intermediate sample discrimination network, performing iterative adversarial training of the intermediate sample generation network and the intermediate sample discrimination network with reference to the mutual information and the reconstruction loss, and iteratively maximizing the mutual information and minimizing the reconstruction loss during the adversarial training until an iteration stop condition is met (sections 3.2-3.3, 4.1, 4.2.1-4.2.2, and Fig. 4 teach a GPU performing the training of the disclosure, including “adversarial training” by inputting real and fake data (inputting the real samples and the dummy samples) of the “image space distribution” (mutual information) into a discriminator of a GAN network, and maximizing a minmax game iteratively over “70 epochs” (stop condition is met); wherein “Adversarial networks are trained by optimizing the following loss function of a two-player minimax game” in equation 1. Further GPUs are known to be communicatively coupled to one or more memories for executing instructions for performing embodiments of the disclosure.).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Chen’s teaching of adversarial training utilizing vectors and mutual information, as modified by Kamath’s teachings of machine learning image transformation network reconstruction loss computations, to include Frid-Adar’s teachings of Gan network adversarial training conditions with respect to image space distribution in order to improve “classification results” via the trained model (Frid-Adar, sections 3.2, 4.1, 4.2.1-4.2.2, and Fig. 4).
Regarding claims 2, 18, and 22, the combination of Chen, Kamath, and Frid-Adar teach all the claim limitations of claims 1, 17, and 21 above; and further teach obtaining real noise vectors in the real samples (Chen, section 4 teaches using the data extracted from the MINST dataset and obtaining “incompressible noise” vectors),
wherein the combining comprises:
combining the real category feature vectors with the corresponding real category label vectors and the real noise vectors; and the determining comprises:
determining mutual information between the dummy samples and corresponding real category feature combination vectors, the real category feature combination vectors being obtained by combining the real category feature vectors and the corresponding real category label vectors (Chen, sections 4, 7.1-7.2, and equation 3 teach “minmax game” computation for “mutual information” determination of the vectors of attributes that are allocated to numerical identities as digits).
Regarding claims 3, 19, and 23, the combination of Chen, Kamath, and Frid-Adar teach all the claim limitations of claims 2, 18, and 23 above; and further teach further comprising: extracting feature vectors from the real samples respectively (Chen, section 4 teaches determining feature from the input vector);
performing feature decomposition on the feature vectors, and sifting feature vectors for category discrimination, to obtain the real category feature vectors (Chen, section 4 teaches vector decomposition for source and latent code of semantic features); and
obtaining feature vectors that remain after the sifting, to obtain the real noise sectors (Chen, section 4 teaches vector decomposition for source noise (real) and latent code of “salient structured semantic features of the data distribution”).
Regarding claims 4, 20, and 24, the combination of Chen, Kamath, and Frid-Adar teach all the claim limitations of claims 3, 19, and 23 above; and further teach wherein the performing comprises: pre-assigning, for each real sample, corresponding category feature coefficients to feature sub-vectors in a feature vector of the real sample, to obtain category feature coefficient vectors (Chen, sections 4 and 7.1-7.2 teach the vectors of attributes are allocated to numerical identity as digits);
constructing an objective function of the category feature coefficient vectors according to a sum of squared residuals of products between a real category label vector of the real sample and the feature sub-vectors (Frid-Adar, sections 2.1, 3.2, and Fig. 4 teach determining the “unit variance over the entire mini-batch” during GAN training of the minmax game for real samples and class determinations and ROI extracted data);
iteratively adjusting the category feature coefficients of the feature sub-vectors in the category feature coefficient vectors to iteratively search for a minimum value of the objective function of the category feature coefficient vectors (Frid-Adar, sections 2.1, 3.2, and Fig. 4 teach “[t]he discriminator is trained to maximize D(x) for images with x ∼p data and to minimize D(x) for images with x ∼/∼ p data…[and] the generator is trained to maximize D(G(z)), or equivalently minimize 1 −D(G(z))” during training epochs); and
forming, in response to a determination that the iterative adjustment has stopped, the real category feature vector according to feature sub-vectors having category feature coefficients that are not 0 (Frid-Adar, sections 3.2, 4.1, 4.2.1-4.2.2, Fig. 4, and Table 1 teach training the GAN on input vectors according to minmax game results iteratively over “70 epochs”, and incorporating classes to generate labels for the samples, wherein “[f]or each class i ∈ {1 , . . . , k}”, thus the class not equal to 0).
Chen, Kamath, and Frid-Adar are combinable for the same rationale as set forth above with respect to claims 1, 17, and 21.
Regarding claims 5 and 16, the combination of Chen, Kamath, and Frid-Adar teach all the claim limitations of claims 2 and 13 above; and further teach further comprising: sampling from a first probability distribution according to a preset dummy category label vector, to obtain dummy category feature vectors, the first probability distribution being obtained by fitting the real category feature vectors (Frid-Adar, sections 2.1 and 3.2 teaches obtaining vectors from sample distribution of uniform probabilities with associated ROI extracted attributes to input into a generator);
combining the dummy category feature vectors and the corresponding dummy category label vector, to obtain dummy spliced vectors; and inputting the dummy spliced vectors to the intermediate sample generation network, to output the dummy samples (Frid-Adar, sections 2.1 and 3.2 teach inputting classes of the samples and the samples (spliced vectors) into the generator and the generator outputting combined fake samples),
wherein the determining comprises: determining mutual information between the dummy samples and the corresponding dummy spliced vectors (Frid-Adar, sections 3.2, 4.1, 4.2.1-4.2.2, and Fig. 4 teach acquiring fake data samples (dummy samples) of the input samples and classes (spliced vectors) within the “image space distribution” (mutual information)).
Chen, Kamath, and Frid-Adar are combinable for the same rationale as set forth above with respect to claims 1, 17, and 21.
Regarding claim 6, the combination of Chen, Kamath, and Frid-Adar teach all the claim limitations of claim 1 above; and further teach wherein the dummy spliced vectors further comprise dummy noise vectors (Chen, section 4 teaches the generator adding noise to the output vectors), and the method further comprises:
obtaining the real noise vectors in the real samples, the real noise vectors being different from the real category feature vectors (Chen, section 4 teaches using the data extracted from the MINST dataset and obtaining “incompressible noise” vectors);
fitting, the real noise vectors, to obtain a second probability distribution (Chen, section 3-4 teach the generator creating noisy samples as the “generator’s distribution PG” from the acquired noise data); and
sampling from the second probability distribution to obtain the dummy noise vectors (Chen, section 3-4 teach the “generator’s distribution PG” used for fooling the discriminator),
wherein the determining the mutual information between the dummy samples and the corresponding dummy spliced vectors comprises:
determining mutual information between the dummy samples and corresponding dummy category feature combination vectors, the dummy category feature combination vectors being obtained by combining the dummy category feature vectors and the dummy category label vectors (Chen, sections 3-4, 7.1-7.2, and equation 3 teach a discriminator and generator “minmax game” computation for “mutual information” determination; wherein vectors analyzed comprise variable attributes of data (feature) extracted from “images from the MNIST dataset” input to the generator to transform “noise variable z ~ Pnoise(z) into a sample G(z)” (dummy sample); and the vectors of attributes are allocated to numerical identity as digits (categories)).
Regarding claim 7, the combination of Chen, Kamath, and Frid-Adar teach all the claim limitations of claim 6 above; and further teach wherein the inputting the spliced vectors to the trained sample generation network in response to the determination that the iteration stop condition is met, to output the dummy sample set comprises:
using the real category feature vectors and the dummy category feature vectors as levels in a category feature vector factor and the real noise vectors and the dummy noise vectors as levels in a noise vector factor; performing cross-grouping on the levels in the category feature vector factor and on the levels in the noise vector factor, to obtain to-be-spliced combinations (Chen, sections 3-4, 7.1-7.2, and equation 3 teach obtaining dataset attributes and noise from the MINST dataset, and the generator creating samples for noisy attributes of the data, then calculating “matching each category in c1 to a digit type” (to-be-spliced combinations) and proceeds to the other categorical codes in c1, c2, c3, and c4 (to-be-spliced combinations));
obtaining a category label vector corresponding to a category feature vector comprised in each to-be-spliced combination (Chen, sections 3-4, 7.1-7.2 teach category vectors comprising variable attributes of data extracted from “images from the MNIST dataset” and “matching each category in c1 to a digit type”);
combining vectors comprised in each to-be-spliced combination with the corresponding category label vector (Chen, sections 4 and 7.1-7.2 teach the vectors of attributes are allocated to numerical identity as digits, and “matching each category in c1 to a digit type”); and
inputting vectors obtained by the combining the vectors in the same to-be-spliced combinations to the trained sample generation network in response to the determination that the iteration stop condition is met, to output final dummy samples (Chen, sections 3-4, 7.1, and equation 3 teach “minmax game” computation for training the generator to a sufficient level and then iteratively training on the dataset and latent codes (spliced vectors), after training the generator inputting the matched category and digit type samples for generating fake samples).
Regarding claim 8, the combination of Chen, Kamath, and Frid-Adar teach all the claim limitations of claim 7 above; and further teach wherein the to-be-spliced combinations comprise at least one of:
a to-be-spliced combination comprising a real noise vector, a real category feature vector, and a real label vector corresponding to the real category feature vector (Chen, sections 3-4, 7.1-7.2, and equation 3 teach obtaining dataset attributes (real category feature vector) and “incompressible noise z” (real noise vector) from the MINST dataset, and the generator creating samples for noisy attributes of the data, then calculating “matching each category in c1 to a digit type” (real label vector corresponding to the real category feature vector) and proceeds to the other categorical codes in c1, c2, c3, and c4 (to-be-spliced combinations));
a to-be-spliced combination comprising a dummy noise vector, a dummy category feature vector, and a dummy label vector corresponding to the dummy category feature vector;
a to-be-spliced combination comprising a dummy noise vector, a real category feature vector, and a real label sector corresponding to the real category feature vector; and
a to-be-spliced combination comprising a real noise sector. a dummy category feature vector, and a dummy label vector corresponding to the dummy category feature vector.
Regarding claim 11, the combination of Chen, Kamath, and Frid-Adar teach all the claim limitations of claim 1 above; and further teach wherein each real category feature vector includes a plurality of features used for representing real sample categories (Chen, section 4 teaches vectors comprising variable attributes (plurality of features) of data extracted from “images from the MNIST dataset”), and the method further includes:
selecting target features from the real category feature vectors (Chen, sections 4 and 7.1-7.2 teach the vectors of attributes are allocated to numerical identity as digits (target features));
evenly selecting feature values within a preset range for the target features, and maintaining feature values of non-target features of the real category feature vectors unchanged, to obtain a plurality of associated real category feature vectors (Chen, section 4 teaches “an MNIST digit is generated by an independent 1-of-10 (range) variable and two independent continuous variables (range)”); and
inputting the plurality of associated real category feature vectors to the trained sample generation network in response to the determination that the iteration stop condition is met to output associated dummy samples (Chen, sections 3-4 teaches “we provide the generator network with both the incompressible noise z and the latent code c, so the form of the generator becomes G(z,c)” for the input vector; and outputs “samples from the generator distribution PG by transforming a noise variable z ~ Pnoise(z) into a sample G(z) (dummy samples through mapping)”).
Regarding claim 12, the combination of Chen, Kamath, and Frid-Adar teach all the claim limitations of claim 1 above; and further teach wherein each real sample is a real image sample, each dummy sample is a dummy image sample, and each real category feature vector is a feature vector used for discriminating a category of the real image sample (Chen, sections 3-4, 7.1-7.2 teach vectors comprising variable attributes of data extracted from “images from the MNIST dataset” input to the generator to transform “noise variable z ~ Pnoise(z) into a sample G(z)” (dummy image sample); and the vectors of attributes are allocated to numerical identity as digits (categories)).
Chen, Kamath, and Frid-Adar are combinable for the same rationale as set forth above with respect to claim 1.
Prior Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Xu et al (US Pub 20040073764) teaches utilizing a GAN for image transformation and determining reconstruction loss between synthetic and base images.
Conclusion
14. Any inquiry concerning this communication or earlier communications from the examiner should be directed to CLINT MULLINAX whose telephone number is 571-272-3241. The examiner can normally be reached on Mon - Fri 8:00-4:30 PT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alexey Shmatov can be reached on 571-270-3428. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/C.M./Examiner, Art Unit 2123
/ALEXEY SHMATOV/Supervisory Patent Examiner, Art Unit 2123
Prosecution Insights