Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
The objections to the specification and claims are withdrawn based on the amendment filed on 4/22/2026
Applicant’s arguments in light of the claim amendments filed on 4/22/2026 with respect to the prior art rejections of claims 41-60 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 41-60 are rejected under 35 USC 103 as being unpatentable over BAGAN: Data Augmentation with Balancing GAN to Mariani et al. (hereinafter Mariani) in view of SMOTE: Synthetic Minority Over-sampling Technique to Chawla et al. (hereinafter Chawla).
Per claim 41, Mariani discloses A method performed by a computer hardware arrangement (Mariani: Section 5.2… implementing BAGAN methodology on computer hardware that executes the recited procedures of dataset preparation, generative-model training, dataset augmentation, and ResNet-18 classifier training, construed as a method performed by a computer hardware arrangement, "1) select this class as minority class, 2) generate an imbalanced dataset by dropping a percentage of images for this class from the training set, 3) train the considered generative models, 4) augment the imbalanced dataset to restore its balance by means of the generative models, 5) train a ResNet-18 classifier for the augmented dataset, and 6) measure the classifier accuracy for the minority class over the test set"), the method comprising:
training at least one model on at least one dataset including a plurality of data types (Mariani: Section 5…trains classifier and generative models on multi-class image datasets including MNIST (10 classes), CIFAR-10 (10 classes), Flowers (5 classes), and GTSRB (43 classes), and each distinct class label being a data type, "We validate the proposed methodology on a set of four datasets. We consider: MNIST, CIFAR-10, Flowers, and GTSRB"; Table 1…"Target datasets' information including resolution, number of classes, and per-class image distribution statistics for the training set");
determining at least one misclassification of one of the plurality of data types (Mariani: Section 5.2…trains a ResNet-18 classifier on the imbalanced dataset and measures per-class accuracy on the test set, thereby determining which class(es) the classifier misclassifies (i.e., the minority class(es) for which accuracy is low), "…5) train a ResNet-18 classifier for the augmented dataset, and 6) measure the classifier accuracy for the minority class over the test set"; Section 1…"The accuracy of image classification techniques can significantly deteriorate when the training dataset is imbalanced"; Section 4…the act of measuring per-class accuracy on the minority class intrinsically determines at least one misclassification under because per-class accuracy below 100% mathematically requires at least one misclassified class-c sample and Mariani's express disclosure of measuring minority-class accuracy and its express acknowledgment that imbalanced-dataset accuracy "can significantly deteriorate" together necessarily establish that at least one misclassification of the minority class is determined whenever Mariani's recited measurement step is performed);
assigning a classification score to each of the plurality of data types (Mariani: Section 4, GAN initialization…the discriminator's final dense layer applies a softmax activation that emits, for each input image, a per-class probability score, “The last layer of the discriminator Dd is a dense layer with a softmax activation function and generates the final discriminator output”; Section 4…the per-class softmax scores are construed as classification scores assigned to each data type, "To complete the discriminator, a final dense layer Dd with a softmax activation function translates the latent features into the probability that the image is fake or that it belongs to one of the problem classes c1 — cn");
receiving at least one synthetic dataset (Mariani: Section 4, GAN training and Section 5.2… training pipeline receives synthetic images output by the BAGAN generator and uses them to augment the original training set, "augment the imbalanced dataset to restore its balance by means of the generative models");
wherein the at least one synthetic dataset includes more of a particular one of the plurality of data types than the at least one dataset, wherein the particular one of the plurality of data types is based on the at least one misclassification (Mariani: Abstract and Section 1… the express purpose of BAGAN is to add additional minority-class images so that the augmented synthetic dataset contains more samples of the minority class than the original imbalanced dataset, and the minority class is precisely the class on which the classifier underperforms (i.e., the class subject to the determined misclassification), "we propose balancing GAN (BAGAN) as an augmentation tool to restore balance in imbalanced datasets ... The generative model learns useful features from majority classes and uses these to generate images for minority classes"; Section 5.2…"4) augment the imbalanced dataset to restore its balance by means of the generative models");
training the at least one model on the synthetic dataset (Mariani: Section 5.2… trains the ResNet-18 classifier on the BAGAN-augmented dataset (i.e., the dataset that has received the BAGAN-generated synthetic samples), "5) train a ResNet-18 classifier for the augmented dataset"); …
Mariani does not expressly disclose, but with Chawla does teach:
determining if the at least one misclassification is generated during the training of the at least one model on the at least one synthetic dataset based on the assigned classification score being below a particular threshold (Chawla: Section 5.4, Additional comparison to changing the decision thresholds… Chawla expressly teaches comparing each per-class classification score against a decision threshold to determine class membership during classifier training on a synthetically over-sampled (SMOTE) dataset, where samples whose minority-class score falls below the threshold are classified as majority (i.e., the score-below-threshold comparison produces the misclassification determination), "Provost (2000) suggested that simply changing the decision threshold should always be considered as an alternative to more sophisticated approaches. In the case of C4.5, this would mean changing the decision threshold at the leaves of the decision trees. For example, a leaf could classify examples as the minority class even if more than 50% of the training examples at the leaf represent the majority class. We experimented by setting the decision thresholds at the leaves for the C4.5 decision tree learner at 0.5, 0.45, 0.42, 0.4, 0.35, 0.32, 0.3, 0.27, 0.25, 0.22, 0.2, 0.17, 0.15, 0.12, 0.1, 0.05, 0.0"; Section 5…additionally, Chawla varies Ripper's loss ratio from 0.9 to 0.001 "as a means of varying misclassification cost" using the threshold relation LR/(LR+1) reported in Section 3, which independently reads on the claimed score-versus-threshold misclassification determination, "We also varied Ripper's loss ratio (Cohen & Singer, 1996; Lewis & Catlett, 1994) from 0.9 to 0.001 (as a means of varying misclassification cost) and compared the effect of this variation with the combination of SMOTE and under-sampling").
Mariani and Chawla are analogous art because they are from within the same field of endeavor, specifically the use of synthetic-data oversampling to improve machine-learning classifier accuracy on imbalanced multi-class datasets. They address the same problem of poor classifier performance on under-represented minority data types and explicitly remediate that problem by introducing additional synthetic samples of the under-represented class. Mariani recites the need to evaluate classifier accuracy on the minority class after augmentation (Mariani: Section 5.2), which is precisely the threshold-based misclassification-cost evaluation methodology developed by Chawla (Chawla: Section 5).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine Mariani's BAGAN-based class-balancing synthetic-image generation with Chawla's threshold-based misclassification-cost evaluation. Both references address the same imbalanced-classifier problem with the same general mechanism (synthetic minority-class oversampling followed by classifier retraining). Pairing Mariani's generative-model augmentation with Chawla's already-established probability-threshold misclassification metric is a routine engineering choice yielding predictable improvement with more reliable per-class accuracy reporting and a principled stopping criterion for the augment-and-retrain loop.
The suggestion/motivation for doing so would have been the explicit teaching in Chawla that classifier evaluation on imbalanced data should not rely on raw accuracy (Chawla: Section 1…"this is not appropriate when the data is imbalanced and/or the costs of different errors vary markedly") but on threshold-based misclassification metrics (Chawla: Section 2…”In the presence of imbalanced datasets with unequal error costs, it is more appropriate to use the ROC curve or other similar techniques"), combined with Mariani's explicit acknowledgment that BAGAN's contribution should be assessed by measuring classifier accuracy on the minority class after augmentation (Mariani: Section 5.2). A PHOSITA reading both references together would have been motivated to use Chawla's per-class threshold-based determination as the precise mechanism for triggering and gating Mariani's augment-retrain cycle.
Per claim 42, Mariani combined with Chawla discloses claim 41. Mariani further teaches sending a request for the at least one synthetic dataset (Mariani: Section 4, GAN training and Figure 3(c)… pipeline programmatically invokes the BAGAN generator by drawing a batch of class-conditioned latent vectors and feeding them to the generator, which constitutes a request that returns synthetic images of the requested classes, "a batch of conditional latent vectors Zc is drawn at random by applying a uniform distribution on the labels c. These vectors are processed by the generator and the output images are fed into the discriminator").
Per claim 43, Mariani combined with Chawla discloses claim 41. Mariani further teaches the at least one synthetic dataset is based on the misclassification (Mariani: Abstract and Section 1… BAGAN's generated synthetic dataset is constructed expressly to remediate misclassification of the minority class, i.e., the synthetic dataset is based on the misclassification under BRI, "we propose balancing GAN (BAGAN) as an augmentation tool to restore balance in imbalanced datasets ... by generating new minority-class images").
Per claim 44, Mariani combined with Chawla discloses claim 41. Mariani further teaches sending a request for additional data related to a particular one of the data types (Mariani: Section 4 … Mariani's class-conditional latent vector generator accepts a specific class label c as input and returns a synthetic image conditioned on that class label, which is a request specifying the particular data type, "the class-conditional latent vector generator takes as input uniformly distributed class labels c").
Per claim 45, Mariani combined with Chawla discloses claim 41. Mariani further teaches the at least one dataset includes one of (i) only real data, (ii) only synthetic data, or (iii) a combination of real data and synthetic data (Mariani: Section 5…Mariani's at-least-one dataset is composed of real images from MNIST, CIFAR-10, Flowers, and GTSRB, which constitutes subpart (i) only real data under BRI, "We validate the proposed methodology on a set of four datasets. We consider: MNIST, CIFAR-10, Flowers, and GTSRB").
Per claim 46, Mariani combined with Chawla discloses claim 41. Mariani further teaches the at least one model is trained on at least one non-synthetic dataset and at least one further synthetic dataset (Mariani: Section 5.2…the ResNet-18 classifier is trained on an augmented dataset that combines the original (non-synthetic) imbalanced training set with BAGAN-generated synthetic images, satisfying training on both a non-synthetic dataset and a further synthetic dataset, "4) augment the imbalanced dataset to restore its balance by means of the generative models, 5) train a ResNet-18 classifier for the augmented dataset").
Per claim 47, Mariani combined with Chawla discloses claim 41. Mariani further teaches the at least one dataset includes an identification of each of the plurality of data types in the at least one dataset (Mariani: Section 4 GAN training… training procedure consumes labeled (i.e., per-image class-identified) data, optimizing a sparse categorical cross-entropy loss against per-image class labels, which presumes that each sample carries an identification of its data type, "When training the discriminator D we optimize the sparse categorical cross-entropy loss function to match the class labels for real images and the fake label for the generated ones").
Per claim 48, Mariani combined with Chawla discloses claim 41. Mariani further teaches verifying an accuracy of the at least one model using at least one verification model (Mariani: Section 5.1…applies a separate, independently-trained ResNet-18 classifier (the verification model) to verify whether generated and classified images match their target classes, which constitutes verifying accuracy using a verification model under BRI, "we classify them by means of a deep learning model trained on the whole original dataset and we verify if the predicted classes match the target ones").
Per claim 49, Mariani combined with Chawla discloses claim 41. Mariani further teaches the at least one model is a machine learning procedure (Mariani: Section 5.2…at-least-one model is a ResNet-18 deep convolutional neural network, which is squarely a machine learning procedure under BRI, "5) train a ResNet-18 classifier for the augmented dataset").
Claims 50-58 are substantially similar in scope and spirit to claims 41-49, respectively. Therefore, the rejections of claims 41-49 are applied accordingly. The methods of Mariani combined with Chawla are intrinsically executed by computer hardware the includes instructions stored in memory
Claims 59 and 60 are substantially similar in scope and spirit to claims 41 and 49, respectively. Therefore, the rejections of claims 41 and 49 are applied accordingly. The methods of Mariani combined with Chawla are intrinsically executed by computer hardware in combination with the instructions/software, making up the system.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALAN CHEN whose telephone number is (571)272-4143. The examiner can normally be reached M-F 10-7.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kamran Afshar can be reached at (571) 272-7796. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ALAN CHEN/ Primary Examiner, Art Unit 2125
Prosecution Insights