Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 09/02/2026 has been entered.
Response to Amendment
This Office action has been issued in response to amendment filed on 09/02/2026, Claims (1-11, 15-16) and 14 are pending. Applicants' arguments have been carefully and respectfully considered and addressed. Accordingly, this action has been made FINAL necessitated by amendment.
Claims 12-13 are cancelled.
Response to Arguments
Applicants' arguments have been carefully and respectfully considered and addressed. The arguments presented are moot based on amendment.
With regards to arguments that pertain to 103 rejection, Applicant arguments and amendment were fully considered and are moot in view of the new ground rejection wherein Li et al. US Patent Application Publication US 20220084204 A1 (hereinafter Li and further in view of Lee et al. US Patent Application Publication US 20180285771 A1 (hereinafter Lee) and further in view of Steven Liu et al. Diverse Image Generation via Self-Conditioned GANs, 2020 (hereinafter Liu) and further in view of Jia-Jie Zhu et al. Generative Adversarial Active Learning, 2017 (hereinafter Zhu) and further in view of Aleksei Triastcyn et al. Generating Higher-Fidelity Synthetic Datasets with Privacy Guarantees, 2020 (hereinafter Triastcyn) for teaching the previously presented and amended claims.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-9, 11, 14-16 are rejected under AIA 35 U.S.C. 103(a) as being unpatentable over Li et al. US Patent Application Publication US 20220084204 A1 (hereinafter Li and further in view of Lee et al. US Patent Application Publication US 20180285771 A1 (hereinafter Lee) and further in view of Steven Liu et al. Diverse Image Generation via Self-Conditioned GANs, 2020 (hereinafter Liu) and further in view of Jia-Jie Zhu et al. Generative Adversarial Active Learning, 2017 (hereinafter Zhu) and further in view of Aleksei Triastcyn et al. Generating Higher-Fidelity Synthetic Datasets with Privacy Guarantees, 2020 (hereinafter Triastcyn).
Regarding claim 1, Li teaches A process for training a first artificial neural network structure (1), the process comprising ([0120] wherein Li describes improving a neural network and determines the accuracy of generator network of GAN) classifying data samples are classified in different classes (4) by the first artificial neural network structure (1), whereby at least some of the classes (4) are unsupervised classes (6), which are generated and/or filled by unsupervised learning (FIGS. 3A-3B, 4, 6, [0067], [0069], [0089], [0092] wherein Li describes a neural network for classifying data samples in unsupervised manner by generating labels and notations for images, wherein labels indicates classification to each pixel within the image) wherein the generated artificial candidates (7) are labelled in a supervised learning for labelling and/or annotating the said unsupervised class (7) ([0078], [0103], [0489] wherein Li incorporates artificial network to generate labels and annotation classes for images). ([0076] wherein Li incorporates GAN that is a class of artificial intelligence system that uses two types of artificial neural networks contesting with each other. A GAN includes a first type of artificial neural networks, referred to as a generator network, that generates candidates and a second type of artificial neural networks, referred to as a discriminator network, that evaluates generated candidates. A generator network learns to map from a latent space to a particular data distribution of interest (a data distribution of changes to input images that are indistinguishable from photographs to human eyes), while a discriminator network discriminates between instances from a training dataset and candidates produced by generator network. In at least one embodiment, a GAN can have a generator network and two discriminator networks. A first discriminator network evaluates synthetic images generated by generator network, and a second discriminator network evaluates synthetic images and corresponding labels generated by generator network).
Li does not teach wherein the unsupervised classes (6) have no pre-existing labels assigned to the data samples in that class, and wherein the unsupervised classes (6); training a second artificial neural network structure (2) based on classification results produced by the first artificial neural network structure (1) to generate artificial candidates (7) belonging to at least one of the unsupervised classes ( 6) identified by the first artificial neural network structure (1);
However in analogous art of training artificial neural network structure, Lee teaches wherein the unsupervised classes (6) have no pre-existing labels assigned to the data samples in that class, and wherein the unsupervised classes (6); training a second artificial neural network structure (2) based on classification results produced by the first artificial neural network structure (1) to generate artificial candidates (7) belonging to at least one of the unsupervised classes ( 6) identified by the first artificial neural network structure (1)(FIGS. 1-4. [0006-0007], [0024-0048] wherein Lee processes unlabeled data through a semi-supervised learning and training the supervised learning based on the result of the updated classifier and augmented labeled data produced by the semi-supervised and transmitted to supervised learning. Wherein the assigned labeled are used as input to the first learning as illustrated in FIG. 3, wherein the first learning neural network is marked as 100 receiving augmented labeled data 112).
PNG
media_image1.png
540
1002
media_image1.png
Greyscale
It would have been obvious to a person in the ordinary skill in the art before the effective filing date of the claimed invention to combine Li with Lee by incorporating the method of wherein the unsupervised classes (6) have no pre-existing labels assigned to the data samples in that class, and wherein the unsupervised classes (6); training a second artificial neural network structure (2) based on classification results produced by the first artificial neural network structure (1) to generate artificial candidates (7) belonging to at least one of the unsupervised classes ( 6) identified by the first artificial neural network structure (1); of Lee into the method of classifying data samples are classified in different classes (4) by the first artificial neural network structure (1), whereby at least some of the classes (4) are unsupervised classes (6), which are generated and/or filled by unsupervised learning of Li for the purpose of identifying high confidence data from the unlabeled data whose label can be inferred based upon the labeled data and the classifier (Lee: [0024]).
Li does not teach wherein the generated artificial candidates (7) are labelled in a supervised learning such that only the generated artificial candidates (7) are presented to a human operator for labelling and the data samples of the unsupervised class (6) are not disclosed to the human operator, and wherein the labels assigned to the generated artificial candidates (7) are used as input to the first artificial neural network structure (1) for labelling the said unsupervised class (6).
However in analogous art of training artificial neural network structure, Liu teaches wherein the generated artificial candidates (7) are labelled in a supervised learning (FIG. 2, §21 “conditional GAN training”, p. 14275-14276, wherein Liu teaches generating an artificial candidates belonging to automatically discovered clusters and generating images conditioned on cluster index, wherein the candidates are labelled in a supervised learning) such that only the generated artificial candidates (7) are presented to a human operator for labelling (§4, p.4-5 wherein Zhu describes generating instances for querying and having human annotators label those instances) and the data samples of the unsupervised class (6) are not disclosed to the human operator (§5.3, p.5 wherein Triastcyn teaches using generated samples in place of real data for annotation and obtaining annotations without exposing the original sensitive data) wherein the labels assigned to the generated artificial candidates (7) are used as input to the first artificial neural network structure (1) for labelling the said unsupervised class (6) (§4, p.5 wherein Zhu teaches adding the human labeled generated candidates to the training dataset and retraining the learner. Zhu permits implementing the learner as a convolutional neural network. Thus, Zhu teaches using labeled generated candidates as training input to a neural network learner).
It would have been obvious to a person in the ordinary skill in the art before the effective filing date of the claimed invention to combine Li with Liu, Zhu and Triastcyn to present Liu’s class conditioned artificial candidates for human labelling as taught by Zhu while withholding the original samples of the corresponding class as suggested by Triastcyn, to obtain human annotations while protecting the original data.
Regarding claim 2, Li as modified by Lee, Liu, Zhu and Triastcyn teaches wherein the process is a process for image classification whereby the data samples are images especially taken by at least one surveillance camera ([0143-0148], [0168] wherein Li’s data samples are images captured by a camera).
Regarding claim 3, Li as modified by Lee, Liu, Zhu and Triastcyn teaches wherein the first artificial network structure (1) is trained with the labelled and/or annotated artificial candidates (7) in order to label and/or annotate the said supervised class (6) (FIGS. 3A-3B, 4, 6, [0056], [0067], [0070], [0078], [0103], [0107] wherein Li describes training neural network with labelled updated images).
Regarding claim 4, Li as modified by Lee, Liu, Zhu and Triastcyn teaches wherein the first artificial neural network structure (1) is a convolutional artificial neural network and/or that the data samples are images ([0006-0008], [0067], [0072-0074] wherein Li’s data samples are images).
Regarding claim 5, Li as modified by Lee, Liu, Zhu and Triastcyn teaches wherein a part of the classes (4) are supervised classes (5), which are generated and/or filled by supervised learning (FIGS. 3A-3B, 4, 6, [0056], [0067], [0070], [0078], [0103], [0107] wherein Li describes generating labels by supervised learning).
Regarding claim 6, Li as modified by Lee, Liu, Zhu and Triastcyn teaches wherein the second artificial neural network structure (2) is trained by improving a loss-function of the probability density function of the respective unsupervised class (6) (§4, p.4-5, wherein Zhu describes formulating a generative adversarial active learning for the loss function based on uncertainty sampling principals, wherein the formulating includes optimization problem as described in Algorithm 1 that describes minimizing the loss), (§2-3, p. 3-4, wherein Liu describes an algorithm for training and updating a generator and discriminator as part of the self-Conditioned GAN training, wherein the training and updating is based on probability. Liu describes a method for training class-conditional GANs on unlabeled images by clustering on features obtained via unsupervised feature learning methods, the method directly clusters on discriminator features that inherently exist in GANs, leading to a simpler method and achieving higher quality generation in the experiments (Section 4.4). Wherein a Mixture Density GAN proposes to use loglikelihoods of a Gaussian mixture distribution in discriminator feature space as the GAN objective. GAN-Tree uses clustering to split a GAN into a tree hierarchy of GANs for better mode coverage. Liu teaches training a class conditioned generator by minimizing its adversarial objective against a discriminator conditioned on the same automatically discovered cluster).
Regarding claim 7, Li as modified by Lee, Liu, Zhu and Triastcyn teaches wherein the second artificial neural network structure (2) comprises a generative artificial neural network ([0006], [0067], [0072-0073], [0078] wherein Li describes generating training dataset using a generative adversarial network (GAN) that generates synthetic images and an associated trained neural network that generates labels for synthetic images generated by the GAN), (§3, p. 14276 wherein Liu describes Equation 1 and algorithm 1 and wherein the second artificial neural network structure comprises a generative artificial neural network by disclosing a class conditional generator G(s, c), trained to generate images corresponding to an automatically discovered cluster c. The generator therefore teaches the claimed generative artificial neural network).
Regarding claim 8, Li as modified by Lee, Liu, Zhu and Triastcyn teaches wherein the second artificial neural network structure (2) comprises a discriminative artificial neural network, whereby the generative and the discriminative artificial neural network form a generative adversarial network (GAN) ([0076] wherein Li incorporates GAN that is a class of artificial intelligence system that uses two types of artificial neural networks contesting with each other. A GAN includes a first type of artificial neural networks, referred to as a generator network, that generates candidates and a second type of artificial neural networks, referred to as a discriminator network, that evaluates generated candidates. A generator network learns to map from a latent space to a particular data distribution of interest (a data distribution of changes to input images that are indistinguishable from photographs to human eyes), while a discriminator network discriminates between instances from a training dataset and candidates produced by generator network. In at least one embodiment, a GAN can have a generator network and two discriminator networks. A first discriminator network evaluates synthetic images generated by generator network, and a second discriminator network evaluates synthetic images and corresponding labels generated by generator network), (§3 “Conditional GAN training”, p. 14276, wherein Liu discloses a class conditional discriminator D(x,c) trained together with a class conditional generator C(z,c) under an adversarial objective. The discriminator distinguishes real images belonging to cluster c from images produced by the generator, while the generator learns to produce images that fool the discriminator).
Regarding claim 9, Li as modified by Lee, Liu, Zhu and Triastcyn teaches wherein the first artificial neural network structure (1) is realized as a discriminative artificial neural network, whereby the generative and the discriminative artificial neural network form a generative adversarial network (GAN) ([0076] wherein Li incorporates GAN that is a class of artificial intelligence system that uses two types of artificial neural networks contesting with each other. A GAN includes a first type of artificial neural network, referred to as a generator network, that generates candidates and a second type of artificial neural network, referred to as a discriminator network, that evaluates generated candidates. A generator network learns to map from a latent space to a particular data distribution of interest (a data distribution of changes to input images that are indistinguishable from photographs to human eyes), while a discriminator network discriminates between instances from a training dataset and candidates produced by generator network. In at least one embodiment, a GAN can have a generator network and two discriminator networks. A first discriminator network evaluates synthetic images generated by generator network, and a second discriminator network evaluates synthetic images and corresponding labels generated by generator network), (§3 “Conditional GAN training”, p. 14276, wherein Liu discloses a class conditional discriminator D(x,c) trained together with a class conditional generator C(z,c) under an adversarial objective. The discriminator distinguishes real images belonging to cluster c from images produced by the generator, while the generator learns to produce images that fool the discriminator).
Regarding claim 11, Li as modified by Lee, Liu, Zhu and Triastcyn teaches wherein, characterized that in the unsupervised class (6) only the artificial candidates (7) are labelled and/or annotated ([0078], [0103], [0489] wherein Li incorporates artificial network to generate labels and annotation classes for images). ([0076] wherein Li incorporates GAN that is a class of artificial intelligence system that uses two types of artificial neural networks contesting with each other. A GAN includes a first type of artificial neural networks, referred to as a generator network, that generates candidates and a second type of artificial neural networks, referred to as a discriminator network, that evaluates generated candidates. A generator network learns to map from a latent space to a particular data distribution of interest (a data distribution of changes to input images that are indistinguishable from photographs to human eyes), while a discriminator network discriminates between instances from a training dataset and candidates produced by generator network. In at least one embodiment, a GAN can have a generator network and two discriminator networks. A first discriminator network evaluates synthetic images generated by generator network, and a second discriminator network evaluates synthetic images and corresponding labels generated by generator network), (§5.3, “Learning Performance”, wherein Triastcyn proposes using generated samples to fully replace real data for manual annotation, thereby eliminating manual labeling of potentially sensitive original data. Triastcyn illustrates this approach with doctors annotating generating medical data).
Regarding claim 11, Li as modified Lee, Liu, Zhu and Triastcyn teaches wherein, characterized that in the unsupervised class (6) only the artificial candidates (7) are labelled and/or annotated ([0078], [0103], [0489] wherein Li incorporates artificial network to generate labels and annotation classes for images). ([0076] wherein Li incorporates GAN that is a class of artificial intelligence system that uses two types of artificial neural networks contesting with each other. A GAN includes a first type of artificial neural networks, referred to as a generator network, that generates candidates and a second type of artificial neural networks, referred to as a discriminator network, that evaluates generated candidates. A generator network learns to map from a latent space to a particular data distribution of interest (a data distribution of changes to input images that are indistinguishable from photographs to human eyes), while a discriminator network discriminates between instances from a training dataset and candidates produced by generator network. In at least one embodiment, a GAN can have a generator network and two discriminator networks. A first discriminator network evaluates synthetic images generated by generator network, and a second discriminator network evaluates synthetic images and corresponding labels generated by generator network).
Regarding claim 14, Li teaches A non-transitory, computer-readable medium having stored thereon instructions that when executed by a computer cause the computer ([0570]). Claim 14 is similar in scope to claim 1, therefore the claim is rejected under similar rationale.
Regarding claim 15, Li as modified Lee, Liu, Zhu and Triastcyn teaches wherein the second artificial neural network structure (2) is configured to generate artificial candidates (7) belonging to the unsupervised class (6), such that the artificial candidates (7) are classified into the unsupervised class (6) by the first artificial neural network structure (1), and wherein the unsupervised class (6) comprises unlabeled original data samples and labeled and/or annotated artificial candidates (7), whereby the artificial candidates (7) at least partially label and/or annotate the unsupervised class (6) (FIGS. 1-4. [0006-0007], [0024-0048] wherein Lee describes generating augmented labeled data using the semi-supervised learning using a classifier and generating data of potential new classes), (§3 “Conditional GAN training”, p. 14276, wherein Liu discloses thought its class conditional generator G(z,c) which generates images corresponding to an automatically discovered clsuter), (§4, p. 5, algorithm 1, wherein Zhu teaches labelling generated candidates and adding them to the training set to update a classifier, including an expressly contemplated neural network classifier).
Regarding claim 16, Li as modified by Lee, Liu, Zhu and Triastcyn teaches wherein when the first artificial neural network (1) incorrectly classifies the artificial candidates (7) into a class (4) other than the unsupervised class (6), the incorrectly classified artificial candidates (7) are returned to the second artificial neural network structure (2) in order to train the network structure of the second artificial neural network structure (2) (FIGS. 1-4. [0006-0007], [0024-0048] wherein Lee identifies high confidence data from the unlabeled data whose label can be inferred based upon the labeled data and the classifier. Wherein the updated classifier and the unlabeled data are processed by the active learning module executed by a computer program of the machine learning tool, which identifies an uncertain or unusual subset of the unlabeled data as well as samples of the high confidence data for validation and requests labels from a labeler in the form of a data label request. The labeler determines the labels for the unlabeled data in the data label request and returns the labels for requested data including labels for the samples of the high confidence data. The active learning module combines the labels for requested data with the uncertain unlabeled data and sampled high confidence data identified in the data label request to produce the augmented labeled data), (§3, p. 14276, wherein Liu teaches training the generator using feedback from a discriminator that evaluates generated images against intended cluster).
Claim 10 is rejected under AIA 35 U.S.C. 103(a) as being unpatentable over Li et al. US Patent Application Publication US 20220084204 A1 (hereinafter Li and further in view of Lee et al. US Patent Application Publication US 20180285771 A1 (hereinafter Lee) and further in view of Steven Liu et al. Diverse Image Generation via Self-Conditioned GANs, 2020 (hereinafter Liu) and further in view of Jia-Jie Zhu et al. Generative Adversarial Active Learning, 2017 (hereinafter Zhu) and further in view of Aleksei Triastcyn et al. Generating Higher-Fidelity Synthetic Datasets with Privacy Guarantees, 2020 (hereinafter Triastcyn) and further in view of Diederik P. Kingma et al. Auto-Encoding Variational Bayes, 2014 (hereinafter Kingma).
Regarding claim 10, Li, Lee, Liu, Zhu and Triastcyn do not teach wherein the generative artificial neural network is a variational autoencoder (VAEs).
However in analogous art of training artificial neural network structure, Kingma teaches wherein the generative artificial neural network is a variational autoencoder (VAEs) (§3, “Example: Variational Auto-Encoder”, p.5, p. 11, Equations 9-10, wherein Kingma discloses a variational autoencoder comprising a neural network probabilistic encoder and a neural network decoder defining the generative distribution with their parameters jointly optimized using auto-encoding variational Bayes algorithm).
It would have been obvious to a person in the ordinary skill in the art before the effective filing date of the claimed invention to combine Kingma with Li, Lee, Liu, Zhu and Triastcyn to modify the candidate generation stage of Li, Liu, Zhu and Triastcyn to employ the variational autoencoder, the modification would train the variational autoencoder on data samples belonging to an unsupervised class identified through Liu’s clustering process and use its decoder to generate artificial candidates for the subsequent labeling process. The motivation would been to obtain Kingma’s efficient, scalable learning of a probabilistic generative model using stochastic gradient optimization, while producing artificial data resembling the training data (Kingma: §1, p.1, §2.1, p.2, §3, p.5)
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HASSAN MRABI whose telephone number is (571)272-8875. The examiner can normally be reached on Monday-Friday, 7:30am-5pm. Alt, Friday, EST.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Viker Lamardo can be reached on 571-270-5871. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HASSAN MRABI/Examiner, Art Unit 2144
Prosecution Insights