DETAILED ACTION
Status of Claims
This Office action is responsive to communications filed on 2026-04-15. Claim(s) 1-15 is/are pending and are examined herein.
Claim(s) 12 and 15 is/are objected to.
Claim(s) 1-15 is/are rejected under 35 USC 103.
Notice of Pre-AIA or AIA Status
The present application, filed on or after 2013-03-16, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Regarding objections for informalities and rejections under 35 USC 112, the applicant’s amendments resolve some, but not all, of the issues raised in the previous Office action. Pending issues are reiterated below.
Regarding rejections under 35 USC 103, the applicant’s arguments have been fully considered but they are unpersuasive.
The applicant asserts that Kingma does not disclose bifurcated training using clean and adversarial samples [remarks, page 9], but the rejection did not rely upon Kingma for this disclosure. As explained in the previous Office action, Zonooz already discloses bifurcated training using clean and adversarial samples, and the disclosures of Kingma are relied upon to simply to modify the classifiers of Zonooz to make use of probabilistic encoders.
The applicant asserts that “the stated motivation that variational auto-encoders help reduce dimensionality and handle uncertainty is a general statement about VAEs and does not explain why a person of ordinary skill in the art would modify Zonooz’s specific ACT framework to incorporate a probabilistic encoder and latent space sampling mechanism” [remarks, page 10]. The examiner notes that a general statement about the well-known advantages of VAEs is sufficient as a motivation to combine in view of the generic language used in the pending independent claims. In particular, the pending independent claims do not mention the “latent space sampling mechanism” to which the applicant appears to refer in their remarks. The claims merely require that the “first/second instance of the latent space corresponds to the first/second probability distribution over the latent space” [emphasis added], without specifying specific anything about the nature of this “correspondence”, and this broad language does not necessitate that the first/second instance be obtained by sampling the latent space with respect to the first/second probability distribution, as suggested in the applicant’s remarks.
The applicant asserts that “Zonooz provides no teaching of how its classifier should operate on samples drawn from a probability distribution of a latent space” [remarks, page 10]. However, the pending independent claims do not include a recitation of “samples drawn from a probability distribution of a latent space” (as noted in item (b) above), and, moreover, they do not include any clear language regarding “how [the] classifier should operate on” such samples. Regarding this latter point, the pending independent claims require only that the “first/second instance of the latent space… produce[s] a first/second classification result” [emphasis added], without providing any details regarding how the first/second classification result are “produced” from the first/second instance of latent space. To the extent that the applicant believes that some specific sequence of intermediate steps between the output of the probabilistic encoder and the final outputs of the classifier is a key innovative aspect of their invention, this aspect is entirely absent from the pending independent claims.
The applicant asserts that incorporating a VAE into the architecture of Zonooz may present technical difficulties because a VAE “does not directly output a classification result” [remarks, pages 9-10]. More precisely, the applicant asserts that “Zonooz depends on identifying regions in the input space where the discrepancy between the two models is maximum” and that a sampling procedure would cause the “the discrepancy calculation itself [to] change in nature” and that would introduce variability that would “undermine Zonooz’s objective of precisely identifying regions in the input space where the discrepancy between the two models is maximum” [remarks, pages 9-10]. This line of argumentation is not persuasive for at least the following reasons:
First, the examiner notes that the applicant’s description of the method described in Zonooz is fragmentary. The method in Zonooz generates adversarial examples where the discrepancy is expected to be large, but the training goes on to minimize the discrepancy between the two models [Zonooz, abstract; see also, figure 2]. In other words, maximizing discrepancy is relevant in Zonooz only insofar as it is used to generate adversarial examples. The examiner notes that this is yet another point where the language used in the pending claims is at a high level of generality: there is no description in the pending claims regarding how adversarial samples are generated. The claims recite merely “collecting a plurality of data samples… wherein the plurality of data samples comprises clean data samples and adversarial data samples” [emphasis added], without providing any description regarding how the adversarial data samples might be “collected”.
Second, the examiner disagrees the use of a VAE as an initial part of the classifiers would cause the “the discrepancy calculation [to] change in nature” [remarks, page 10]. The measure of discrepancy used in Zonooz relies on the final outputs of the classifiers [Zonooz, 0021-0026], and the same quantity would continue to be a sound measure of the discrepancy between the two models even if the models included VAEs as an initial part. Nothing about the “nature” of the discrepancy calculation itself would change by this modification.
Third, the examiner disagrees with the applicant’s assertion that the introduction of variability due to a sampling procedure would “undermine” any objectives [remarks, page 10]. The applicant has provided no evidence for this assertion. Moreover, one of the fundamental insights in the field of statistical learning as a whole is that the introduction of randomness and/or variability frequently makes models more robust (e.g., by avoiding overfitting to particularities that may be present in the training data). A person of ordinary skill in the art before the effective filing date of the invention would have understood this fundamental insight of their field of work and would not conclude, as the applicant does in their remarks, that the introduction of variability due to a sampling procedure would necessarily lead to an “undermining” of any objectives.
Fourth, the examiner reiterates that the claims are recited at a high level of generality. In particular that there is no description in the claim regarding precisely how the applicant incorporates a probabilistic encoder into the classification architecture so as to avoid the difficulties that the applicant purports would arise in the proposed combination. As noted in items (b-c) above, the pending independent claims recite only a “first/second instance of the latent space” that “corresponds to the first/second probability distribution over the latent space” and that “produce[s] a first/second classification result” [emphasis added], without explaining either the nature of this “correspondence” or the steps that “produce” the classification results from the instances of the latent space. Even if the introduction of variability due to the addition of a sampling procedure were to cause problems (a point which the examiner does not concede), the invention as presently claimed provides no indication as to how the applicant envisions avoiding this purported problem.
The applicant asserts that the proposed combination “is an impermissible hindsight reconstruction” [remarks, page 9]. However, it must be recognized that any judgment on obviousness is in a sense necessarily a reconstruction based upon hindsight reasoning. So long as it takes into account only knowledge which was within the level of ordinary skill at the time the claimed invention was made, and does not include knowledge gleaned only from the applicant's disclosure, such a reconstruction is proper. See In re McLaughlin, 443 F.2d 1392, 170 USPQ 209 (CCPA 1971). The examiner notes that the conclusion of the previous Office action [Office action of 2026-01-20, pages 23-24] recorded numerous references providing documentary evidence that the combination as proposed would have been well understood by a person of ordinary skill in the art before the effective filing date of the invention (at least, at the level of generality of the pending independent claims). For example, Hoy discloses a method for “solv[ing] a binary classification problem” [Hoy, section 1 first paragraph] using a “deep learning based system” [Hoy, abstract], where the method includes “newly proposed inference techniques like Variational Recurrent Neural Networks… [which] are extensions of the well known Variational Auto-Encoder” [Hoy, section 1 second paragraph]. Similarly, Berlin describes a classifier including an encoder which is specifically indicated as being a variational autoencoder [Berlin, figure 8a/b and 0043]. Similarly, Tucot describes a classifier including a “bottleneck layer” which produces a “low dimensional representation” and which may be a “variational autoencoder” [Turcot, 0147]. Similarly, Dong discloses a classifier including an encoder in which data is produced by a sampling process [Dong, abstract; see also, figure 2, figure 4 step 408] The examiner maintains that, in view of the numerous prior art references which describe the use of VAEs as a part of a classifier, it is clear that it would have been obvious to a person of ordinary skill in the art before the effective filing date of the application how to modify the classifiers of Zonooz so as to incorporate probabilistic encoders which “produce a first/second classification result” from the “first/second instance of the latent space” that “corresponds to the first/second probability distribution over the latent space”, as recited in the pending independent claims.
The complete prior art mapping, with only superficial updates in view of the applicant’s amendments, is given below. If the applicant believes that a key innovative aspect of their invention lies in some specific relationship between probabilistic encoder and the overall classifier, the applicant is invited to amend the claims to explicitly describe the nature of this relationship in a manner that is consistent with the originally filed specification and that makes clear how their invention differs from all of the prior art made of record.
Claim Objections
Claim(s) 12 and 15 is/are objected to because of the following informalities:
Claim 12 recites a first term corresponding to maximizing a mutual information between probability distributions of encodings of pairs of the clean data samples and the adversarial data samples [emphasis added]. However, the parent claim already introduces a “first probability distribution over the latent space” and a “second probability distribution over the latent space” and the specification [specification, 0034] (as well as claims 4 and 11) indicate that the mutual information being maximized is in fact the one between these probability distributions. For consistency of nomenclature, clarity of antecedent basis, and alignment with claim 5, the examiner suggests amending claim 12 to recite “a first term corresponding to maximizing a mutual information between the first probability distribution over the latent space and the second probability distribution over the latent space”.
Claim 15 recites outputting the shared parameters [emphasis added] but this lacks antecedent basis. The examiner suggests removing the word “the” (i.e., “outputting a
Appropriate correction is required.
Claim Rejections - 35 USC 103
The following is a quotation of 35 USC 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 USC 102(b)(2)(C) for any potential 35 USC 102(a)(2) prior art against the later invention.
Claim(s) 1-3, 8-10, and 15 is/are rejected under 35 USC 103 as being unpatentable over Zonooz et al. (US20210166123A1, published 2021-06-03; hereafter “Zonooz”) in view of Kingma et al. (Auto-Encoding Variational Bayes, published 2014-05-01; hereafter “Kingma”).
Claim 1
Zonooz discloses:
A computer-implemented method for training a neural network, ([Zonooz, 0002, 0035]: Zonooz discloses an invention “relat[ing] to a method for training a robust deep neural network model” [Zonooz, 0002] and implemented by means of “a general or specific purpose computer or distributed system programmed with computer software” [Zonooz, 0035]. The deep neural network model (or, more precisely, its architecture) maps to the “neural network” of the claim.)
the neural network comprising a [probabilistic] encoder and a classifier, ([Zonooz, 0019, 0029]: Zonooz discloses an embodiment which is directed towards “a binary classification problem” [Zonooz, 0019]. It also discloses the use of one of two possible architectures: ResNet-18 and WRN-28-10 [Zonooz, 0029]. Any proper subset of consecutive layers including the final layer can be mapped to the “classifier” of the claim (since it produces the final classification output), and any preceding subset can be mapped to the “encoder” of the claim (since it “encodes” the input). Merely for the sake of concreteness, the remainder of this mapping assumes the ResNet-18 architecture, and, as this architecture has 18 layers, layer 18 can be mapped to the “classifier” of the claim and layers 1-17 to the “encoder” of the claim. See below for more regarding these mappings. In particular, see the combination with Kingma regarding the probabilistic encoder.)
the method comprising: collecting a plurality of data samples as input for training the neural network, wherein the plurality of data samples comprises clean data samples and adversarial data samples; ([Zonooz, 0020]: Zonooz discloses the use of two models: a “standard model [which] is trained on the original images” and a “robust model [which] is trained on adversarial images” [Zonooz, 0020]. The original and adversarial images map, respectively, to the “clean data samples” and the “adversarial data samples” of the claim. See the next parenthetical for more regarding this mapping.)
jointly training a first instance of the neural network and a second instance of the neural network, wherein the first instance of the neural network is trained using the clean data samples to produce a first output and the second instance of the neural network is trained using the adversarial data samples to produce a second output, ([Zonooz, 0009, 0020, 0023, 0029]: As noted above, Zonooz discloses the use of two models: a “standard model [which] is trained on the original images” and a “robust model [which] is trained on adversarial images” [Zonooz, 0020], with both models using the ResNet-18 architecture [Zonooz, 0029]. Furthermore, Zonooz discloses “collaborative[ly]” training the two models “in conjunction” [Zonooz, 0009]. More precisely, it discloses that the training makes use of a “mimicry loss which is used to align each model with the other one” [Zonooz, 0023]. The standard and robust models map, respectively, to the “first instance of the neural network” and the “second instance of the neural network” of the claim, and their outputs map respectively to the “first output” and the “second output” of the claim. The collaborative training of the two models maps to the “jointly training” step of the claim.)
wherein the training of the first instance of the neural network comprises: training the [probabilistic] encoder of the first instance of the neural network to encode the clean data samples [into a first probability distribution over a latent space;] and training the classifier of the first instance of the neural network to classify a first instance of the latent space to produce a first classification result, ([Zonooz, 0020, 0029]: As described above, the standard model maps to the “first instance of the neural network” of the claim, the first 17 layers of the architecture to the “encoder of the first instance of the neural network” of the claim, and the final layer to the “classifier of the first instance of the neural network” of the claim. The output of the “encoder of the first instance of the neural network” maps to the “first instance of the latent space” of the claim (cf. the combination with Kingma as described below for more about the latent space), and the output of the “classifier of the first instance of the neural network” to the “first classification result” of the claim. With these mappings, the function of the “encoder of the first instance of the neural network” falls under the broadest reasonable interpretation of “encod[ing] the clean data samples” as recited by the claim, and, the function of the “classifier of the first instance of the neural network” falls under the broadest reasonable interpretation of “classifiy[ing the] first instance of the latent space to produce a first classification result” as recited by the claim.)
and wherein the training of the second instance of the neural network comprises: training the [probabilistic] encoder of the second instance of the neural network to encode the adversarial data samples [into a second probability distribution over the latent space;] and training the classifier of the second instance of the neural network to classify a second instance of the latent space to produce a second classification result, ([Zonooz, 0020, 0029]: The mappings from this limitation are essentially the same as those explained in the previous parenthetical, except that it is the robust model that maps to the “second instance of the neural network” of the claim.)
optimizing a multi-objective loss function defined on the first output of the first instance of the neural network and the second output of the second instance of the neural network; ([Zonooz, 0023]: Zonooz discloses that “[e]ach model… is trained with two losses: a task specific loss and a mimicry loss which is used to align each model with the other one” [Zonooz, 0023]. The loss function used to train either model (denoted L_G [Zonooz, 0025] or L_F [Zonooz, 0026]) maps to the “multi-objective loss function” of the claim: it is “multi-objective” since it incorporates both the task-specific loss and the mimicry loss, and it defined based on both the “first output” and the “second output” as mapped above.)
and outputting shared parameters of the first instance of the neural network and the second instance of the neural network. ([Zonooz, 0023, 0025-0026]: As noted above, Zonooz discloses using a “mimicry loss which is used to align each model with the other one” [Zonooz, 0023]. Since the use of this mimicry loss results in an alignment between the network parameters of the standard model (denoted φ [Zonooz, 0026]) and those of the robust model (denoted θ [Zonooz, 0025]), the network parameters φ and θ fall under broadest reasonable interpretation of the “shared parameters” of the claim.)
Zonooz does not distinctly disclose the use of a probabilistic encoder. In other words, Zonooz does not distinctly disclose:
[the neural network comprising a] probabilistic [encoder] … [training the] probabilistic [encoder of the first instance of the neural network to encode the clean data samples] into a first probability distribution over a latent space; … wherein the first instance of the latent space corresponds to the first probability distribution over the latent space, … [training the] probabilistic [encoder of the second instance of the neural network to encode the adversarial data samples] into a second probability distribution over the latent space; … wherein the second instance of the latent space corresponds to the second probability distribution over the latent space,
Kingma is also in the field of machine learning. Moreover, Zonooz in view of Kingma discloses:
[the neural network comprising a] probabilistic [encoder] … [training the] probabilistic [encoder of the first instance of the neural network to encode the clean data samples] into a first probability distribution over a latent space; … wherein the first instance of the latent space corresponds to the first probability distribution over the latent space, … [training the] probabilistic [encoder of the second instance of the neural network to encode the adversarial data samples] into a second probability distribution over the latent space; … wherein the second instance of the latent space corresponds to the second probability distribution over the latent space, ([Kingma, section 1]: Kingma discloses a “variational auto-encoder” [Kingma, section 1 second paragraph]. In the combination, a variational auto-encoder is used as a part of the “encoder” from Zonooz as mapped above, thereby resulting in the “probabilistic encoder” of the claim. The variational auto-encoder performs “[e]fficient approximate posterior inference of the latent variable z given an observed value x” [Kingma, page 2]. This posterior (denoted p_ θ(z|x) [Kingma, figure 1 caption]) maps to the first/second “probability distribution over the latent space” of the claim, the “latent space” of the claim being the space of values which the latent variable z can take. The probability distribution then falls under the broadest reasonable interpretation of “correspond[ing] to” the first/second “instance of the latent space” as mapped under Zonooz above. The examiner notes that the claim as recited does not require any particular relationship between the first/second “instance of the latent space” and the first/second “probability distribution over the latent space”.)
Before the effective filing date of the invention, it would have been obvious to a person of ordinary skill in the art to combine the adversarial training method disclosed by Zonooz with the variational autoencoder disclosed by Kingma because variational autoencoders help to reduce data dimensionality in a way that is well-suited for handling uncertainty, thereby allowing the combination to make more robust classifications.
Claim 2
Zonooz in view of Kingma discloses the elements of the parent claim(s). It also discloses:
[The method of claim 1, wherein the optimizing of the multi-objective loss function comprises] minimizing the multi-objective loss function, wherein the multi-objective loss function measures a difference between the first output and the second output. ([Zonooz, 0012, 0023, 0025-0026]: As noted above, the loss functions disclosed in Zonooz map to the “multi-objective loss function” of the claim. Zonooz specifically indicates that the loss function is “minimize[d]” [Zonooz, 0012; see also, 0023 and 0026]. The loss function uses “Kullback-Leibler Divergence (D_{KL}) as the mimicry loss” [Zonooz, 0023; see also, 0025-0026]. This Kullback-Leibler divergence falls under the broadest reasonable interpretation of “measur[ing] a difference between the first output and the second output” as recited by the claim.)
The same motivation to combine applies.
Claim 3
Zonooz in view of Kingma discloses the elements of the parent claim(s). It also discloses:
[The method of claim 2, wherein] the joint training is performed with a plurality of latent representations for the clean data samples and the adversarial samples that are sampled multiple times. ([Zonooz, 0009, 0020, 0027 algorithm 1]: As noted under the parent claim, Zonooz discloses “collaborative[ly]” training two models [Zonooz, 0009], one on original data and another on adversarial data [Zonooz, 0020]. This process includes multiple iterations of sampling original data and generating adversarial examples, one for each iteration of the “while” loop in [Zonooz, 0027 algorithm 1]. In other words, each iteration of the loop is one of the “multiple times” of the claim, and the results produced by the encoder during all of these iterations map to the “plurality of latent representations” of the claim.)
The same motivation to combine applies.
Claim 8
Zonooz discloses:
An artificial intelligence (Al) system for training a neural network for classifying a plurality of data samples, ([Zonooz, 0002, 0035]: Zonooz discloses an invention “relat[ing] to a method for training a robust deep neural network model” [Zonooz, 0002] and implemented by means of “a general or specific purpose computer or distributed system programmed with computer software” [Zonooz, 0035]. The computer maps to the “AI system” of the claim, and the deep neural network model (or, more precisely, its architecture) maps to the “neural network” of the claim.)
the neural network comprising a [probabilistic] encoder and a classifier, ([Zonooz, 0019, 0029]: Zonooz discloses an embodiment which is directed towards “a binary classification problem” [Zonooz, 0019]. It also discloses the use of one of two possible architectures: ResNet-18 and WRN-28-10 [Zonooz, 0029]. Any proper subset of consecutive layers including the final layer can be mapped to the “classifier” of the claim (since it produces the final classification output), and any preceding subset can be mapped to the “encoder” of the claim (since it “encodes” the input). Merely for the sake of concreteness, the remainder of this mapping assumes the ResNet-18 architecture, and, as this architecture has 18 layers, layer 18 can be mapped to the “classifier” of the claim and layers 1-17 to the “encoder” of the claim. See below for more regarding these mappings. In particular, see the combination with Kingma regarding the probabilistic encoder.)
the Al system comprising: a processor; and a memory having instructions stored thereon, wherein the processor is configured to execute the stored instructions to cause the Al system to: ([Zonooz, 0035]: Zonooz discloses that “one or more processors and/or microcontrollers can operate via instructions of the computer code and the software is preferably stored on one or more tangible non-transitory memory storage devices” [Zonooz, 0035].)
collect a plurality of data samples as input for training the neural network, wherein the plurality of data samples comprises clean data samples and adversarial data samples; ([Zonooz, 0020]: Zonooz discloses the use of two models: a “standard model [which] is trained on the original images” and a “robust model [which] is trained on adversarial images” [Zonooz, 0020]. The original and adversarial images map, respectively, to the “clean data samples” and the “adversarial data samples” of the claim. See the next parenthetical for more regarding this mapping.)
jointly train a first instance of the neural network and a second instance of the neural network, wherein the first instance of the neural network is trained using the clean data samples to produce a first output and the second instance of the neural network is trained using the adversarial data samples to produce a second output, ([Zonooz, 0009, 0020, 0023, 0029]: As noted above, Zonooz discloses the use of two models: a “standard model [which] is trained on the original images” and a “robust model [which] is trained on adversarial images” [Zonooz, 0020], with both models using the ResNet-18 architecture [Zonooz, 0029]. Furthermore, Zonooz discloses “collaborative[ly]” training the two models “in conjunction” [Zonooz, 0009]. More precisely, it discloses that the training makes use of a “mimicry loss which is used to align each model with the other one” [Zonooz, 0023]. The standard and robust models map, respectively, to the “first instance of the neural network” and the “second instance of the neural network” of the claim, and their outputs map respectively to the “first output” and the “second output” of the claim. The collaborative training of the two models maps to the “jointly training” step of the claim.)
wherein the training of the first instance of the neural network comprises: training the [probabilistic] encoder of the first instance of the neural network to encode the clean data samples [into a first probability distribution over a latent space;] and training the classifier of the first instance of the neural network to classify a first instance of the latent space to produce a first classification result, ([Zonooz, 0020, 0029]: As described above, the standard model maps to the “first instance of the neural network” of the claim, the first 17 layers of the architecture to the “encoder of the first instance of the neural network” of the claim, and the final layer to the “classifier of the first instance of the neural network” of the claim. The output of the “encoder of the first instance of the neural network” maps to the “first instance of the latent space” of the claim (cf. the combination with Kingma as described below for more about the latent space), and the output of the “classifier of the first instance of the neural network” to the “first classification result” of the claim. With these mappings, the function of the “encoder of the first instance of the neural network” falls under the broadest reasonable interpretation of “encod[ing] the clean data samples” as recited by the claim, and, the function of the “classifier of the first instance of the neural network” falls under the broadest reasonable interpretation of “classifiy[ing the] first instance of the latent space to produce a first classification result” as recited by the claim.)
and wherein the training of the second instance of the neural network comprises: training the [probabilistic] encoder of the second instance of the neural network to encode the adversarial data samples [into a second probability distribution over the latent space;] and training the classifier of the second instance of the neural network to classify a second instance of the latent space to produce a second classification result, ([Zonooz, 0020, 0029]: The mappings from this limitation are essentially the same as those explained in the previous parenthetical, except that it is the robust model that maps to the “second instance of the neural network” of the claim.)
optimize a multi-objective loss function defined on the first output of the first instance of the neural network and the second output of the second instance of the neural network; ([Zonooz, 0023]: Zonooz discloses that “[e]ach model… is trained with two losses: a task specific loss and a mimicry loss which is used to align each model with the other one” [Zonooz, 0023]. The loss function used to train either model (denoted L_G [Zonooz, 0025] or L_F [Zonooz, 0026]) maps to the “multi-objective loss function” of the claim: it is “multi-objective” since it incorporates both the task-specific loss and the mimicry loss, and it defined based on both the “first output” and the “second output” as mapped above.)
and output shared parameters of the first instance of the neural network and the second instance of the neural network. ([Zonooz, 0023, 0025-0026]: As noted above, Zonooz discloses using a “mimicry loss which is used to align each model with the other one” [Zonooz, 0023]. Since the use of this mimicry loss results in an alignment between the network parameters of the standard model (denoted φ [Zonooz, 0026]) and those of the robust model (denoted θ [Zonooz, 0025]), the network parameters φ and θ fall under broadest reasonable interpretation of the “shared parameters” of the claim.)
Zonooz does not distinctly disclose the use of a probabilistic encoder. In other words, Zonooz does not distinctly disclose:
[the neural network comprising a] probabilistic [encoder] … [training the] probabilistic [encoder of the first instance of the neural network to encode the clean data samples] into a first probability distribution over a latent space; … wherein the first instance of the latent space corresponds to the first probability distribution over the latent space, … [training the] probabilistic [encoder of the second instance of the neural network to encode the adversarial data samples] into a second probability distribution over the latent space; … wherein the second instance of the latent space corresponds to the second probability distribution over the latent space,
Kingma is also in the field of machine learning. Moreover, Zonooz in view of Kingma discloses:
[the neural network comprising a] probabilistic [encoder] … [training the] probabilistic [encoder of the first instance of the neural network to encode the clean data samples] into a first probability distribution over a latent space; … wherein the first instance of the latent space corresponds to the first probability distribution over the latent space, … [training the] probabilistic [encoder of the second instance of the neural network to encode the adversarial data samples] into a second probability distribution over the latent space; … wherein the second instance of the latent space corresponds to the second probability distribution over the latent space, ([Kingma, section 1]: Kingma discloses a “variational auto-encoder” [Kingma, section 1 second paragraph]. In the combination, a variational auto-encoder is used as a part of the “encoder” from Zonooz as mapped above, thereby resulting in the “probabilistic encoder” of the claim. The variational auto-encoder performs “[e]fficient approximate posterior inference of the latent variable z given an observed value x” [Kingma, page 2]. This posterior (denoted p_ θ(z|x) [Kingma, figure 1 caption]) maps to the first/second “probability distribution over the latent space” of the claim, the “latent space” of the claim being the space of values which the latent variable z can take. The probability distribution then falls under the broadest reasonable interpretation of “correspond[ing] to” the first/second “instance of the latent space” as mapped under Zonooz above. The examiner notes that the claim as recited does not require any particular relationship between the first/second “instance of the latent space” and the first/second “probability distribution over the latent space”.)
Before the effective filing date of the invention, it would have been obvious to a person of ordinary skill in the art to combine the adversarial training method disclosed by Zonooz with the variational autoencoder disclosed by Kingma because variational autoencoders help to reduce data dimensionality in a way that is well-suited for handling uncertainty, thereby allowing the combination to make more robust classifications.
Claims 9-10 inherit limitations from claim 8 and recite additional limitations which are substantially similar to those recited by claim 2-3, respectively, so they are rejected by the same rationale.
Claim 15
Zonooz discloses:
A non-transitory computer-readable medium having stored thereon computer-executable instructions, which when executed by a computer, cause the computer to execute operations for training a neural network for classifying a plurality of data samples, ([Zonooz, 0002, 0035]: Zonooz discloses an invention “relat[ing] to a method for training a robust deep neural network model” [Zonooz, 0002] and implemented by means of “a general or specific purpose computer or distributed system programmed with computer software” [Zonooz, 0035]. The deep neural network model (or, more precisely, its architecture) maps to the “neural network” of the claim. See below regarding the “plurality of data samples” of the claim.)
the neural network comprising a [probabilistic] encoder and a classifier, ([Zonooz, 0019, 0029]: Zonooz discloses an embodiment which is directed towards “a binary classification problem” [Zonooz, 0019]. It also discloses the use of one of two possible architectures: ResNet-18 and WRN-28-10 [Zonooz, 0029]. Any proper subset of consecutive layers including the final layer can be mapped to the “classifier” of the claim (since it produces the final classification output), and any preceding subset can be mapped to the “encoder” of the claim (since it “encodes” the input). Merely for the sake of concreteness, the remainder of this mapping assumes the ResNet-18 architecture, and, as this architecture has 18 layers, layer 18 can be mapped to the “classifier” of the claim and layers 1-17 to the “encoder” of the claim. See below for more regarding these mappings. In particular, see the combination with Kingma regarding the probabilistic encoder.)
the operations comprising: collecting the plurality of data samples as input for the training of the neural network, wherein the plurality of data samples comprises clean data samples and adversarial data samples; ([Zonooz, 0020]: Zonooz discloses the use of two models: a “standard model [which] is trained on the original images” and a “robust model [which] is trained on adversarial images” [Zonooz, 0020]. The original and adversarial images map, respectively, to the “clean data samples” and the “adversarial data samples” of the claim. See the next parenthetical for more regarding this mapping.)
jointly training a first instance of the neural network and a second instance of the neural network, wherein the first instance of the neural network is trained using the clean data samples to produce a first output and the second instance of the neural network is trained using the adversarial data samples to produce a second output, ([Zonooz, 0009, 0020, 0023, 0029]: As noted above, Zonooz discloses the use of two models: a “standard model [which] is trained on the original images” and a “robust model [which] is trained on adversarial images” [Zonooz, 0020], with both models using the ResNet-18 architecture [Zonooz, 0029]. Furthermore, Zonooz discloses “collaborative[ly]” training the two models “in conjunction” [Zonooz, 0009]. More precisely, it discloses that the training makes use of a “mimicry loss which is used to align each model with the other one” [Zonooz, 0023]. The standard and robust models map, respectively, to the “first instance of the neural network” and the “second instance of the neural network” of the claim, and their outputs map respectively to the “first output” and the “second output” of the claim. The collaborative training of the two models maps to the “jointly training” step of the claim.)
wherein the training of the first instance of the neural network comprises: training the [probabilistic] encoder of the first instance of the neural network to encode the clean data samples [into a first probability distribution over a latent space;] and training the classifier of the first instance of the neural network to classify a first instance of the latent space to produce a first classification result, ([Zonooz, 0020, 0029]: As described above, the standard model maps to the “first instance of the neural network” of the claim, the first 17 layers of the architecture to the “encoder of the first instance of the neural network” of the claim, and the final layer to the “classifier of the first instance of the neural network” of the claim. The output of the “encoder of the first instance of the neural network” maps to the “first instance of the latent space” of the claim (cf. the combination with Kingma as described below for more about the latent space), and the output of the “classifier of the first instance of the neural network” to the “first classification result” of the claim. With these mappings, the function of the “encoder of the first instance of the neural network” falls under the broadest reasonable interpretation of “encod[ing] the clean data samples” as recited by the claim, and, the function of the “classifier of the first instance of the neural network” falls under the broadest reasonable interpretation of “classifiy[ing the] first instance of the latent space to produce a first classification result” as recited by the claim.)
and wherein the training of the second instance of the neural network comprises: training the [probabilistic] encoder of the second instance of the neural network to encode the adversarial data samples [into a second probability distribution over the latent space;] and training the classifier of the second instance of the neural network to classify a second instance of the latent space to produce a second classification result, ([Zonooz, 0020, 0029]: The mappings from this limitation are essentially the same as those explained in the previous parenthetical, except that it is the robust model that maps to the “second instance of the neural network” of the claim.)
optimizing a multi-objective loss function defined on the first output of the first instance of the neural network and the second output of the second instance of the neural network; ([Zonooz, 0023]: Zonooz discloses that “[e]ach model… is trained with two losses: a task specific loss and a mimicry loss which is used to align each model with the other one” [Zonooz, 0023]. The loss function used to train either model (denoted L_G [Zonooz, 0025] or L_F [Zonooz, 0026]) maps to the “multi-objective loss function” of the claim: it is “multi-objective” since it incorporates both the task-specific loss and the mimicry loss, and it defined based on both the “first output” and the “second output” as mapped above.)
and outputting the shared parameters of the first instance of the neural network and the second instance of the neural network. ([Zonooz, 0023, 0025-0026]: As noted above, Zonooz discloses using a “mimicry loss which is used to align each model with the other one” [Zonooz, 0023]. Since the use of this mimicry loss results in an alignment between the network parameters of the standard model (denoted φ [Zonooz, 0026]) and those of the robust model (denoted θ [Zonooz, 0025]), the network parameters φ and θ fall under broadest reasonable interpretation of the “shared parameters” of the claim.)
Zonooz does not distinctly disclose the use of a probabilistic encoder. In other words, Zonooz does not distinctly disclose:
[the neural network comprising a] probabilistic [encoder] … [training the] probabilistic [encoder of the first instance of the neural network to encode the clean data samples] into a first probability distribution over a latent space; … wherein the first instance of the latent space corresponds to the first probability distribution over the latent space, … [training the] probabilistic [encoder of the second instance of the neural network to encode the adversarial data samples] into a second probability distribution over the latent space; … wherein the second instance of the latent space corresponds to the second probability distribution over the latent space,
Kingma is also in the field of machine learning. Moreover, Zonooz in view of Kingma discloses:
[the neural network comprising a] probabilistic [encoder] … [training the] probabilistic [encoder of the first instance of the neural network to encode the clean data samples] into a first probability distribution over a latent space; … wherein the first instance of the latent space corresponds to the first probability distribution over the latent space, … [training the] probabilistic [encoder of the second instance of the neural network to encode the adversarial data samples] into a second probability distribution over the latent space; … wherein the second instance of the latent space corresponds to the second probability distribution over the latent space, ([Kingma, section 1]: Kingma discloses a “variational auto-encoder” [Kingma, section 1 second paragraph]. In the combination, a variational auto-encoder is used as a part of the “encoder” from Zonooz as mapped above, thereby resulting in the “probabilistic encoder” of the claim. The variational auto-encoder performs “[e]fficient approximate posterior inference of the latent variable z given an observed value x” [Kingma, page 2]. This posterior (denoted p_ θ(z|x) [Kingma, figure 1 caption]) maps to the first/second “probability distribution over the latent space” of the claim, the “latent space” of the claim being the space of values which the latent variable z can take. The probability distribution then falls under the broadest reasonable interpretation of “correspond[ing] to” the first/second “instance of the latent space” as mapped under Zonooz above. The examiner notes that the claim as recited does not require any particular relationship between the first/second “instance of the latent space” and the first/second “probability distribution over the latent space”.)
Before the effective filing date of the invention, it would have been obvious to a person of ordinary skill in the art to combine the adversarial training method disclosed by Zonooz with the variational autoencoder disclosed by Kingma because variational autoencoders help to reduce data dimensionality in a way that is well-suited for handling uncertainty, thereby allowing the combination to make more robust classifications.
Claim(s) 4-5 and 11-12 is/are rejected under 35 USC 103 as being unpatentable over Zonooz in view of Kingma, further in view of Becker (Mutual information maximization: models of cortical self-organization, published 1996; hereafter, “Becker”).
Claim 4
Zonooz in view of Kingma already discloses the elements of the parent claim(s). It also discloses:
[The method of claim 1, further comprising] parameterizing the multi-objective loss function based on [a mutual information of the first probability distribution over the latent space and the second probability distribution over the latent space] and entropy losses of the first classification result and the second classification result. ([Zonooz, 0023, 0025-0026]: As noted above, the loss functions disclosed by Zonooz make use of a “task specific loss” given by “a natural cross-entropy between the output of the model and the ground truth” [Zonooz, 0023]. The task specific losses map to the “entropy losses” of the claim.)
The same motivation to combine applies.
Zonooz in view of Kingma does not distinctly disclose:
[the multi-objective loss function based on] a mutual information of the first probability distribution over the latent space and the second probability distribution over the latent space
Becker is in the field of neural networks. Moreover, Zonooz in view of Kingma and Becker discloses:
[the multi-objective loss function based on] a mutual information of the first probability distribution over the latent space and the second probability distribution over the latent space ([Becker, section 2]: Becker discloses the use of a “mutual information cost function” [Becker, section 2 pages 11-12 paragraph beginning “To see”] which ensures agreement between the outputs of “two different neural units or neural network modules” by “maximiz[ing] the mutual information between the outputs of two (or more) different network modules” [Becker, section 2 page 11 paragraph beginning “In collaboration”]. In the combination, the neural network modules of Becker are taken to be variational autoencoders disclosed by Kingma, i.e., the “probabilistic encoder[s]” of the claims, so that the outputs whose mutual information is being maximized are the probability distributions over the latent space of the claim.)
Before the effective filing date of the invention, it would have been obvious to a person of ordinary skill in the art to incorporate a term for maximizing mutual information into a cost/loss function as disclosed by Becker into the adversarial training method disclosed by Zonooz in view of Kingma because maximizing mutual information “selects features which agree across multiple input channels” [Becker, section 2 pages 11-12 paragraph beginning “To see how”] and “provides a means for information in one channel to modulate learning in another channel” [Becker, section 2 page 12 paragraph beginning “One advantage”], so the combination would ensure that probabilistic encodings produced by the two networks are better aligned.
Claim 5
Zonooz in view of Kingma discloses the elements of the parent claim(s). It also discloses:
[The method of claim 1, wherein] the multi-objective loss function comprises … a second term corresponding to minimizing a symmetrized Kullback-Leibler divergence between encodings of one of the clean data samples or the adversarial data samples in the pair conditioned on another data sample in the pair, ([Zonooz, 0025-0026]: The second terms on the right-hand side of [Zonooz, 0025 equation 1] and [Zonooz, 0026 equation 2] are Kullback-Leibler divergence terms and fall under the broadest reasonable interpretation of “corresponding to… minimizing” a symmetrized Kullback-Leibler divergence: since the loss functions L_F and L_G are minimized, so too are each of those Kullback-Leibler divergences (since the coefficients in front of them are positive), and so too is a corresponding symmetrized Kullback-Leibler divergence (namely, the one defined to be the sum of the two Kullback-Leibler divergences, which is also known as Jeffreys divergence).)
a third term corresponding to a clean cross-entropy loss determined for classifying the clean data samples, ([Zonooz, 0026]: The first term on the right-hand side of [Zonooz, 0026 equation 2] is a cross-entropy between the output of the model on the original/clean data and ground-truth labels. In other words, this term “corresponds to… a clean cross-entropy loss” as recited by the claim.)
and a fourth term corresponding to an adversarial cross-entropy loss determined for classifying the adversarial data samples. ([Zonooz, 0025]: The first term on the right-hand side of [Zonooz, 0025 equation 1] is a cross-entropy between the output of the model on the adversarial examples and ground-truth labels. In other words, this term “corresponds to… an adversarial cross-entropy loss” as recited by the claim.)
Zonooz in view of Kingma does not distinctly disclose:
[the multi-objective loss function comprises] a first term corresponding to maximizing a mutual information between the first probability distribution over the latent space and the second probability distribution over the latent space,
Becker is in the field of neural networks. Moreover, Zonooz in view of Kingma and Becker discloses:
[the multi-objective loss function comprises] a first term corresponding to maximizing a mutual information between the first probability distribution over the latent space and the second probability distribution over the latent space, ([Becker, section 2]: Becker discloses the use of a “mutual information cost function” [Becker, section 2 pages 11-12 paragraph beginning “To see”] which ensures agreement between the outputs of “two different neural units or neural network modules” by “maximiz[ing] the mutual information between the outputs of two (or more) different network modules” [Becker, section 2 page 11 paragraph beginning “In collaboration”]. In the combination, the neural network modules of Becker are taken to be variational autoencoders disclosed by Kingma, i.e., the “probabilistic encoder[s]” of the claims, so that the outputs whose mutual information is being maximized are the probability distributions over the latent space of the claim, i.e., the “probability distributions of encodings” of the claim. The mutual information term incorporated into the loss functions of Zonooz then maps to the “first term” of the claim.)
Before the effective filing date of the invention, it would have been obvious to a person of ordinary skill in the art to incorporate a term for maximizing mutual information into a cost/loss function as disclosed by Becker into the adversarial training method disclosed by Zonooz in view of Kingma because maximizing mutual information “selects features which agree across multiple input channels” [Becker, section 2 pages 11-12 paragraph beginning “To see how”] and “provides a means for information in one channel to modulate learning in another channel” [Becker, section 2 page 12 paragraph beginning “One advantage”], so the combination would ensure that probabilistic encodings produced by the two networks are better aligned.
Claims 11-12 inherit limitations from claim 8 and recite additional limitations which are substantially similar to those recited by claims 4-5, respectively, so they are rejected by the same rationale.
Claim(s) 6 and 13 is/are rejected under 35 USC 103 as being unpatentable over Zonooz in view of Kingma, further in view of Xiao et al. (US20190012575A1, published 2019-01-10; hereafter, “Xiao”).
Claim 6
Zonooz in view of Kingma already discloses the elements of the parent claim(s). It also discloses:
[The method of claim 1, wherein the collecting the plurality of data samples comprises: receiving the clean data samples over a communication channel;] and modifying each clean data sample of the clean data samples to generate a corresponding adversarial data sample of the adversarial data samples. ([Zonooz, 0020]: Zonooz discloses adding an “adversarial perturbation, δ” to the original/clean data in order to obtain the adversarial examples. This addition of a perturbation maps to the “modifying” recited by the claim.)
Zonooz in view of Kingma does not distinctly disclose:
receiving the clean data samples over a communication channel,
Xiao is also in the field of machine learning. Moreover, Zonooz in view of Kingma and Xiao discloses:
receiving the clean data samples over a communication channel, ([Xiao, 0047]: Xiao discloses “receiv[ing] a new training data set… through a wired connection or a wireless connection”. In the combination, the training data set corresponds to the original data in Zonooz and the “clean data samples” of the claim.)
Before the effective filing date of the invention, it would have been obvious to a person of ordinary skill in the art to combine the adversarial training method disclosed by Zonooz in view of Kingma with the process of receiving training data through a wired or wireless connection as disclosed by Xiao because receiving data through such a connection would be faster than manually copying data from one device or component to another.
Claims 13 inherits limitations from claim 8 and recites additional limitations which are substantially similar to those recited by claim 6, so it is rejected by the same rationale.
Claim(s) 7 and 14 is/are rejected under 35 USC 103 as being unpatentable over Zonooz in view of Kingma and Xiao, further in view of Madry et al. (Towards Deep Learning Models Resistant to Adversarial Attacks, published 2019-09-04; hereafter, “Madry”).
Claim 7
While Zonooz in view of Kingma and Xiao discloses generating adversarial examples by perturbing the original examples, it does not distinctly disclose:
[The method of claim 6, wherein the modifying comprises:] applying an adversarial example generation method on each clean data sample of the clean data samples, wherein the adversarial example generation method is one of a projected gradient descent method, a fast-gradient sign method, a limited-memory Broyden-Fletcher-Goldfarb-Shanno method, a Jacobian-based saliency map attack, or a Carlini & Wagner attack.
Madry is in the field of machine learning. Moreover, Zonooz in view of Xiao and Madry discloses:
[The method of claim 6, wherein the modifying comprises:] applying an adversarial example generation method on each clean data sample of the clean data samples, wherein the adversarial example generation method is one of a projected gradient descent method, a fast-gradient sign method, a limited-memory Broyden-Fletcher-Goldfarb-Shanno method, a Jacobian-based saliency map attack, or a Carlini & Wagner attack. ([Madry, page 2]: Madry discloses the use of “projective gradient descent (PGD)” for generating adversarial examples.)
Before the effective filing date of the invention, it would have been obvious to a person of ordinary skill in the art to combine the adversarial training method disclosed by Zonooz in view of Kingma and Xiao with the use of PGD as disclosed by Madry because PDG is a “reliable first-order adversary” which generates the “strongest attack” (i.e., the most convincing adversarial examples) [Madry, page 2], so the use of examples generated by PGD would result in a most robust system overall.
Claims 14 inherits limitations from claim 8 and recites additional limitations which are substantially similar to those recited by claim 7, so it is rejected by the same rationale.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Shishir AGRAWAL whose telephone number is +1 703-756-1183. The examiner can normally be reached Monday through Thursday, 08:30-14:30 Pacific Time.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alexey SHMATOV can be reached on +1 571-270-3428. The fax phone number for the organization where this application or proceeding is assigned is +1 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at +1 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call +1 800-786-9199 (IN USA OR CANADA) or +1 571-272-1000.
/S.A./Examiner, Art Unit 2123
/ALEXEY SHMATOV/Supervisory Patent Examiner, Art Unit 2123
Prosecution Insights