DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This final office action is in response to the amendment filed 29 October 2025.
Claims 1-18 are pending. Claims 1 and 10 are independent claims.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-18 remain rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
When considering subject matter eligibility under 35 USC 101, it must be determined whether the claim is directed to one of the four statutory categories of invention, i.e., process, machine, manufacture, or composition of matter (Step 1).
When the claim falls within one of the four statutory categories, the second step of the analysis is to determine whether the claim is directed to a judicial exception (Step 2A). This analysis is broken into two prongs. The first prong (Step 2A, Prong 1) determines whether or not the claims recited a judicial exception (e.g., mathematical concepts, mental processes, certain methods of organizing human activity). If the claims recite a judicial exception under Step 2A, Prong 1, it is determined whether or not the claims integrate the judicial exception into a practical application.
If it is determined that the claims do not integrate the judicial exception into a practical application, the analysis proceeds to determining whether the claim is a patent-eligible application of the exception (Step 2B). If an abstract idea is present in the claim, any element or combination of elements in the claim must be sufficient to ensure that the claim integrates the judicial exception into a practical application or amounts to significantly more than the abstract idea itself. The applicant is advised to consult the 2019 PEG for more details regarding the analysis.
Step 1:
With regard to the current application, claims 1-9 recite a method (process) for performing incremental learning and claims 10-18 recite an electronic device (machine) for performing incremental learning. Therefore, claims 1-18 fall within one of the four statutory categories.
Claims 1 and 10:
Step 2A, Prong 1
Following the determination that the claims fall within one of the statutory categories (Step 1), it must be determined if the claims recite a judicial exception (Step 2A, Prong 1). In this instance, the claims are determined to recite a judicial exception (abstract idea; mental process).
With respect to claims 1 and 10, the claims recite:
designating a pre-trained first model for at least one past data class as a first teacher (mental process; As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, this limitation encompasses designating a model as a first teacher based upon a user opinion)
designating the trained second model as a second teacher (mental process; As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, this limitation encompasses designating a model as a second teacher based upon a user opinion)
performing dual-teacher information distillation by maximizing mutual information… of the first teach and second teacher (mental process; As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, this limitation encompasses performing an evaluation of information observed from the first and second teach to maximize mutual information)
a combined student model configured to perform classification on the past data class and the at least one new data class, wherein the combined student model is configured to… generate a classification weight of the at least one new data class (mental process; As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, this limitation encompasses calculating a classification weight based upon the evaluation of a classification of data based upon the maximizing mutual information)
performing the dual-teacher information distillation further comprises (mental process; As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, the dual-teacher information distillation is defined by the following “generating…”, “applying…,” “applying…,” and “determining” steps. These four steps are identified as mental processes, thus, the performing the dual-teach information distillation is a mental process implemented by observation and judgement (“generating…”), evaluation (“applying…”), evaluation (“applying…”), and evaluation (“determining…”))
generating a first set of synthetic samples for a first class at a first time and a second set of synthetic samples at a second time (mental process; As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, this limitation encompasses performing, by a user, a generation of first set of synthetic samples at a first time and a second set of synthetic samples at a second time. These synthetic data samples may be generated in the mind by a user for based upon a user observation and judgement of the first and second classes)
applying data-free generative replay to generate a first set of synthetic samples… for a first class at a time (mental process; As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, this limitation encompasses performing, by a user, a generation of first synthetic samples based upon an evaluation of the first class at a first time)
applying data-free generative reply to generate a second set of synthetic samples… for a second class at a second time, wherein the second time is after the first time (mental process; As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, this limitation encompasses performing, by a user, a generation of second synthetic samples based upon an evaluation at a second time)
determining a dual-teacher information distillation loss based on the first set of synthetic samples and the second set of synthetic samples (mental process; As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, this limitation encompasses performing an evaluation to calculate a loss between the two sets of samples)
Step 2A, Prong 2:
Accordingly, after determining that a claim recites a judicial exception in Step 2A Prong One, examiners should evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception in Step 2A Prong Two. A claim that integrates a judicial exception into a practical application will apply, rely on, or use the judicial exception in a manner that imposes a meaningful limit on the judicial exception, such that the claim is more than a drafting effort designed to monopolize the judicial exception (MPEP 2106.04(d)).
The claims disclose the following additional limitations:
training a second model using at least one new data class
intermediate layers
a first conditional generator and a second conditional generator
These limitations amount to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
The claim further recites the additional elements
transferring the information jointly from both the first teach and the second teacher
and incrementally collect a feature extractor output
The judicial exception is not integrated into a practical application because it is directed toward extra-solution activity of gathering data for use in the claimed process. As described in MPEP 2106.05(g), limitations that amount to merely adding insignificant extra-solution activity to a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
Additionally, claim 10 recites the element:
an electronic device comprising a non-transitory computer readable memory and a processor, wherein the processor executes the instructions stored in the non-transitory computer readable memory
These limitations recite using an electronic device having a processor and non-transitory computer readable memory as a tool to perform the abstract idea. This is not indicative of integration into a practical application. This amounts to merely reciting the words “apply it” to the judicial exception (Alice Corp. v. CLS Bank, 573 US 208, 221, 110 USPQ2d 1976, 1982-83).
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do no integrate the judicial exception into a practical application.
Step 2B:
Based on the determination in Step 2A of the analysis that the claims are directed toward a judicial exception, in must be determined if any claims contain any element or combination of elements sufficient to ensure that the claims amount to significantly more than the judicial exception (Step 2B).
The claims disclose the following additional limitations:
training a second model using at least one new data class
intermediate layers
a first conditional generator and a second conditional generator
These limitations amount to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
The claim further recites the additional elements
transferring the information jointly from both the first teach and the second teacher
and incrementally collect a feature extractor output
The judicial exception is not integrated into a practical application because it is directed toward extra-solution activity of gathering data for use in the claimed process. As described in MPEP 2106.05(g), limitations that amount to merely adding insignificant extra-solution activity to a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application. Additionally, the courts have found limitations directed to obtaining information electronically, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II), “receiving or transmitting data over a network”, "electronic record keeping," and "storing and retrieving information in memory").
Finally, claim 10 recites the element:
an electronic device comprising a non-transitory computer readable memory and a processor, wherein the processor executes the instructions stored in the non-transitory computer readable memory
These limitations recite using an electronic device having a processor and non-transitory computer readable memory as a tool to perform the abstract idea. This is not indicative of integration into a practical application. This amounts to merely reciting the words “apply it” to the judicial exception (Alice Corp. v. CLS Bank, 573 US 208, 221, 110 USPQ2d 1976, 1982-83).
In this instance, after considering all claim elements individually and as an ordered combination, it is determined that the claims do not include any additional elements that are sufficient to amount to significantly more than the judicial exception.
Claims 2 and 11:
With respect to dependent claims 2 and 11, the claims depend upon independent claims 1 and 10, respectively. The analysis of claims 1 and 10 is incorporated herein by reference.
Step 2A, Prong 2:
Accordingly, after determining that a claim recites a judicial exception in Step 2A Prong One, examiners should evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception in Step 2A Prong Two. A claim that integrates a judicial exception into a practical application will apply, rely on, or use the judicial exception in a manner that imposes a meaningful limit on the judicial exception, such that the claim is more than a drafting effort designed to monopolize the judicial exception (MPEP 2106.04(d)).
The claims disclose the following additional limitations:
training at least one of a first conditional generator or a second conditional generator to generate synthetic data, given the first model or the second model, without using any stored training data, wherein the synthetic data is configured to mimic training data used to train the first teacher or the second teacher
These limitations amount to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do no integrate the judicial exception into a practical application.
Step 2B:
Based on the determination in Step 2A of the analysis that the claims are directed toward a judicial exception, in must be determined if any claims contain any element or combination of elements sufficient to ensure that the claims amount to significantly more than the judicial exception (Step 2B).
The claims disclose the following additional limitations:
training at least one of a first conditional generator or a second conditional generator to generate synthetic data, given the first model or the second model, without using any stored training data, wherein the synthetic data is configured to mimic training data used to train the first teacher or the second teacher
These limitations amount to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
In this instance, after considering all claim elements individually and as an ordered combination, it is determined that the claims do not include any additional elements that are sufficient to amount to significantly more than the judicial exception.
Claims 3 and 12:
With respect to dependent claims 3 and 12, the claims depend upon dependent claims 2 and 11, respectively. The analysis of claims 2 and 11 is incorporated herein by reference.
Step 2A, Prong 1:
With respect to claims 3 and 12, the claims recite the elements:
determining a cross-entropy loss between a label input… and a value output from the first teacher or the second teacher (mental process; As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, this limitation encompasses performing an evaluation of cross-entropy loss and output from the first or second teacher)
determining a batch-normalization statistics loss by matching means and variance variables stored in [memory]…of the first teacher or the second teacher with mean and variance variables computed at the [memory]… of the first teacher or the second teacher (mental process; As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, this limitation encompasses performing an evaluation of variance variables to determine a batch-normalization loss)
incrementally adjusting… to account for the cross-entropy loss and… statistics loss (mental process; As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, this limitation encompasses performing a judgment to incrementally adjust a process of information distillation based upon observed data and the evaluation of loss)
Step 2A, Prong 2:
Accordingly, after determining that a claim recites a judicial exception in Step 2A Prong One, examiners should evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception in Step 2A Prong Two. A claim that integrates a judicial exception into a practical application will apply, rely on, or use the judicial exception in a manner that imposes a meaningful limit on the judicial exception, such that the claim is more than a drafting effort designed to monopolize the judicial exception (MPEP 2106.04(d)).
The claims disclose the following additional limitations:
conditional generator
batch-normalization layers
These limitations amount to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
Further, the claims recite the additional element:
information output from the conditional generator
This limitation amounts to extra solution activity because it is a mere nominal or tangential addition to the claim, amounting to mere data output (see MPEP 2106.05(g)).
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do no integrate the judicial exception into a practical application.
Step 2B:
Based on the determination in Step 2A of the analysis that the claims are directed toward a judicial exception, in must be determined if any claims contain any element or combination of elements sufficient to ensure that the claims amount to significantly more than the judicial exception (Step 2B).
The claims disclose the following additional limitations:
conditional generator
batch-normalization layers
These limitations amount to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
Further, the claims recite the additional element:
information output from the conditional generator
This limitation amounts to extra solution activity because it is a mere nominal or tangential addition to the claim, amounting to mere data output (see MPEP 2106.05(g)). Additionally, the courts have similarly found limitations directed to displaying a result, recited at a high level of generality, to be well-understood, routine, and conventional. See (MPEP 2106.05(d)(II), "presenting offers and gathering statistics.", “determining an estimated outcome and setting a price”).
In this instance, after considering all claim elements individually and as an ordered combination, it is determined that the claims do not include any additional elements that are sufficient to amount to significantly more than the judicial exception.
Claims 4 and 13:
With respect to dependent claims 4 and 13, the claims depend upon independent claims 1 and 10, respectively. The analysis of claims 1 and 10 is incorporated herein by reference.
Step 2A, Prong 2:
Accordingly, after determining that a claim recites a judicial exception in Step 2A Prong One, examiners should evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception in Step 2A Prong Two. A claim that integrates a judicial exception into a practical application will apply, rely on, or use the judicial exception in a manner that imposes a meaningful limit on the judicial exception, such that the claim is more than a drafting effort designed to monopolize the judicial exception (MPEP 2106.04(d)).
The claims disclose the following additional limitations:
wherein the… model… is updated using weigh imprinting by accessing stored training data
These limitations amount to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do no integrate the judicial exception into a practical application.
Step 2B:
Based on the determination in Step 2A of the analysis that the claims are directed toward a judicial exception, in must be determined if any claims contain any element or combination of elements sufficient to ensure that the claims amount to significantly more than the judicial exception (Step 2B).
The claims disclose the following additional limitations:
wherein the… model… is updated using weigh imprinting by accessing stored training data
These limitations amount to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
In this instance, after considering all claim elements individually and as an ordered combination, it is determined that the claims do not include any additional elements that are sufficient to amount to significantly more than the judicial exception.
Claims 5 and 14:
With respect to dependent claims 5 and 14, the claims depend upon independent claims 1 and 10, respectively. The analysis of claims 1 and 10 is incorporated herein by reference.
Step 2A, Prong 2:
Accordingly, after determining that a claim recites a judicial exception in Step 2A Prong One, examiners should evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception in Step 2A Prong Two. A claim that integrates a judicial exception into a practical application will apply, rely on, or use the judicial exception in a manner that imposes a meaningful limit on the judicial exception, such that the claim is more than a drafting effort designed to monopolize the judicial exception (MPEP 2106.04(d)).
The claims disclose the following additional limitations:
wherein the trained… model… is trained by using a “none” class in response to the training data not being accessible
These limitations amount to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do no integrate the judicial exception into a practical application.
Step 2B:
Based on the determination in Step 2A of the analysis that the claims are directed toward a judicial exception, in must be determined if any claims contain any element or combination of elements sufficient to ensure that the claims amount to significantly more than the judicial exception (Step 2B).
The claims disclose the following additional limitations:
wherein the trained… model… is trained by using a “none” class in response to the training data not being accessible
These limitations amount to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
In this instance, after considering all claim elements individually and as an ordered combination, it is determined that the claims do not include any additional elements that are sufficient to amount to significantly more than the judicial exception.
Claims 6 and 15:
With respect to dependent claims 6 and 15, the claims depend upon independent claims 1 and 10, respectively. The analysis of claims 1 and 10 is incorporated herein by reference.
Step 2A, Prong 1:
The claims recite the elements:
accounting for the dual-teacher information distillation loss when performing the dual-teacher information distillation (mental process; As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, this limitation encompasses performing an evaluation to account for distillation loss when performing information distillation)
Claims 7 and 16:
With respect to dependent claims 7 and 16, the claims depend upon dependent claims 2 and 10, respectively. The analysis of claims 2 and 11 is incorporated herein by reference.
Step 2A, Prong 2:
Accordingly, after determining that a claim recites a judicial exception in Step 2A Prong One, examiners should evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception in Step 2A Prong Two. A claim that integrates a judicial exception into a practical application will apply, rely on, or use the judicial exception in a manner that imposes a meaningful limit on the judicial exception, such that the claim is more than a drafting effort designed to monopolize the judicial exception (MPEP 2106.04(d)).
The claims disclose the following additional limitations:
wherein training the first conditional generator or the second conditional generator further comprises using a pre-trained model to generate the synthetic data to that is used to train the first conditional generator or the second conditional generator without using any stored training data
These limitations amount to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do no integrate the judicial exception into a practical application.
Step 2B:
Based on the determination in Step 2A of the analysis that the claims are directed toward a judicial exception, in must be determined if any claims contain any element or combination of elements sufficient to ensure that the claims amount to significantly more than the judicial exception (Step 2B).
The claims disclose the following additional limitations:
wherein training the first conditional generator or the second conditional generator further comprises using a pre-trained model to generate the synthetic data to that is used to train the first conditional generator or the second conditional generator without using any stored training data
These limitations amount to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
In this instance, after considering all claim elements individually and as an ordered combination, it is determined that the claims do not include any additional elements that are sufficient to amount to significantly more than the judicial exception.
Claims 8 and 17:
With respect to dependent claims 8 and 17, the claims depend upon independent claims 1 and 10, respectively. The analysis of claims 1 and 10 is incorporated herein by reference.
Step 2A, Prong 2:
Accordingly, after determining that a claim recites a judicial exception in Step 2A Prong One, examiners should evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception in Step 2A Prong Two. A claim that integrates a judicial exception into a practical application will apply, rely on, or use the judicial exception in a manner that imposes a meaningful limit on the judicial exception, such that the claim is more than a drafting effort designed to monopolize the judicial exception (MPEP 2106.04(d)).
The claims disclose the following additional limitations:
wherein the second model designated as the second teacher is trained with new data for each new class that is introduced
These limitations amount to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do no integrate the judicial exception into a practical application.
Step 2B:
Based on the determination in Step 2A of the analysis that the claims are directed toward a judicial exception, in must be determined if any claims contain any element or combination of elements sufficient to ensure that the claims amount to significantly more than the judicial exception (Step 2B).
The claims disclose the following additional limitations:
wherein the second model designated as the second teacher is trained with new data for each new class that is introduced
These limitations amount to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
In this instance, after considering all claim elements individually and as an ordered combination, it is determined that the claims do not include any additional elements that are sufficient to amount to significantly more than the judicial exception.
Claims 9 and 18:
With respect to dependent claims 9 and 18, the claims depend upon independent claims 1 and 10, respectively. The analysis of claims 1 and 10 is incorporated herein by reference.
Step 2A, Prong 2:
Accordingly, after determining that a claim recites a judicial exception in Step 2A Prong One, examiners should evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception in Step 2A Prong Two. A claim that integrates a judicial exception into a practical application will apply, rely on, or use the judicial exception in a manner that imposes a meaningful limit on the judicial exception, such that the claim is more than a drafting effort designed to monopolize the judicial exception (MPEP 2106.04(d)).
The claims disclose the following additional limitations:
wherein data output from the second teacher and data output from the first teacher are applied to the combined student model to perform dual-teacher information distillation
These limitations amount to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do no integrate the judicial exception into a practical application.
Step 2B:
Based on the determination in Step 2A of the analysis that the claims are directed toward a judicial exception, in must be determined if any claims contain any element or combination of elements sufficient to ensure that the claims amount to significantly more than the judicial exception (Step 2B).
The claims disclose the following additional limitations:
wherein data output from the second teacher and data output from the first teacher are applied to the combined student model to perform dual-teacher information distillation
These limitations amount to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
In this instance, after considering all claim elements individually and as an ordered combination, it is determined that the claims do not include any additional elements that are sufficient to amount to significantly more than the judicial exception.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 4, 8-10, 13, and 17-18 are rejected under 35 U.S.C. 103 as being unpatentable over Meng et al. (US 2020/0334538, published 22 October 2020, hereafter Meng) and further in view of Tao et al. (US 11620576, filed 22 June 2020, hereafter Tao) and further in view of Sainz de Cea et al. (US 2021/0035285, filed 19 August 2019, hereafter Sainz de Cea) and further in view of Sarpatwar et al. (US 2021/0397988, filed 22 June 2020, hereafter Sarpatwar) and further in view of Huang et al. (WO 2018/206504, published 15 November 2018, hereafter Huang).
As per independent claim 1, Meng discloses a method of performing class-incremental learning, the method comprising:
designating a pre-trained first model for at least one past data class as a first teacher (Figure 1A, item S110; paragraphs 0019-0020: Here, a teacher model is used to produce a teacher posterior representing training data. When the teacher posterior matches a ground truth label, the teacher posterior is used to train the student)
training a second model using at least one new data class (Figure 1A, item S120; paragraphs 0019-0020: Here, a student model is trained using at least one of the generated teacher posterior or the ground truth data (new data class))
designating the trained second model as a second teacher (Figure 1A, item S130; paragraphs 0019-0020 0058-0059: Here, the teach model and student model are used for training via forward and/or backward propagation)
performing dual-teacher information distillation by maximizing mutual information at intermediate layers of the first teach and second teacher (Figure 2C; paragraphs 0058-0059: Here, a plurality of posteriors are used to train the student model. The divergence is minimized (maximizing mutual information) between the posteriors and the student model)
determining a dual-teach information distillation loss based on the first set of samples and the second set of samples (paragraphs 0025-0030)
Meng fails to specifically disclose transferring the information jointly from both the first teacher and second teacher to a combined student model.
However, Tao, which is analogous to the claimed invention because it is directed toward transferring data from multiple teachers to a student model in a machine learning environment, discloses transferring the information jointly from both the first teacher and second teacher to a combined student model (Figure 1; column 4, line 31- column 5, line 5: Here, a plurality of teach models (Figure 1, item 115(1), 115(2), 115(n)) transfer knowledge to a student model (Figure 1, item 120)). It would have been obvious to one of ordinary skill in the art at the time of the applicant’s effective filing date to have combined Tao with Meng, with a reasonable expectation of success, as it would have allowed for creating a more robust student model. This would have provided a user with the advantage of leveraging various teachers to create a student model able to more accurately classify data.
Meng fails to specifically disclose performing classification on the past data class and the at least one new data class and wherein the combined student model is configured to incrementally collect a feature extractor output, and generate a classification weight of the at least one new data class based on the feature extractor output.
However, Sainz de Cea, which is analogous to the claimed invention because it is directed toward an adaptive classifier, discloses performing classification on the past data class and the at least one new data class, wherein the combined student model is configured to incrementally collect a feature extractor output, and generate a classification weight of the at least one new data class based on the feature extractor output (paragraph 0047: Here, a first classification is performed based on a prior (first) set of examination data and a second classification is performed based on a current (second) set of examination data to generate compressed multi-dimensional data. These data from these two sets of multi-dimensional data are then compared using quality measures (weights) to detect and classify outliers in the data). It would have been obvious to one of ordinary skill in the art at the time of the applicant’s effective filing date to have combined Sainz de Cea with Meng-Tao, with a reasonable expectation of success, as it would have allowed for detecting outliers within a set of combined classifications (Sainz de Cea: paragraph 0047). This would have facilitated an improved classification, as only data that is an outlier to the combined sets would be marked as an outlier.
Meng fails to specifically disclose:
applying data-free generative replay to generate a first set of synthetic samples with a first conditional generator for a first class at a first time
applying data-free generative replay to generate a second set of synthetic samples with a second conditional generator for a second class at a second time, wherein the second time is after the first time
However, Sarpatwar, which is analogous to the claimed invention because it is directed toward training teacher model using synthetic data, discloses:
applying data-free generative replay to generate a first set of synthetic samples with a first conditional generator for a first class at a first time (Figure 5; paragraphs 0006 and 0079-0080)
applying data-free generative replay to generate a second set of synthetic samples with a second conditional generator for a second class at a second time, wherein the second time is after the first time (Figure 5; paragraphs 0006 and 0079-0080)
It would have been obvious to one of ordinary skill in the art at the time of the applicant’s effective filing date to have combined Sarpatwar with Meng-Tao-Sainz de Cea, with a reasonable expectation of success, as it would have facilitated customizing synthetic data to account for problems with the original dataset. This would have provide the benefit of creating a model which satisfies domain constraints.
Meng fails to specifically disclose wherein the dual-teach information distillation further comprises generating a firs set of synthetic samples for a first class at a first time and a second set of synthetic samples for a second class at a second time. However, Huang, which is analogous to the claimed invention because it is directed toward training a reinforcement learning model using multiple sets of synthetic data, discloses wherein the dual-teach information distillation further comprises generating a firs set of synthetic samples for a first class at a first time and a second set of synthetic samples for a second class at a second time (page 2, lines 13-27: Here, a Generative Adversarial Network (GAN) is trained using data from the real environment. This training data includes a data slice corresponding to a state-reward pair and a state-action pair. A data generator trained with the first state-reward pair is used to generate a first set of synthetic data. The examiner considers this data be generated at the claimed “first time” and the state as being analogous to the claimed “first class.” A portion of the first synthetic data, generated at a first time, is processed to generate a resulting data slice. The first synthetic data corresponding to a second state-action pair and the result data slice corresponding to a second state-reward pair. The second state-action pair portion of the first synthetic data is merged with the second state-reward pair from the relations network to generate second synthetic data. This second synthetic data is generated at a “second time, wherein the second time is after the first time” because this second synthetic data is at least partially generated based on the first synthetic data generated at the “first time.” Further, the examiner considers the second state as analogous to the claimed “second class.” Further, a third synthetic data set may be generated from the first synthetic data and the second synthetic data (page 3, lines 3-4)). It would have been obvious to one of ordinary skill in the art at the time of the applicant’s effective filing date to have combined Huang with Meng-Tao-Sainz de Cea- Sarpatwar, with a reasonable expectation of success, as it would have allowed improving training of a reinforcement learning model by improving the quality of synthetic data used for training (Huang: page 7, lines 16-24).
As per dependent claim 2, Meng, Tao, Sainz de Cea, Sarpatwar, and Huang disclose the limitations similar to those in claim 1, and the same rejection is incorporated herein. Meng fails to specifically disclose training at least one of the first conditional generator or a second conditional generator or generate synthetic data, given the first model or the second model, without using any stored data, wherein the synthetic data is configured to mimic training data used to train the first teacher or the second teacher.
However, Sarpatwar, which is analogous to the claimed invention because it is directed toward training teacher model using synthetic data, discloses training at least one of the first conditional generator or a second conditional generator or generate synthetic data, given the first model or the second model, without using any stored data, wherein the synthetic data is configured to mimic training data used to train the first teacher or the second teacher (paragraph 0006).
It would have been obvious to one of ordinary skill in the art at the time of the applicant’s effective filing date to have combined Sarpatwar with Meng-Tao, with a reasonable expectation of success, as it would have facilitated training using synthetic data. This would have provide the benefit of creating a model which satisfies domain constraints).
As per dependent claim 3, Meng, Tao, Sainz de Cea, Sarpatwar, and Huang disclose the limitations similar to those in claim 2, and the same rejection is incorporated herein. Meng discloses:
determining a cross-entropy loss between a label input into the conditional generator and a value output from the first teacher or the second teacher (paragraphs 0020-0022)
determining a batch-normalization statistics loss by matching mean and variance variables stored in the batch-normalization layers of the first teacher or the second teach with mean and variance variables computed at the same batch-normalization layers of the first teacher or the second teacher for information output from the conditional generator
Meng fails to specifically disclose adjusting the conditional generator to account for the change in parameters. However, Sarpatwar, which is analogous to the claimed invention because it is directed toward training teacher model using synthetic data, discloses adjusting the conditional generator to account for the change in parameters (paragraph 0006). It would have been obvious to one of ordinary skill in the art at the time of the applicant’s effective filing date to have combined Sarpatwar with Meng-Tao, with a reasonable expectation of success, as it would have facilitated customizing synthetic data to account for problems with the original dataset. This would have provide the benefit of creating a model which satisfies domain constraints.
As per dependent claim 4, Meng discloses wherein the first model designated as the first teacher is updated using weight imprinting by accessing stored training data (paragraphs 0022-0023).
As per dependent claim 6, Meng, Tao, Sainz de Cea, Sarpatwar, and Huang disclose the limitations similar to those in claim 1, and the same rejection is incorporated herein. Meng discloses:
accounting for the dual-teacher information distillation loss when performing dual-teacher information distillation (paragraphs 0025-0030)
As per dependent claim 7, Meng, Tao, Sainz de Cea, Sarpatwar, and Huang disclose the limitations similar to those in claim 2, and the same rejection is incorporated herein. Sarpatwar discloses wherein training the first conditional generator or the second conditional generator further comprises a pre-trained model to generate the synthetic data that is used to train the first conditional generator or the second conditional generator without using any stored training data (Figure 5; paragraph 0006). It would have been obvious to one of ordinary skill in the art at the time of the applicant’s effective filing date to have combined Sarpatwar with Meng-Tao, with a reasonable expectation of success, as it would have facilitated customizing synthetic data to account for problems with the original dataset. This would have provide the benefit of creating a model which satisfies domain constraints.
As per dependent claim 8, Meng discloses wherein the second model designated as the second teacher is trained with new data for each new class that is introduced (Figures 2A-2B; paragraphs 0034-0035: Here, the model is adapted to each new domain).
As per dependent claim 9, Meng discloses wherein data output from the second teacher and data output from the first teacher are applied to the combined student model to perform dual-teach information distillation (Figure 2B; paragraph 0035).
With respect to claim 10, the applicant discloses the limitations similar to those in claim 1. Additionally, Meng discloses an electronic device comprising a non-transitory computer readable memory and a process (Figure 4). Claim 10 is similarly rejected.
With respect to claims 11-13 and 15-18, the applicant discloses the limitations substantially similar to those in claims 2-4 and 6-9, respectively. Claims 11-13 and 15-18 are similarly rejected.
Claims 5 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Meng, Tao, Sainz de Cea, Sarpatwar, and Huang, and further in view of Stone et al. (US 11640447, filed 18 April 2018, hereafter Stone).
As per dependent claim 5, Meng, Tao, Sainz de Cea, Sarpatwar, and Huang disclose the limitations similar to those in claim 1, and the same rejection is incorporated herein. Meng fails to specifically disclose wherein the trained second model designated as the second teacher is trained by using a “none” class in response to training data not being accessible.
However, Stone, which is analogous to the claimed invention because it is directed toward training a model classifier, discloses wherein the trained second model is trained by using a “none” class in response to training data not being accessible (column 11, line 52- column 12, line 2). It would have been obvious to one of ordinary skill in the art at the time of the applicant’s effective filing date to have combined Stone with Wang, with a reasonable expectation of success, as it would have allowed for assigning values to each feature. This would have insured that all fields in the model are filled in order to improve classification.
With respect to claim 14, the applicant discloses the limitations substantially similar to those in claim 5. Claim 14 is similarly rejected.
Response to Arguments
Applicant's arguments filed with respect to the rejection of claims under 35 USC 101 have been fully considered but they are not persuasive.
With respect to the rejection of claims under 35 USC 101, the applicant argues that the claims recite features that reflect a specific technical improvement in class-incremental learning systems that address the challenge of catastrophic forgetting through architectural and procedural innovations not feasible to perform mentally (page 9).
To support this position, the applicant argues that under Step 2A, Prong 1, the claims are directed toward “Maximizing mutual information between two models at intermediate neural network layers involves numerical gradient computation, matrix operations, and tensor backpropagation, none of which are practical outside of a computer-implemented setting (page 10).” However, the examiner notes that the present rejection does not allege that maximizing mutual information between two models at intermediate neural network layers involves numerical gradient computation, matrix operations, and tensor backpropagation, none of which are practical outside of a computer-implemented setting under Step 2A, Prong 1. Further, the claims do not appear to recite maximizing mutual information between two models at intermediate neural network layers involves numerical gradient computation, matrix operations, and tensor backpropagation, none of which are practical outside of a computer-implemented setting. Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). For these reasons, this argument is not persuasive.
The applicant further argues that under Step 2A, Prong 2 the claims recite an improvement to the functioning of a computer or to another technology or technical field as explained in MPEP 2106.05(a)(II) (page 10). To support this assertion, the applicant argues that the improvement is achieved by mitigating catastrophic forgetting and the specific technical implementation includes dual-teach information distillation combined with incremental feature-based weight generation to improve model robustness and eliminates the need for retraining on full historical data, thereby enhancing computational efficiency and memory usage (page 10).
The examiner notes that the claim recites the additional elements:
training a second model using at least one new data class
intermediate layers
a first conditional generator and a second conditional generator
These limitations amount to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
The claim further recites the additional elements
transferring the information jointly from both the first teach and the second teacher
and incrementally collect a feature extractor output
The judicial exception is not integrated into a practical application because it is directed toward extra-solution activity of gathering data for use in the claimed process. As described in MPEP 2106.05(g), limitations that amount to merely adding insignificant extra-solution activity to a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application. Specifically, the claims appear to invoke computers or other machinery merely as a tool to perform the existing abstract idea. For this reason, this argument is not persuasive.
Finally, under Step 2B, the applicant argues that the claim as a whole includes an inventive concept (page 10). To support this assertion, the applicant argues that the imprinted classification weight derived from normalized feature outputs (Specification: paragraphs 0063-0065) is a novel solution that allows the student model to incorporated new classes without catastrophic forgetting, thereby preserving performance on old classes without access to prior training (page 11).
However, the examiner notes that the claims fail to recite the limitation of imprinting classification weight derived from feature outputs. Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). For these reasons, this argument is not persuasive.
Applicant’s arguments with respect to the rejection of claims 1-18 under 35 USC 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground of rejection is made in view of Meng, Tao, Sainz de Cea, Sarpatwar, and Huang.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Auerbach et al. (US 11763176): Discloses generating a plurality of sets of synthetic training data (column 31, lines 3-37)
Hazard et al. (US 2021/0064018): Discloses generating synthetic data samples for a reinforcement learning process (paragraph 0078)
Lewis et al. (US 2020/0364347): Discloses adversarial reinforcement learning including one or more synthetic data generators (Figure 1A; paragraph 0063)
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KYLE R STORK whose telephone number is (571)272-4130. The examiner can normally be reached 8am - 2pm; 4pm - 6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Omar Fernandez Rivas can be reached at 571/272-2589. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KYLE R STORK/Primary Examiner, Art Unit 2128