DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Applicants’ claim for the benefit of a prior-filed application under 35 U.S.C. 119(e) or under 35 U.S.C. 120, 121, 365(c), or 386(c) is acknowledged. The present application claims foreign priority based on Korean Patent Application No. 10-2022-0147423 filed November 7, 2022. The examiner notes that a certified copy (in Korean) of the above-noted application was received on March 24, 2023.
Information Disclosure Statement
Acknowledgment is made of the information disclosure statements filed March 24, 2023, which comply with 37 CFR 1.97. As such, the information disclosure statements have been placed in the application file and the information referred to therein has been considered by the examiner.
Response to Amendment
This office action is final and in response to the amendment filed on March 20, 2026, which was in response to the non-final office action mailed January 23, 2026.
Claims 1-11 and 13 are pending and have been examined. Claims 1-11 and 13 are rejected.
Claim Objections
The numbering of claims is not in accordance with 37 CFR 1.126 which requires the original numbering of the claims to be preserved throughout the prosecution. When new claims are presented, they must be numbered consecutively beginning with the number next following the highest numbered claims previously presented (whether entered or not).
Misnumbered Claim 13 should be renumbered as Claim 12.
Claim Rejections - 35 USC § 101
35 U.S.C. § 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-11 and 13 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
According to the USPTO guidelines, a claim is directed to non-statutory subject matter if:
Step 1: The claim does not fall within one of the four statutory categories of invention (process, machine, manufacture, or composition of matter), or,
Step 2: The claim recites a judicial exception, e.g. an abstract idea, without reciting additional elements that amount to significantly more than the judicial exception, as determined using the following analysis:
Step 2A, Prong 1: Does the claim recite an abstract idea, law of nature, or natural phenomenon?
Step 2A, Prong 2: Does the claim recite additional elements that integrate the judicial exception into a practical application?
Step 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception?
MPEP 2106.04(a)(2)(I) states: “The mathematical concepts grouping is defined as mathematical relationships, mathematical formulas or equations, and mathematical calculations.”
MPEP 2106.04(a)(2)(III) states: “Accordingly, the “mental processes” abstract idea grouping is defined as concepts performed in the human mind, and examples of mental processes include observations, evaluations, judgements, and opinions.
Further, the MPEP states: “The courts do not distinguish between mental processes that are performed entirely in the human mind and mental processes that require a human to use a physical aid (e.g. pen and paper or a slide run) to perform the claim limitation.
Using the two-step inquiry, it is clear that Claims 1-11 and 13 are each directed to an abstract idea as shown below:
With respect to Claim 1:
Step 1: Yes, Claim 1 recites a learning method, also known as a process, which is one of the four statutory categories of patentable subject matter.
Step 2A, Prong 1: Yes, a judicial exception is recited in this claim as it recites mathematical calculations:
“adding noise to weights of the first neural network”; The claim is explicitly reciting that math is being performed (adding) and this is consistent with the specification at [0035].
“calculating a loss function using the first output data, the second output data, and a true value corresponding to the input data”; The claim is explicitly reciting a math calculation. A loss function is a mathematical equation.
“updating weights of the second neural network based on calculation of the loss function”; The claim is explicitly reciting that math is being performed (updating weights based on a mathematical calculation) and this is consistent with the specification at [0042], [0043], and [0051].
“and determining weights of the first neural network by adding noise to updated weights of the second neural network”; Determining weights of a first neural network by adding noise to updated weights of a second neural network is explicitly reciting mathematical calculations being performed and is consistent with the specification at [0036] and [0037].
Step 2A, Prong 2: Furthermore, MPEP 2106.05(g) Insignificant Extra-Solution Activity has found mere data gathering and post-solution activity to be insignificant extra-solution activity:
“A learning method of a neural network, the learning method comprising: preparing a second neural network having the same weights as a first neural network which is pre-trained”; There is no description in the claim itself, or in the specification, on what is meant by “preparing” the second neural network. Thus, this limitation amounts to covering any method of “preparing” a neural network with no restriction on how that is accomplished and no description of the mechanism used for “preparing”, thus, this only amounts to “apply it” and mere instructions to implement an abstract idea on a computer since preparing a second neural network having the same weights as a pre-trained first neural network means using a computer as a tool to perform an abstract idea - see MPEP 2106.05(f)(1).
“generating a first output data of the first neural network and generating a second output data of the second neural network by providing input data to the first neural network and the second neural network”; This part of the claim only amounts to using generically recited neural networks in their ordinary capacity (e.g. receiving input data and generating output data accordingly) in accordance with MPEP 2106.05(f)(2).
Step 2B: No, the additional elements of Claim 1 do not provide significantly more than the abstract idea itself. Preparing a second neural network with the same weights as a first pre-trained neural network only amounts to “apply it” and mere instructions to implement an abstract idea on a computer - see MPEP 2106.05(f)(1). Providing input data and generating output data are well-understood, routine, and conventional activity of transmitting or receiving data over a network - see MPEP 2106.05(d).
With respect to Claim 2:
Step 2A, Prong 1: Inherits the limitations and abstract ideas from Claim 1. An additional judicial exception is recited in this claim as it recites a mathematical calculation:
“calculating a first loss function using the first output data and the true value;
calculating a second loss function using the first output data and the second output data;
calculating a third loss function using the second output data and the true value;
combining the first loss function, the second loss function, and the third loss function.”; Recites a calculation of loss functions, which are mathematical calculations that quantify the difference between a model’s predicted output and the actual true values.
Step 2A, Prong 2: The claim does not recite additional elements that integrate the judicial exception into a practical application.
Step 2B: The claim does not recite additional elements that amount to significantly more than the judicial exception.
With respect to Claim 3:
Step 2A, Prong 1: Inherits the limitations and abstract ideas from Claim 2. An additional judicial exception is recited in this claim as it recites a mathematical calculation:
“wherein the first loss function corresponds to a cross-entropy between the first output data and the true value, and third loss function corresponds to a cross-entropy between the second output data and the true value”; Recites a calculation of loss functions, which are mathematical calculations that quantify the difference between a model’s predicted output and the actual true values.
Step 2A, Prong 2: The claims do not recite additional elements that integrate the judicial exception into a practical application.
Step 2B: The claims do not recite additional elements that amount to significantly more than the judicial exception.
With respect to Claim 4:
Step 2A, Prong 1: Inherits the limitations and abstract ideas from Claim 2. An additional judicial exception is recited in this claim as it recites a mathematical calculation:
“wherein the second loss function corresponds to a Kullback-Leibler divergence function receiving a distribution generated from the first output data and a distribution generated from the second output data”; Recites a calculation of a loss function, specifically a Kullback-Leibler divergence function, which is a mathematical calculation that quantifies the information lost when one probability distribution is used to approximate a true distribution.
Step 2A, Prong 2: The claims do not recite additional elements that integrate the judicial exception into a practical application.
Step 2B: The claims do not recite additional elements that amount to significantly more than the judicial exception.
With respect to Claim 5:
Step 2A, Prong 1: Inherits the limitations and abstract ideas from Claim 4. An additional judicial exception is recited in this claim as it recites a mathematical calculation:
“wherein the second loss function is determined according to the equation:
PNG
media_image1.png
61
521
media_image1.png
Greyscale
wherein Ldist is the second loss function, Y1 is the first output data, Y2 is the second output data, LKw is the Kullback-Leibler divergence function, and T is a temperature coefficient used to adjust the characteristics of the distributions used in the second loss function Ldist”; Recites a specific mathematical equation that can be mathematically calculated using the stated variables.
Step 2A, Prong 2: The claims do not recite additional elements that integrate the judicial exception into a practical application.
Step 2B: The claims do not recite additional elements that amount to significantly more than the judicial exception.
With respect to Claim 6:
Step 2A, Prong 1: Inherits the limitations and abstract ideas from Claim 2. An additional judicial exception is recited in this claim as it recites a mathematical calculation:
“wherein the first loss function, the second loss function, and the third loss function are linearly combined, and wherein a sum of a coefficient applied to the first loss function and a coefficient applied to the second loss function is equal to 1”; Recites a linear combination of loss functions and the sum of a coefficient applied to the first and second loss function equal to 1, which are a series of mathematical calculations that combine mathematical objects (i.e. functions and variables) by multiplying each by a constant scalar and adding the results together.
Step 2A, Prong 2: The claims do not recite additional elements that integrate the judicial exception into a practical application.
Step 2B: The claims do not recite additional elements that amount to significantly more than the judicial exception.
With respect to Claim 7:
Step 2A, Prong 1: Inherits the limitations and abstract ideas from Claim 1.
Step 2A, Prong 2: The following is a generic link according to MPEP 2106.05(h):
“wherein the first neural network is identical to the second neural network”; This claim element sets forth that the first neural network is identical to the second neural network. This just further describes what the second neural network is, and does not add a limitation that would amount to a practical application or significantly more.
Step 2B: The following is a generic link according to MPEP 2106.05(h):
“wherein the first neural network is identical to the second neural network”; This claim element sets forth that the first neural network is identical to the second neural network. This just further describes what the second neural network is, and does not add a limitation that would amount to a practical application or significantly more.
With respect to Claim 8:
Step 2A, Prong 1: Inherits the limitations and abstract ideas from Claim 1. An additional judicial exception is recited in this claim as it recites a mathematical calculation:
“wherein during the generating of the first output data and the second output data, noise is not applied to weights of the second neural network”; Not applying noise to weights of a second neural network during the generating of a first output data and second output data is considered a mathematical calculation and this is consistent with the specification at [0034] and [0035].
Step 2A, Prong 2: The claim does not recite additional elements that integrate the judicial exception into a practical application.
Step 2B: The claim does not recite additional elements that amount to significantly more than the judicial exception.
With respect to Claim 9:
Step 2A, Prong 1: Inherits the limitations and abstract ideas from Claim 1. An additional judicial exception is recited in this claim as it recites a mathematical calculation:
“wherein the noise added to the weights of the first neural network is determined using a result of modeling noise generated in a hardware accelerator”; Noise added to the weights of a first neural network using a result of modeling noise generated in a hardware accelerator is considered a mathematical calculation because noise generated in a hardware accelerator and being added to weights of a neural network is performed using a mathematical formula, which is consistent with the specification at [0035] and [0036].
Step 2A, Prong 2: The claim does not recite additional elements that integrate the judicial exception into a practical application.
Step 2B: The claim does not recite additional elements that amount to significantly more than the judicial exception.
With respect to Claim 10:
Step 2A, Prong 1: Inherits the limitations and abstract ideas from Claim 1. An additional judicial exception is recited in this claim as it recites a mathematical calculation:
“wherein new values of the noise are generated each time the adding of noise to the weights
of the first neural network is performed”; Generating new values of noise each time the adding of noise to the weights of a first neural network is performed is considered a mathematical calculation because new values of noise arise from noise being added to weights. This is consistent with the specification at [0036].
Step 2A, Prong 2: The claim does not recite additional elements that integrate the judicial exception into a practical application.
Step 2B: The claim does not recite additional elements that amount to significantly more than the judicial exception.
With respect to Claim 11:
Step 2A, Prong 1: Inherits the limitations and abstract ideas from Claim 1.
Step 2A, Prong 2: The claim does not recite additional elements that integrate the judicial exception into a practical application:
“wherein the weights of the first neural network and the second neural network are adjusted
so that the first output data and the second output data become identical”; Adjusting the weights of a first neural network and a second neural network so that a first output data and a second output data become identical only amounts to “apply it” and mere instructions to implement an abstract idea on a computer - see MPEP 2106.05(f)(1).
Step 2B: The claim does not recite additional elements that amount to significantly more than the judicial exception. Adjusting the weights of a first neural network and a second neural network so that a first output data and a second output data become identical only amounts to “apply it” and mere instructions to implement an abstract idea on a computer - see MPEP 2106.05(f)(1).
With respect to Claim 13:
Step 1: Yes, Claim 13 recites a learning method, also known as a process, which is one of the four statutory categories of patentable subject matter.
Step 2A, Prong 1: Yes, a judicial exception is recited in this claim as it recites mathematical calculations:
“adding noise to weights of the first neural network”; The claim is explicitly reciting that math is being performed (adding) and this is consistent with the specification at [0035].
“calculating a loss function using the first output data, the second output data, and a true value corresponding to the input data”; The claim is explicitly reciting a math calculation. A loss function is a mathematical equation.
“calculating a first loss function using the first output data and the true value;”; Recites a calculation of loss functions, which are mathematical calculations that quantify the difference between a model’s predicted output and the actual true values.
“calculating a second loss function using the first output data and the second output data;”; Recites a calculation of loss functions, which are mathematical calculations that quantify the difference between a model’s predicted output and the actual true values.
“calculating a third loss function using the second output data and the true value;”; Recites a calculation of loss functions, which are mathematical calculations that quantify the difference between a model’s predicted output and the actual true values.
“combining the first loss function, the second loss function, and the third loss function”; Recites a calculation of loss functions, which are mathematical calculations that quantify the difference between a model’s predicted output and the actual true values.
“wherein the second loss function corresponds to a Kullback-Leibler divergence function receiving a distribution generated from the first output data and a distribution generated from the second output data”; Recites a calculation of a loss function, specifically a Kullback-Leibler divergence function, which is a mathematical calculation that quantifies the information lost when one probability distribution is used to approximate a true distribution.
“wherein the second loss function is determined according to the equation:
PNG
media_image1.png
61
521
media_image1.png
Greyscale
wherein Ldist is the second loss function, Y1 is the first output data, Y2 is the second output data, LKw is the Kullback-Leibler divergence function, and T is a temperature coefficient used to adjust the characteristics of the distributions used in the second loss function Ldist”; Recites a specific mathematical equation that can be mathematically calculated using the stated variables.
Step 2A, Prong 2: Furthermore, MPEP 2106.05(g) Insignificant Extra-Solution Activity has found mere data gathering and post-solution activity to be insignificant extra-solution activity:
“A learning method of a neural network, the learning method comprising: preparing a second neural network having the same weights as a first neural network which is pre-trained”; There is no description in the claim itself, or in the specification, on what is meant by “preparing” the second neural network. Thus, this limitation amounts to covering any method of “preparing” a neural network with no restriction on how that is accomplished and no description of the mechanism used for “preparing”, thus, this only amounts to “apply it” and mere instructions to implement an abstract idea on a computer since preparing a second neural network having the same weights as a pre-trained first neural network means using a computer as a tool to perform an abstract idea - see MPEP 2106.05(f)(1).
“generating a first output data of the first neural network and generating a second output data of the second neural network by providing input data to the first neural network and the second neural network”; This part of the claim only amounts to using generically recited neural networks in their ordinary capacity (e.g. receiving input data and generating output data accordingly) in accordance with MPEP 2106.05(f)(2).
Step 2B: No, the additional elements of Claim 1 do not provide significantly more than the abstract idea itself. Preparing a second neural network with the same weights as a first pre-trained neural network only amounts to “apply it” and mere instructions to implement an abstract idea on a computer - see MPEP 2106.05(f)(1). Providing input data and generating output data are well-understood, routine, and conventional activity of transmitting or receiving data over a network - see MPEP 2106.05(d).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e. changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
The following is a quotation of 35 U.S.C. § 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or non-obviousness.
Claim(s) 1-4 and 6-11 are rejected under 35 U.S.C. 103 as being unpatentable over “Distilling the Knowledge in a Neural Network” by Hinton et al., (non-patent literature published on March 9, 2015, hereinafter “Hinton”), in view of “Mean Teachers are Better Role Models: Weight-Averaged Consistency Targets Improve Semi-Supervised Deep Learning Results“ by Tarvainen et al., (non-patent literature published on April 16, 2018, hereinafter “Tarvainen”), in further view of “Noisy Machines: Understanding Noisy Neural Networks and Enhancing Robustness to Analog Hardware Errors Using Distillation” by Zhou et al., (non-patent literature published on January 14, 2020, hereinafter “Zhou”).
With respect to Claim 1:
Hinton teaches:
“A learning method of a neural network, the learning method comprising: preparing a second neural network having the same weights as a first neural network which is pre-trained;” (Page 1, Section 1 “Introduction”, teaches the existence of two neural network models, including a first model that is trained. The pre-trained first model can then be trained again, known as distillation, to transfer knowledge to a second model.)
“generating a first output data of the first neural network and generating a second output data of the second neural network by providing input data to the first neural network and the second neural network;” (Page 2, Section 1 “Introduction”, teaches the training of a cumbersome model, also known as a neural network, where a training set of input data is provided to a first neural network and generates soft targets, also known as a first output data, of a first neural network. The same training set or a separate “transfer” set is also provided as input data to a distilled model, also known as a second neural network, and generates soft targets, also known as a second output data, of a second neural network.)
“calculating a loss function using the first output data, the second output data, and a true value corresponding to the input data.” (Page 3, Section 2 “Distillation”, teaches a calculation of a weighted average, which is used as a loss function in machine learning, of two different objective functions. This loss function is calculated using soft targets (output data) from a neural network, soft targets (output data) from a distilled neural network, and the correct labels (true values) corresponding with the input data.)
“updating weights of the second neural network based on calculation of the loss function;”
(Page 3, Section 2 “Distillation”, recites a calculation of a weighted average, which is used as a loss function in machine learning, of two different objective functions. This loss function is calculated using soft targets (output data) from a first neural network, soft targets (output data) from a second neural network, and the correct labels (true values) corresponding with the input data. Page 3, Section 2.1 further recites each case in the transfer set contributing a cross-entropy gradient with respect to each logit of a neural network, where gradients are use during backpropagation to update the weights of a second neural network.)
Hinton does not appear to explicitly disclose:“A learning method of a neural network, the learning method comprising: preparing a second
neural network having the same weights as a first neural network which is pre-trained”
“adding noise to weights of the first neural network”
“and determining weights of the first neural network by adding noise to updated weights of
the second neural network”
However, Tarvainen teaches:
“A learning method of a neural network, the learning method comprising: preparing a second neural network having the same weights as a first neural network which is pre-trained” (Page 3, Section 2 recites the Temporal Ensembling method algorithm, where the teacher model’s weights are depicted as θ’ and the student model’s weights are depicted as θ. Within the algorithm, it uses θ’= θ, meaning there is a mathematical assumption that the teacher model’s weights and the student model’s weights are initially the same before the exponential moving average (EMA) method proceeds.)
“and determining weights of the first neural network by adding noise to updated weights of
the second neural network” (Page 3, Figure 2 recites the weights of a neural network are updated with gradient descent (updated weights of the second neural network), wherein another neural network’s weights are updated as an exponential moving average of the other neural network’s weights.)
It would have been obvious to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the present application to implement Claim 1 that utilized the teachings of Hinton and the teachings of Tarvainen, which are both in the same field of invention. A PHOSITA would have been motivated to take Hinton’s knowledge distillation framework and incorporate Tarvainen’s exponential moving average model into it in order to provide more accurate and stable neural network outputs (Tarvainen, Page 2, Section 1 “Introduction”), which would in turn improve the effectiveness of Hinton’s overall distillation process.
Hinton and Tarvainen combined do not appear to explicitly disclose:
“adding noise to weights of the first neural network”
“and determining weights of the first neural network by adding noise to updated weights of
the second neural network”
However, Zhou teaches:
“adding noise to weights of the first neural network” (Page 6, Section 5.1 recites the process of noise injection to the weights of a neural network (adding noise to weights of a neural network) during knowledge distillation in order to expose network training to a more realistic loss that occurs in real noisy analog devices.)
“and determining weights of the first neural network by adding noise to updated weights of
the second neural network” (Page 6, Section 5.1 from Zhou recites new weight values of layer “l” are drawn from a Gaussian distribution at each forward pass during a combined knowledge distillation and noise injection process. Weight values of a neural network (first neural network) are drawn from a Gaussian distribution based off of existing weight values, where the process consists of noise injection (determining weights by adding noise).)
It would have been obvious to a PHOSITA before the effective filing date of the present application to implement a method like Claim 1 that utilized the teachings of Hinton and Tarvainen with the teachings of Zhou, which are all in the same field of invention. A PHOSITA would have been motivated to apply the noise-addition technique of Zhou to the updating neural network weights through EMA technique in Hinton and Tarvainen’s combined framework in order to improve model stability and robustness to noise perturbations, including the reliability of generated neural network outputs between two neural networks.
With respect to Claim 2:
Hinton, Tarvainen, and Zhou combined teach:
“wherein calculating the loss function comprises: calculating a first loss function using the first output data and the true value;” (Page 1, Section 1 “Introduction” from Hinton teaches a cumbersome trained model, also known as the first trained model, is the same as a supervised pre-trained teacher model, which means an inherent loss function is computed using the first model’s output value and a true value before the knowledge distillation process begins.)
“calculating a second loss function using the first output data and the second output data;” (Page 3, Section 2 “Distillation” from Hinton teaches an objective function described as a cross entropy, which is a type of loss function, is calculated using the soft targets from the cumbersome model (first output data from the first model) and softmax of the distilled model (second output data from the second model).
“calculating a third loss function using the second output data and the true value;” (Page 3, Section 2 “Distillation” from Hinton teaches a second objective function described as a cross entropy, which is a type of loss function, is calculated using the logits in softmax of the distilled model (second output data from the second model) and the correct labels (the true value).
“combining the first loss function, the second loss function, and the third loss function.” (Page 3, Section 2 “Distillation” from Hinton teaches the use of “a weighted average of two different objective functions”. A weighted average is a combination of functions where each function contributes proportionally based on the assigned weight amount. Further, Hinton teaches the calculation of a first loss function during the training of the first model using the first model’s output data and a true value. Another loss function is calculated between the first model output data and second model output data. Another loss function is calculated between the second model output data and a true value. Hinton teaches combining multiple loss functions. It is inherently understood that combining three loss functions (as described in Claim 2) can be done instead of just two loss functions since combining multiple loss functions is a well-known phenomenon when training neural networks.)
With respect to Claim 3:
Hinton, Tarvainen, and Zhou combined teach:
“The learning method of claim 2, wherein the first loss function corresponds to a cross-entropy between the first output data and the true value,” (Page 1, Section 1 “Introduction” from Hinton teaches a cumbersome model that has been trained. A PHOSITA would understand that training a teacher model for classification using labeled data typically involves calculating a cross-entropy loss function between the teacher model’s output and the true values before the knowledge distillation process begins. So, Hinton teaches a first cross-entropy loss between the first output data (from the cumbersome model) and the true value.
“and third loss function corresponds to a cross-entropy between the second output data and the true value” (Page 3, Section 2 “Distillation” from Hinton teaches an objective function described as a cross entropy, which is a type of loss function, is calculated using the logits in softmax of the distilled model (second output data from the second model) and the correct labels (the true value).
With respect to Claim 4:
Hinton, Tarvainen, and Zhou combined teach:
“The learning method of claim 2, wherein the second loss function corresponds to a Kullback-Leibler divergence function receiving a distribution generated from the first output data and a distribution generated from the second output data” (Page 3, Section 2 “Distillation” from Hinton teaches a cross entropy between soft targets from the cumbersome model and soft targets from the distilled model. It is inherently understood that using cross-entropy with soft targets from a first model and softmax predictions from a second model is functionally equivalent to minimizing Kullback-Leibler divergence in knowledge distillation.)
With respect to Claim 6:
Hinton, Tarvainen, and Zhou combined teach:
“wherein the first loss function, the second loss function, and the third loss function are linearly combined, and wherein a sum of a coefficient applied to the first loss function and a coefficient applied to the second loss function is equal to 1.” (Pages 2 and 3, Section 2 “Distillation” from Hinton teach multiple loss functions combined as a weighted average (also known as a specific type of linear combination where the weights are non-negative numbers that sum to 1). It is inherently understood that including a third loss function and normalizing the coefficients so that their sum equals to one, akin to extending a function to include additional loss functions and normalizing weights within a loss function, is a well-known technique when it comes to training neural networks since they keep the combined loss function at a consistent scale to ensure no single loss dominates the training process.)
With respect to Claim 7:
Hinton, Tarvainen, and Zhou combined teach:
“The learning method of claim 1, wherein the first neural network is identical to the second neural network.” (Page 3, Section 2 “Figure 2” from Tarvainen recites the Mean Teacher method involving the updating of the teacher model’s weights as an exponential moving average of the student model’s weights. It is inherently understood that this method requires the teacher and student model to share the same underlying (identical) architecture because the teacher model’s weights are computed as an exponential moving average of the student model’s weights on the basis of each parameter. This averaging algorithm requires similar weight and layer structures between the models.)
With respect to Claim 8:
Hinton, Tarvainen, and Zhou combined teach:
“wherein during the generating of the first output data and the second output data, noise is not applied to weights of the second neural network” (Page 7, Section 5.1 from Zhou recites two neural networks that each produce (generate) output targets (data), where one of the neural networks does not have noise applied to their weights during training.)
With respect to Claim 9:
Hinton, Tarvainen, and Zhou combined teach:
“wherein the noise added to the weights of the first neural network is determined using a result of modeling noise generated in a hardware accelerator” (Page 4, Section 3 “NVM cells” from Zhou recites modeling generic NVM cell noise as a Gaussian additive zero-mean (modeling noise generated in a hardware accelerator) on the weights of a neural network model on each particular layer (noise added to the weights of a first neural network).)
With respect to Claim 10:
Hinton, Tarvainen, and Zhou combined teach:“wherein new values of the noise are generated each time the adding of noise to the weights
of the first neural network is performed” (Page 6, Section 5.1 from Zhou recites new weight values of layer “l” are drawn from a Gaussian distribution at each forward pass during a combined knowledge distillation and noise injection process. Each new weight value produces a new noise value, akin to new values of noise generated each time noise is added to the weights of a neural network.)
With respect to Claim 11:
Hinton, Tarvainen, and Zhou combined teach:
“wherein the weights of the first neural network and the second neural network are adjusted
so that the first output data and the second output data become identical” (Page 3, Figure 2 from Tarvainen recites a softmax output from a neural network model being compared with another neural network model’s output with consistency cost. After the weights of a first neural network model are updated (adjusted) with gradient descent, a second neural network model’s weights are updated (adjusted) as an exponential moving average of the first neural network model’s weights. This means utilizing gradient descent changes the weights of neural networks in order to reduce the difference between two different outputs from two neural network models until there is minimal gap, akin to the outputs becoming identical.)
Allowable Subject Matter
Claims 5 and 13 include allowable subject matter.
Response to Arguments
Applicant’s arguments filed on March 20, 2026 have been fully considered, but the examiner believes that not all are fully persuasive.
Claim Rejections - 35 USC § 101
The applicant argues the claims do not merely recite mathematical operations and are
instead integrated into a specific training methodology that produces a hardware-deployable neural network with improved robustness, which also constitutes a real-world neural network training application that integrates any alleged abstract concept into a practical application in Step 2A, Prong 2.
Examiner respectfully disagrees and maintains the 101 rejection (see above). Per MPEP 2106.05(a), improvements in the robustness of a predictive model do not constitute improvements to computer functionality. The claimed invention improves the quality of the mathematical outputs from the neural network model rather than the operation of the computer on which the model is being executed, and merely uses information associated with hardware noise as an input into the neural network training process consisting of mathematical operations. The claims do not recite an improvement to the operation of a hardware accelerator itself. Therefore, the abstract ideas are not integrated into a practical application in Step 2A, Prong 2.
B. The applicant argues that the claims reflects a technical improvement, particularly improving the functioning of neural networks deployed on hardware accelerators.
Examiner respectfully disagrees and maintains the 101 rejection (see above). Firstly, applicant does not provide arguments or reasons for why the claims do not fall under mathematical calculations, which are considered abstract ideas (see MPEP 2106.04). Additionally, per the December 2025 memorandum reciting the revision to MPEP 2106.04(d)(1) on Pages 2 and 3, if the specification sets forth an improvement in technology or a technical field, the claim must be evaluated to ensure that the claim itself disclose the improvement. Per applicant’s argument on page 6 of applicant’s remarks, “the synchronized weight update loop”, is not reflected in Claim 1 or any of its dependents.
Furthermore, the claims have been analyzed above as containing abstract ideas, therefore the improvement is currently being claimed as part of the judicial exception. It is important to keep in mind that an improvement in the abstract idea itself is not an improvement in technology and that the improvement needs to be provided by the one or additional elements.
C. The applicant argues that several claim limitations cannot be performed in the human mind since the claimed method requires training neural networks through iterative forward passes and backpropagation involving thousands to millions of parameters
Examiner respectfully disagrees and maintains the 101 rejection (see above). The limitations from the claims do not specifically state the requirement of “thousands to millions of parameters”, “synchronized weight matrices”, or “multi-dimensional matrix computations” (Page 7 of applicant’s remarks) in order to train neural networks through iterative forward passes and backpropagation. Therefore, Claim 1 can be interpreted as an abstract idea that can be performed in the human mind or with pen and paper.
Claim Rejections - 35 USC § 103
A. The applicant argues that the combination of Hinton, Cui, and Chakraborty does not teach the limitations of Claim 1 and its dependents (Claims 2-4 and 6-7).
Applicant’s arguments have been considered, but are moot because the new ground of rejection does not rely on the combination of references applied in the prior rejection of record for any teaching or matter specifically challenged in the argument, with the exception of applicant’s argument that the feature “updating weights of the second neural network based on calculation of the loss function” is not disclosed in Hinton (Page 3 of applicant’s remarks). Examiner respectfully disagrees. The feature “updating weights of the second neural network based on calculation of the loss function” is taught by Hinton. Page 3, Section 2 “Distillation” from Hinton recites a calculation of a weighted average, which is used as a loss function in machine learning, of two different objective functions. This loss function is calculated using soft targets (output data) from a first neural network, soft targets (output data) from a second neural network, and the correct labels (true values) corresponding with the input data. Page 3, Section 2.1 from Hinton further recites each case in the transfer set contributing a cross-entropy gradient with respect to each logit of a neural network, where gradients are used during backpropagation to update the weights of a second neural network by determining how much the weights should be modified in order to reduce the loss.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Vibha Bhat whose telephone number is (571)-272-7091. The examiner can normally be reached on Monday – Thursday from 8:00 AM to 5:00 PM EST and every other Friday from 8:00 AM to 4:00 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. See MPEP § 713.01. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at https://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Mariela Reyes, can be reached at telephone number (571)-270-1006. The fax phone number for the organization where this application or proceeding is assigned is (571)-273-8300. Information regarding the status of an application may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or (572)-272-1000.
/Vibha Bhat/Examiner
Art Unit 2142
/Mariela Reyes/Supervisory Patent Examiner, Art Unit 2142
Prosecution Insights