Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s argument filed 07/10/2026 have been fully considered but they are not persuasive. The amended claims have resolved the 112 issues mentioned in the Office Action (dated 06/12/2026). Please see below for additional 112 issues with the amended claims.
Applicant’s Argument: On page 10-14 of Applicant’s response to rejections under 35 U.S.C. § 101, applicant states amended claim 1 is not directed to an abstract idea of a mental process. Claim 1 recites a specific computational workflow operating one machine learning artifacts. Applicant argues that amended claim 1 integrates the judicial exception into a practical application and the amended claim 1 recites significantly more than the abstract idea. The specific technological improvement is based on the combination of generating a true label approximate value, training a server-side large model, and adaptively adjusting learning intensity.
Examiner’s Response: Applicant’s argument is not persuasive. During examination, the examiner should analyze the "improvements" consideration by evaluating the specification and the claims to ensure that a technical explanation of the asserted improvement is present in the specification, and that the claim reflects the asserted improvement (see MPEP §2106.05(a)). The MPEP (§2106.05(a)(II)) also warns, “it is important to keep in mind that an improvement in the abstract idea itself (e.g. a recited fundamental economic concept) is not an improvement in technology.” Here, the alleged improvement in the form of “processing latent vectors, generating probability distributions, and adjusting parameters of a loss function for optimization” is an improvement to the abstract idea of a mental process that can be performed in the human mind.
Claim 1 broadly recites the process in a manner that can be performed in the human mind with the aid of pen and paper. The claim limitation “generate a true label approximate value for learning a large model using the local information and a federated global parameter for a global model” does not provide any details on how the true label approximate value is generated. Similarly, the claim limitation “obtain an original prediction probability result by inputting the local latent vector into the global model corresponding to the federated global parameter” can be interpreted as processing a input to generate an output and the claim does not provide additional details to disclose how the result is determined. The claim limitation “generate the true label approximate value having an approximation degree to a true label by processing, based on the local loss value, a probability distribution or a probability value of the original prediction probability result so as to reflect how close the original prediction probability result is to the true label” is a mental process because the approximate value is an estimation that is based on a probability distribution and a loss value.
An important consideration in determining whether a claim improves technology is the extent to which the claim covers a particular solution to a problem or a particular way to achieve a desired outcome, as opposed to merely claiming the idea of a solution or outcome (see MPEP 2106.05(a)). The amended claims do not provide sufficient details to describe any technological improvement. If the specifications explicitly set forth an improvement but in a conclusory manner (see MPEP 2106.04(d)(1): a bare assertion of an improvement without the detail necessary to be apparent to a person of ordinary skill in the art), the examiner should not determine the claim improves technology.
Applicant’s Argument: On page 15-18 of Applicant’s response to rejections under 35 U.S.C. § 103, applicant states that none of the cited references in combination teaches the amended claims. Applicant argues that there is no articulated reason with a rational underpinning as to why the references would be combined together to teach the claimed invention.
Examiner’s Response: Applicant’s argument is not persuasive. Applicant’s arguments with respect to claim 1 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Dai (par. 3, 17-18) describes a federated learning system that aggregates the distribution of model information on a central server to better learn the data heterogeneity across the plurality of client devices. Dai (par. 22) describes providing updates to clients that benefits from collective knowledge without exposing private data to other clients. He (abstract, pg. 4, Section 3.3) discloses a federated learning system in which the client and server transfer knowledge bidirectionally to better learn the representation from local clients and the server. The federated learning system disclosed by He (pg. 4, Section 3.2) does not exchange raw data from the local client to preserve privacy. He also discloses (pg. 3, Section 3.1) that the labels are in the local dataset on the client and (pg. 4-5, Section 3.3, par. 4) that the local cross entropy loss is computed. It is implied that the cross-entropy loss is computed on the client machine and the server does not have the true label. He does not explicitly disclose that the local cross entropy loss is provided to the server to determine the optimization of the global model. Zeng (pg. 4, Section 4.1, par. 1-2) explicitly discloses that the cross-entropy loss is computed on the local data and the loss is transferred to the server with the local updates. In addition, Zeng (abstract) proposes a framework that improves the federated learning system in processing data heterogeneity across a plurality of clients. It would be obvious to a person having ordinary skills in the arts to combine Dai, He, and Zeng because the references do not teach apart but rather the references
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-13 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claims contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventors, at the time the application was filed, had possession of the claimed invention.
Claims 1, 12, and 14 recite “obtain an original prediction probability result by inputting the local latent vector into the global model corresponding to the federated global parameter”. There is no support from the Specification for the claim limitation. In the Specification (pg. 9, lines 11-14), the original prediction probability result is referred to as the local model prediction result. Thus, the original prediction probability result is obtained by the local model in the client devices and not by the global model on the server. Examiner interprets the claim limitation as the local model generates the original prediction probability result.
Claims 2-11 and 13 are dependent claims of claims 1 and 12, respectively. Thus, claims 2-11 and 13 are rejected on the same basis as parent claims 1 and 12.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-13 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 recites “earn the large model, using a loss function …” and it is not clear what constitutes as earning the large model. Examiner interprets the limitation as “learn the large model, using a loss function …”.
Claims 1, 12, and 14 recites “generate a true label approximate value for learning a large model”, and “generate the true label approximate value having an approximation degree”. The claims recite generating a true label approximate value twice and it is not clear from the claims if the true label approximate values are the same value or different values. It is not clear if claim 1 is reciting the same process of generating the true label approximate value or the step of generating the true label approximate value consists of two different generation process. Examiner interprets the limitation “generate the true label approximate value having an approximation degree …” to be a subset of the limitation of “generate a true label approximate value for learning a large model”. The Examiner is interpreting the generating a true label approximate and learning the large model to occur only once rather than occurring twice as recited. Thus, claim 1 can be interpreted as “generate a true label approximate value for learning a large model consist of: generate the true label approximate value having an approximation degree …”. Similarly, Examiner interprets “[l]earn the large model, using a loss function …” to be a subset of the limitation of “learn the large model using prediction results …” recited earlier in claim 1.
Claims 2-11 and 13 are dependent claims of claims 1 and 12, respectively. Thus, claims 2-11 and 13 are rejected on the same basis as parent claims 1 and 12.
Claim 2 recites “… and a local loss value obtained by inputting the local latent vector”. It is not clear whether the local loss value recited in claim 2 is the same or different value as recited in claim 1. Examiner interprets the local loss value recited in claim 2 to be the same value recited in claim 1.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-15 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Subject Matter Eligibility Analysis Step 1:
Claims 1-11 recite a system (“An apparatus for augmenting knowledge using federated learning information”), one of the four statutory categories of patentable subject matter.
Claims 12-13 does not recites one of the four statutory categories of patentable subject matter. Claim 12 recites “A system for augmenting knowledge using federated learning information comprising” wherein the claimed system is software per se. The claims recite a system comprising "a plurality of individual devices” and “a server” but these limitations are not specifically disclosed in the specification as being software or hardware elements. Under the broadest reasonable interpretation, these claim elements can be a virtualized environment. Therefore, the claimed system under the broadest reasonable interpretation can include only software elements and therefore software per se.
Claims 14 recite a process (“A method for augmenting knowledge using federated learning information”), one of the four statutory categories of patentable subject matter. Claims 15 recite an article of manufacture (“A non-transitory computer-readable medium”), one of the four statutory categories of patentable subject matter.
Regarding Claim 1:
Subject Matter Eligibility Analysis Step 2A Prong 1:
“generate a true label approximate value for learning a large model using the local information and a federated global parameter for a global model” (a mental process that can be performed in the human mind with the aid of pen and paper, i.e. judgement)
“generate the true label approximate value having an approximation degree to a true label by processing, based on the local loss value, a probability distribution or a probability value of the original prediction probability result so as to reflect how close the original prediction probability result is to the true label” (a mathematical calculation, see pg. 10, lines 3-13 in Specification)
“[l]earn the large model, using a loss function reflecting a distance between the true label approximate value and a prediction result of the large model, while adjusting a learning intensity of the large model according to a degree of convergence of the prediction result of the large model to the true label approximate value” (a mathematical calculation, see pg. 11, lines 17-19 & pg. 12, lines 1-9 in Specification, training a model using KL-divergence loss function)
Claim 1 therefore recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
“a processor; and a memory storing instructions that, when executed by the processor, cause the apparatus to” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
"receive local information from each of a plurality of individual devices, the local information including a global parameter of a local model, a local latent vector, and a local loss value” (This step is directed to data gathering, which is understood to be insignificant extra solution activity - see MPEP 2106.05(g))
“store the local information” (This step is directed to storing data in memory, which is understood to be insignificant extra solution activity - see MPEP 2106.05(g))
“collect the global parameter of the local model and ” (This step is directed to data gathering, which is understood to be insignificant extra solution activity - see MPEP 2106.05(g))
“learn the large model using a prediction result obtained by inputting the local latent vector into the large model and the true label approximate value” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“obtain an original prediction probability result by inputting the local latent vector into the global model corresponding to the federated global parameter” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
The additional elements as disclosed above alone or in combination do not integrate the judicial exception into practical application as they are mere insignificant extra solution activity in combination of generic computer functions being implemented with generic computer elements in a high level of generality to perform the disclosed abstract idea above. Therefore, Claim 1 is directed to the abstract idea.
Subject Matter Eligibility Analysis Step 2B:
“a processor; and a memory storing instructions that, when executed by the processor, cause the apparatus to” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
"receive local information from each of a plurality of individual devices, the local information including a global parameter of a local model, a local latent vector, and a local loss value” (This step is directed to transmitting or receiving information, which is understood to be insignificant extra solution activity and well understood, routine and conventional activity of transmitting and receiving data as identified by the court - see MPEP 2106.05(d))
“store the local information” (This step is directed to storing data in memory, which is understood to be insignificant extra solution activity and well understood, routine and conventional activity of storing and retrieving information in memory as identified by the court - see MPEP 2106.05(d))
“collect the global parameter of the local model and ” (This step is directed to transmitting or receiving information, which is understood to be insignificant extra solution activity and well understood, routine and conventional activity of transmitting and receiving data as identified by the court - see MPEP 2106.05(d))
“learn the large model using a prediction result obtained by inputting the local latent vector into the large model and the true label approximate value” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“obtain an original prediction probability result by inputting the local latent vector into the global model corresponding to the federated global parameter” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
The additional elements as disclosed above alone or in combination do not recite significantly more than the abstract idea itself as they are mere insignificant extra solution activity in combination of generic computer functions being implemented with generic computer elements in a high level of generality to perform the disclosed abstract idea above. Therefore, Claim 1 is subject-matter ineligible.
Regarding Claim 2:
Subject Matter Eligibility Analysis Step 2A Prong 1: None
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“wherein each of the plurality of individual devices uses local data to learn the local model composed of a local parameter and a global parameter” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“wherein the local information comprises the global parameter of the local model, the local latent vector obtained by inputting the local data into the local model, and a local loss value obtained by inputting the local latent vector into the local model” (merely specifies a particular technological environment in which the abstract idea is to take place, ie. a field of use, and thus does not integrate the abstract idea into a practical application nor cannot provide significantly more than the abstract idea itself - see MPEP 2106.05(h))
Regarding Claim 3:
Subject Matter Eligibility Analysis Step 2A Prong 1:
“generate the true label approximate value” (a mental process that can be performed in the human mind, i.e. judgement)
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“wherein the instructions, when executed by the processor, further cause the apparatus to” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“learn the large model using the generated true label approximate value and a preset loss function” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claim 4:
Subject Matter Eligibility Analysis Step 2A Prong 1:
“the true label approximate value through a softmax temperature-based reverse inference process” (a mental process that can be performed in the human mind with the aide of pen and paper, i.e. judgement)
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“wherein the instructions, when executed by the processor, further cause the apparatus to ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claim 5:
Subject Matter Eligibility Analysis Step 2A Prong 1:
“ a true label approximate value distribution by applying a softmax temperature determined based on the local loss value to a prediction probability result ” (a mental process that can be performed in the human mind with the aide of pen and paper, i.e. judgement)
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“wherein the instructions, when executed by the processor, further cause the apparatus to ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claim 6:
Subject Matter Eligibility Analysis Step 2A Prong 1:
“ the softmax temperature through a relative scale that allows more learning using local information of an individual device that is relatively good at predicting in a current communication round and an absolute scale that determines an amount of learning using local information of each individual device” (a mental process that can be performed in the human mind with the aide of pen and paper, i.e. judgement)
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“wherein the instructions, when executed by the processor, further cause the apparatus to ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claim 7:
Subject Matter Eligibility Analysis Step 2A Prong 1:
“ learning of the large model using a distance-based loss function so that the large model follows the true label approximate value distribution” (a mathematical calculation; see pg. 12, lines 2-3 in Specification)
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“wherein the instructions, when executed by the processor, further cause the apparatus to perform learning of the large model ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claim 8:
Subject Matter Eligibility Analysis Step 2A Prong 1:
“ the true label approximate value using a cross-entropy loss function” (a mathematical calculation; see pg. 12, lines 7-10 in Specification)
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“wherein the instructions, when executed by the processor, further cause the apparatus to generate ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claim 9:
Subject Matter Eligibility Analysis Step 2A Prong 1:
“ the true label approximate value using a loss value set for a prediction result” (a mental process that can be performed in the human mind with the aide of pen and paper, i.e. judgement)
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“wherein the instructions, when executed by the processor, further cause the apparatus to generate ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claim 10:
Subject Matter Eligibility Analysis Step 2A Prong 1:
“ an element with the closest distance to a true label probability value as a true label approximate value by using elements included in the loss value set and the true label probability value calculated using the local loss value” (a mental process that can be performed in the human mind with the aide of pen and paper, i.e. judgement)
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“wherein the instructions, when executed by the processor, further cause the apparatus to reversely infer” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claim 11:
Subject Matter Eligibility Analysis Step 2A Prong 1:
“memory” (a mental process that can be performed in the human mind with the aide of pen and paper, i.e. judgement)
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“wherein the instructions, when executed by the processor, further cause the apparatus to: learn the large model using local information received in a current communication round” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“further learn the large model ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claim 12:
Subject Matter Eligibility Analysis Step 2A Prong 1:
“ the local information including a global parameter of the trained local model, a local latent vector, and a local loss value” (a mental process that can be performed in the human mind, i.e. judgement)
“” (a mental process that can be performed in the human mind, i.e. judgement)
“generate the true label approximate value having an approximation degree to a true label by processing, based on the local loss value, a probability distribution or a probability value of the original prediction probability result so as to reflect how close the original prediction probability result is to the true label” (a mathematical calculation, see pg. 10, lines 3-13 in Specification)
“learn the large model, using a loss function reflecting a distance between the true label approximate value and a prediction result of the large model, while adjusting a learning intensity of the large model according to a degree of convergence of the prediction result of the large model to the true label approximate value” (a mathematical calculation, see pg. 11, lines 17-19 & pg. 12, lines 1-9 in Specification, training a model using KL-divergence loss function)
Claim 12 therefore recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
“” (This step is directed to storing data in memory, which is understood to be insignificant extra solution activity - see MPEP 2106.05(g))
“a plurality of individual devices that store ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“a server that prediction result obtained by inputting the local latent vector in the local information into the large model and the true label approximate value” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
"
“” (This step is directed to storing data in memory, which is understood to be insignificant extra solution activity - see MPEP 2106.05(g))
“wherein the server is further configured to: obtain an original prediction probability result by inputting the local latent vector into the global model corresponding to the federated global parameter” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
The additional elements as disclosed above alone or in combination do not integrate the judicial exception into practical application as they are mere insignificant extra solution activity in combination of generic computer functions being implemented with generic computer elements in a high level of generality to perform the disclosed abstract idea above. Therefore, Claim 12 is directed to the abstract idea.
Subject Matter Eligibility Analysis Step 2B:
“” (This step is directed to storing data in memory, which is understood to be insignificant extra solution activity and well understood, routine and conventional activity of storing and retrieving information in memory as identified by the court - see MPEP 2106.05(d))
“a plurality of individual devices that store ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“a server that in the local information into the large model and the true label approximate value” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
"
“” (This step is directed to storing data in memory, which is understood to be insignificant extra solution activity and well understood, routine and conventional activity of storing and retrieving information in memory as identified by the court - see MPEP 2106.05(d))
“wherein the server is further configured to: obtain an original prediction probability result by inputting the local latent vector into the global model corresponding to the federated global parameter” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
The additional elements as disclosed above alone or in combination do not recite significantly more than the abstract idea itself as they are mere insignificant extra solution activity in combination of generic computer functions being implemented with generic computer elements in a high level of generality to perform the disclosed abstract idea above. Therefore, Claim 12 is subject-matter ineligible.
Regarding Claim 13:
Subject Matter Eligibility Analysis Step 2A Prong 1: None
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“wherein the local information comprises the global parameter of the local model, the local latent vector obtained by inputting the local data into the local model, and the local loss value obtained by inputting the local latent vector into the local model” (merely specifies a particular technological environment in which the abstract idea is to take place, ie. a field of use, and thus does not integrate the abstract idea into a practical application nor cannot provide significantly more than the abstract idea itself - see MPEP 2106.05(h))
Regarding Claim 14:
The claim recites a method that performs the same process as the system as described in claim 1. Therefore, claim 14 is rejected for the same reasons as disclosed for claim 1.
Regarding Claim 15:
The claim recites an article of manufacture that performs the same process as described in claim 14, which is rejected for the same reasons as disclosed for claim 1. Therefore, claim 15 is rejected for the same reasons as disclosed for claim 1.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5 and 7-15 are rejected under 35 U.S.C. 103 as being unpatentable over Dai (US20240054350A1) in view of He, “Group Knowledge Transfer: Federated Learning of Large CNNs at the Edge” and Zeng, “FedCav: Contribution-aware Model Aggregation on Distributed Heterogeneous Data in Federated Learning”. He is provided as a reference in the IDS dated 12/10/2025.
Regarding claim 1, Dai teaches:
“An apparatus for augmenting knowledge using federated learning information comprising” (abstract, A federated learning system consists of a central server that updates a global model based on the received updated local model parameters.)
“a processor; and a memory storing instructions that, when executed by the processor, cause the apparatus to” ([0036-0038], The central server implementing the federated learning framework consist of a processor and memory.)
“receive local information from each of a plurality of individual devices, the local information including a global parameter of a local model, a local latent vector, ” ([0033-0034, 0063], The client sends both the body parameters (global parameter of a local model) and local prototypes (local latent vector) to the server for aggregation. A set of clients respond to the server during each communication round.)
“store the local information” ([0033; 0036-0037; 0041; Fig. 2A], The central server consists of a memory, which consist of a client update submodule that is configured to receive information from client systems. Clients send local updates to the server.)
“collect the global parameter of the local model for learning a large model using the local information and a federated global parameter for a global model” ([0033-0034; 0041; Fig. 2A], A global update submodule is configured to update the global model based on aggregated feedback from clients. The server receives local updates from the clients and aggregates those local updates to update the global model.)
“learn the large model using a prediction result obtained by inputting the local latent vector into the large model ” ([0041; 0078-0079; 0082-0083], The FedNH module receive input training data for training a baseline global model. The server generates the plurality of updated class prototypes of the global model based on the representatives received from the client. The global model may be tested for accuracy during training by computing a metric based on the true label and predicted label.)
“obtain an original prediction ” ([0021; 0028; 0066], The global and local model consists of a classification head and the latent representations are provided as inputs to the classification head to classify the points. The output is processed by the global model based on the global parameters.)
“[l]earn the large model, using a loss function ” ([0028-0035], The local model is trained based on a optimization process of learning a strong body. The body parameters and local prototypes are provided to the server for aggregation and updating the global model.)
Dai does not explicitly disclose an implementation of “receive local information … the local information including ... a local loss value”, “generate a true label approximate value for learning a large model using the local information and a federated global parameter for a global model”, “learn the large model using a prediction result obtained by inputting the local latent vector into the large model and the true label approximate value”, “obtain an original prediction probability result”, “generate the true label approximate value having an approximation degree to a true label by processing, based on the local loss value, a probability distribution or a probability value of the original prediction probability result so as to reflect how close the original prediction probability result is to the true label”, and “[l]earn the large model, using a loss function reflecting a distance between the true label approximate value and a prediction result of the large model, while adjusting a learning intensity of the large model according to a degree of convergence of the prediction result of the large model to the true label approximate value”.
However, He discloses in the same field of endeavor:
“collect the global parameter of the local model and generate a true label approximate value for learning a large model using the local information and a federated global parameter for a global model” ([pg. 3, Section 3.1, par. 1; pg. 4, Section 3.2, par. 1; pg. 4-5, Section 3.3, par. 1-3], The server model is trained using the feature map outputs collected from the plurality of clients and the server model is optimized based on the knowledge received from the plurality of client devices. The server model generates probabilistic predictions using the logit output.)
“learn the large model using a prediction result obtained by inputting the local latent vector into the large model and the true label approximate value” ([pg. 3, Section 3.1, par. 1; pg. 4, Section 3.2, par. 1; pg. 4-5, Section 3.3, par. 1-4], The server model is trained using the feature map outputs collected from the plurality of clients and the server model is optimized based on the KL divergence function that serves to transfer knowledge between the clients and the server based on the probabilistic predictions of the edge model and the server model.)
“obtain an original prediction probability result by inputting the local latent vector into the global model corresponding to the federated global parameter” ([pg. 4-5, Section 3.3, par. 1-3], The edge model generates a probabilistic prediction (original prediction probability result) and the probabilistic prediction from the edge model is provided to the server to calculated the KL divergence and the server loss function.)
“generate the true label approximate value having an approximation degree to a true label by processing, based on the local loss value, a probability distribution or a probability value of the original prediction probability result so as to reflect how close the original prediction probability result is to the true label” ([pg. 3, Section 3.1, par. 1; pg. 4-5, Section 3.3, par. 1-4], Each client has its own dataset that consists of a number of training samples and the corresponding label. A local cross-entropy loss is computed using the output of the local model and the ground truth of the local dataset. The server model is optimized based on the local cross-entropy loss and a second loss that uses KL divergence between the probability distribution of the edge model and the server model. The KL divergence loss attempts to bring the predicted soft labels of the server model as close as possible to the ground truth. The server model is trained for multiple iterations and communication between the clients and the server consists of multiple rounds. Thus, the server will generate an improved probabilistic prediction based on the optimization of the loss functions.)
“[l]earn the large model, using a loss function reflecting a distance between the true label approximate value and a prediction result of the large model, while adjusting a learning intensity of the large model according to a degree of convergence of the prediction result of the large model to the true label approximate value” ([pg. 3, Section 3.1, par. 1; pg. 4-5, Section 3.3, par. 1-4; pg. 7, Section 4.2, par. 2], The server model is optimized based on the local cross-entropy loss and a second loss that uses KL divergence between the probability distribution of the edge model and the server model. The KL divergence loss attempts to bring the predicted soft labels of the server model as close as possible to the ground truth. The server model is trained for multiple iterations and communication between the clients and the server consists of multiple rounds. The server-side learning rate is adjusted during the training of the server model.)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of “generate a true label approximate value for learning a large model using the local information and a federated global parameter for a global model”, “learn the large model using a prediction result obtained by inputting the local latent vector into the large model and the true label approximate value”, “obtain an original prediction probability result”, “generate the true label approximate value having an approximation degree to a true label by processing, based on the local loss value, a probability distribution or a probability value of the original prediction probability result so as to reflect how close the original prediction probability result is to the true label”, and “[l]earn the large model, using a loss function reflecting a distance between the true label approximate value and a prediction result of the large model, while adjusting a learning intensity of the large model according to a degree of convergence of the prediction result of the large model to the true label approximate value” from He into the teaching of Dai. Doing so can improve the performance of federated learning on resource constrained edge devices by implementing a group knowledge transfer training algorithm (He, abstract).
Dai in view of He does not explicitly disclose an implementation of “receive local information … the local information including ... a local loss value”. However, Zeng discloses in the same field of endeavor:
“receive local information from each of a plurality of individual devices, the local information including … a local loss value” ([pg. 4, Section 4.1, par. 1-2], The client computes the inference loss, which is the value of the global model loss function on the local data and sends the inference loss to the server with the local updates. The inference loss is a cross-entropy loss function.)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of “receives local information including ... a local loss value from each of a plurality of individual devices” from Zeng into the teaching of Dai in view of He. Doing so can improve the performance of federated learning on heterogenous data by implementing a global loss function with explicit optimization preference on informative local updates (Zeng, abstract).
Regarding claim 12, claim 12 recites a system that performs the same process as described in Claim 1. Therefore claim 12 is rejected under the same reasons mentioned for claim 1. Please review the claim mapping for claim 1 regarding the new claim limitations for “wherein the server is further configured to”. The additional elements of claim 12 is addressed below. Dai teaches:
“A system for augmenting knowledge using federated learning information comprising” (abstract, A federated learning system consists of a central server that updates a global model based on the received updated local model parameters.)
“a plurality of individual devices that store local data for learning a local model composed of a local parameter and a global parameter, learn the local model using the local data, and extract a portion of information about the learned local model as local information, the local information including a global parameter of the trained local model, a local latent vector, ” ([0030-0034, 0048, 0063, Figure 2B], A plurality of client devices includes a personalized model that is periodically updated based on a global model from the central server. The client computes the averaged representations that provides a local description of class prototypes for classes that is presented in the local dataset of the client. The client’s local training is executed based on the class prototypes and the averaged representations are sent to the central server as feedback. The client sends both the body parameters (global parameter of a local model) and local prototypes (local latent vector) to the server for aggregation. A set of clients respond to the server during each communication round.)
“a server that receives the local information from each of the plurality of individual devices, stores the local information, generates a federated global parameter for a global model using the local information, uses the local information and the federated global parameter to generate a in the local information into the large model ” ([0033-0034; 0041; 0078-0079; 0082-0083; Fig. 2A], The central server consists of a memory, which consist of a client update submodule that is configured to receive information from client systems. Clients sends local updates to the server. A global update submodule is configured to update the global model based on aggregated feedback from clients. The server receives local updates from the clients and aggregate those local updates to update the global model. The FedNH module receive input training data for training a baseline global model. The server generates the plurality of updated class prototypes of the global model based on the representatives received from the client. The global model may be tested for accuracy during training by computing a metric based on the true label and predicted label.)
Dai does not explicitly disclose an implementation of “the local information including ... a local loss value”, and “true label approximate value”. However, He discloses in the same field of endeavor:
“a server that ... uses the local information and the federated global parameter to generate a true label approximate value for learning a large model, and learns the large model using a prediction result obtained by inputting the local latent vector in the local information into the large model and the true label approximate value” ([pg. 1-2, Section 1, par. 2; pg. 4-5, Section 3.3, par. 1-4], The server model is trained using the feature map outputs collected from the plurality of clients and the server model is optimized based on the knowledge received from the plurality of client devices. The server model is optimized based on the local cross-entropy loss and a second loss that uses KL divergence between the probability distribution of the edge model and the server model. The KL divergence loss attempts to bring the predicted soft labels of the server model as close as possible to the ground truth. The server model is trained for multiple iterations and communication between the clients and the server consists of multiple rounds. Thus, the server will generate an improved probabilistic prediction based on the optimization of the loss functions.)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of “true label approximate value” from He into the teaching of Dai. Doing so can improve the performance of federated learning on resource constrained edge devices by implementing a group knowledge transfer training algorithm (He, abstract).
Dai in view of He does not explicitly disclose an implementation of “the local information including ... a local loss value”. However, Zeng discloses in the same field of endeavor:
“… the local information including … a local loss value” ([pg. 4, Section 4.1, par. 1-2], The client computes the inference loss, which is the value of the global model loss function on the local data and sends the inference loss to the server with the local updates. The inference loss is a cross-entropy loss function.)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of “the local information including ... a local loss value” from Zeng into the teaching of Dai in view of He. Doing so can improve the performance of federated learning on heterogenous data by implementing a global loss function with explicit optimization preference on informative local updates (Zeng, abstract).
Regarding claims 2 and 13, Dai in view of He teaches:
“wherein each of the plurality of individual devices uses local data to learn the local model composed of a local parameter and a global parameter” ([Dai, 0017, 0025, 0030, 0046], Each client has a respective personalized local model. The client receives the body parameter and the head parameter from the server and trains the local model with its local training dataset that is not shared with the server. The updated parameters of the body may be shared with the central server.)
“wherein the local information comprises the global parameter of the local model, the local latent vector obtained by inputting the local data into the local model, and a local loss value ” ([Dai, 0017, 0023, 0030-033], The server sends the body parameter to the clients to train the local model using the global parameter. The client computes the averaged representations that provides a local description of class prototypes for classes that is presented in the local dataset of the client. A loss is computed for an optimization process to learn a strong body.)
Dai in view of He does not explicitly disclose an implementation of “wherein the local information comprises ... a local loss value obtained by inputting the local latent vector into the local model”. However, Zeng discloses in the same field of endeavor:
“wherein the local information comprises ... a local loss value obtained by inputting the local latent vector into the local model” ([pg. 4, Section 4.1, par. 1-2], The client computes the inference loss, which is the value of the global model loss function on the local data and sends the inference loss to the server with the local updates.)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of “wherein the local information comprises ... a local loss value obtained by inputting the local latent vector into the local model” from Zeng into the teaching of Dai in view of He. Doing so can improve the performance of federated learning on heterogenous data by implementing a global loss function with explicit optimization preference on informative local updates (Zeng, abstract).
Regarding claim 3, Dai in view of He teaches:
“wherein the instructions, when executed by the processor, further cause the apparatus to: generate the true label approximate value” ([He, Section 3.3, par. 1-3], The server model generates the probabilistic predictions based of the computed logits. It is implied that the server is a computer with a processor to perform the proposed framework.)
“learn the large model using the generated true label approximate value and a preset loss function” ([He, pg. 1-2, Section 1, par. 2; pg. 4-5, Section 3.3, par. 1-3], The server and edge models are trained based on the cross-entropy loss between the predicted values and the ground truth labels and the KL divergence function.)
Regarding claim 4, Dai in view of He teaches:
“wherein the instructions, when executed by the processor, further cause the apparatus to generate the true label approximate value through a softmax temperature-based reverse inference process” ([He, pg. 1-2, Section 1, par. 2; pg. 4-5, Section 3.3, par. 1-3], The server model generates the probabilistic predictions based of the computed softmax of logits and a temperature hyperparameter. It is implied that the server is a computer with a processor to perform the proposed framework.)
Regarding claim 5, Dai in view of He and Zeng teaches:
“wherein the instructions, when executed by the processor, further cause the apparatus to generate the true label approximate value through a softmax temperature-based reverse inference processa true label approximate value distribution ([He, pg. 1-2, Section 1, par. 2; pg. 4-5, Section 3.3, par. 1-3], The server model generates the probabilistic predictions based of the computed softmax of logits and a temperature hyperparameter. It is implied that the server is a computer with a processor to perform the proposed framework.) by applying a softmax temperature determined based on the local loss value to a prediction probability result obtained by inputting the local latent vector into the global model” ([Zeng, pg. 5-6, Section 4.2.3, par. 5-9], The softmax function is computed based on local loss function value.)
Regarding claim 7, Dai in view of He teaches:
“wherein the instructions, when executed by the processor, further cause the apparatus to perform learning of the large model using a distance-based loss function so that the large model follows the true label approximate value distribution” ([He, pg. 1-2, Section 1, par. 2; pg. 4-5, Section 3.3, par. 1-3], The KL divergence function is a distance-based loss function to transfer knowledge from a network to another. It is implied that the server is a computer with a processor to perform the proposed framework.)
Regarding claim 8, Dai in view of He teaches:
“wherein the instructions, when executed by the processor, further cause the apparatus to generate the true label approximate value using a cross-entropy loss function” ([He, pg. 1-2, Section 1, par. 2; pg. 4-5, Section 3.3, par. 1-4], The server model is optimized based on the local cross-entropy loss and a second loss that uses KL divergence between the probability distribution of the edge model and the server model. The KL divergence loss attempts to bring the predicted soft labels of the server model as close as possible to the ground truth. The server model is trained for multiple iterations and communication between the clients and the server consists of multiple rounds. Thus, the server will generate an improved probabilistic prediction based on the optimization of the loss functions. It is implied that the server is a computer with a processor to perform the proposed framework.)
Regarding claim 9, Dai in view of He teaches:
“wherein the instructions, when executed by the processor, further cause the apparatus to generate the true label approximate value using a loss value set for a prediction result obtained by inputting the local latent vector into the global model” ([He, pg. 4, Section 3.2, par. 2; pg. 4-5, Section 3.3, par. 1-3], The server model is trained using the feature map outputs collected from the plurality of clients and the server model is optimized based on the knowledge received from the plurality of client devices. The server model is optimized based on the local cross-entropy loss and a second loss that uses KL divergence between the probability distribution of the edge model and the server model. The KL divergence loss attempts to bring the predicted soft labels of the server model as close as possible to the ground truth. The server model is trained for multiple iterations and communication between the clients and the server consists of multiple rounds. Thus, the server will generate an improved probabilistic prediction based on the optimization of the loss functions. It is implied that the server is a computer with a processor to perform the proposed framework.)
Regarding claim 10, Dai in view of He teaches:
“wherein the instructions, when executed by the processor, further cause the apparatus to reversely infer an element with the closest distance to a true label probability value as a true label approximate value by using elements included in the loss value set and the true label probability value calculated using the local loss value” ([He, pg. 4-5, Section 3.3, par. 1-6], An alternating minimization is proposed to optimize the server model and edge model. In Equation 10, the local cross-entropy loss is computed for the edge model and the KL divergence is computed based on predictions from the edge and server models. Knowledge distillation between the edge devices and server is used to effectively capture knowledge from multiple client datasets. The KL divergence loss brings the soft label and the ground truth close to each other. The server model absorbs the knowledge gained from each of the edge models. It is implied that the server is a computer with a processor to perform the proposed framework.)
Regarding claim 11, Dai teaches:
“wherein the instructions, when executed by the processor, further cause the apparatus to: learn the large model using local information received in a current communication round” ([0030, 0033-0034, 0036], The server updates the global model using information received from the clients during each communication round.)
“further learn the large model by randomly sampling a portion of the local information stored in the memory” ([0021, 0033-0034, 0041, Figure 2A], The client update submodule receives information from the client systems and stores the information in memory. The central server may update the global model based on aggregating information from a subset of active clients at a given communication interval. Thus, the server may only use a portion of the local information received from all the active clients.)
Regarding Claim 14:
Claim 14 recites a method that performs the same process as the system as described in Claim 1. Therefore claim 14 is rejected under the same reasons mention for claim 1.
Regarding Claim 15:
Claim 15 recites an article of manufacture that performs the same process as Claim 14, which is a method of the system as described in Claim 1. Therefore claim 15 is rejected under the same reasons mentioned for claim 1.
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Dai (US20240054350A1) in view of He, “Group Knowledge Transfer: Federated Learning of Large CNNs at the Edge”, Zeng, “FedCav: Contribution-aware Model Aggregation on Distributed Heterogeneous Data in Federated Learning”, and Li “FedRS: Federated Learning with Restricted Softmax for Label Distribution Non-IID Data”. He is provided as a reference in the IDS dated 12/10/2025.
Regarding claim 6, Dai in view of He and Zeng teaches:
“wherein the instructions, when executed by the processor, further cause the apparatus to determine the softmax temperature ” ([Zeng, pg. 5-6, Section 4.2.3, par. 5-9], The softmax function is computed based on local loss function value. It is implied the framework is performed on a computer consisting of a processor.)
Dai in view of He and Zeng does not explicitly disclose an implementation of “determine the softmax temperature through a relative scale that allows more learning using local information of an individual device that is relatively good at predicting in a current communication round and an absolute scale that determines an amount of learning using local information of each individual device”. However, Li discloses in the same field of endeavor:
“... determine the softmax temperature through a relative scale that allows more learning using local information of an individual device that is relatively good at predicting in a current communication round and an absolute scale that determines an amount of learning using local information of each individual device” ([pg. 4-5, Section 4.3, par. 1-7; pg. 4, Figure 4], Scaling factors are added to softmax operations to define restricted softmax. The use of the restricted softmax allows for pushing and pulling of features (relative scale) that may or may not belong to a particular class. If a client does not observe a class, a pushing force is imposed to that particular missing class. The client selection ratio and the specific class distribution in each client determines the strength (absolute scale) of these forces.)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of “determine the softmax temperature through a relative scale that allows more learning using local information of an individual device that is relatively good at predicting in a current communication round and an absolute scale that determines an amount of learning using local information of each individual device” from Li into the teaching of Dai in view of He and Zeng. Doing so can improve the performance of federated learning on heterogenous data by implementing a restricted SoftMax parameter to limit the updates of missing classes (Li, abstract).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GARY MAC whose telephone number is (703)756-1517. The examiner can normally be reached Monday - Friday 8:00 AM - 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abdullah Kawsar can be reached at (571) 270-3169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GARY MAC/Examiner, Art Unit 2127
/ABDULLAH AL KAWSAR/Supervisory Patent Examiner, Art Unit 2127
