Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-14 are rejected under 35 U.S.C. 101
because the claimed invention is directed to an abstract idea without significantly
more.
When considering subject matter eligibility under 35 U.S.C. 101, it must be
determined whether the claim is directed to one of the four statutory categories of
invention, i.e., process, machine, manufacture, or composition of matter (Step 1). If the
claim does fall within one of the statutory categories, the second step in the analysis is
to determine whether the claim is directed to a judicial exception (Step 2A). The Step 2A
analysis is broken into two prongs. In the first prong (Step 2A, Prong 1), it is determined
whether or not the claims recite a judicial exception (e.g., mathematical concepts,
mental processes, certain methods of organizing human activity). If it is determined in
Step 2A, Prong 1 that the claims recite a judicial exception, the analysis proceeds to the
second prong (Step 2A, Prong 2), where it is determined whether or not the claims
integrate the judicial exception into a practical application. If it is determined at step 2A,
Prong 2 that the claims do not integrate the judicial exception into a practical
application, the analysis proceeds to determining whether the claim is a patent-eligible
application of the exception (Step 2B). If an abstract idea is present in the claim, any
element or combination of elements in the claim must be sufficient to ensure that the
claim integrates the judicial exception into a practical application, or else amounts to
significantly more than the abstract idea itself. Applicant is advised to consult the 2019
PEG for more details of the analysis.
Step 1
According to the first part of the analysis, in the instant case, claims 1-4, 5-8, 9-11, 12-14 are directed to a method, a client, a method and a server of federated learning. Thus, each of the claims falls within one of the four statutory categories (i.e. process, machine, manufacture, or composition of matter). Step 2A,
Step 2A, Prong 1
Following the determination of whether or not the claims fall within one of the four
categories (Step 1), it must be determined if the claims recite a judicial exception (e.g.
mathematical concepts, mental processes, certain methods of organizing human
activity) (Step 2A, Prong 1). In this case, the claims are determined to recite a judicial
exception as explained below.
Regarding Claims 1, 5 and 9, 12 these claims recite
Claim 1 and 5:
receiving a global learning model from a server, the global learning model being a neural network model including a plurality of parameters; generating a local learning model based on the global learning model and a RANK-1 matrix , wherein the RANK-1 matrix is an outer product of a first vector and a second vector; generating a client learning model by training the local learning model based on local learning data;
transmitting the client learning model to the server; and receiving, from the server after the transmitting of the client learning model, a request to transmit the RANK-1 matrix, and transmitting the RANK-1 matrix to the server in response to the request.
Claim 9 and 12:
generating a first global learning model based on global learning data; transmitting the first global learning model to a plurality of clients; receiving a client learning model from each of the plurality of clients, each of the client learning models being generated by a respective client of the plurality of clients by training, based on local learning data of the respective client, a local learning model generated based on the first global learning model and a RANK-1 matrix of the respective client; generating a second global learning model based on the plurality of client learning models; training the second global learning model based on the global learning data; requesting the plurality of clients to transmit the RANK-1 matrices; receiving a plurality of RANK-1 matrices from the plurality of clients; and performing inference on the second global learning model based on the plurality of RANK-1 matrices received from the plurality of clients.
The claims recite a mental process. As set forth in MPEP 2106.04(a)(2)(III)(C), “Claims can recite a mental process even if they are claimed as being performed on a computer”. These are recited at a high level such that they are disclosed as a human user performing these functions, simply using a computer as a tool-see spec, [0024-0033], Fig. 1, etc. Thus, the claim recites abstract ideas.
Step 2A, Prong 2
Following the determination that the claims recite a judicial exception, it must be
determined if the claims recite additional elements that integrate the exception into a
practical application of the exception (Step 2A, Prong 2). In this case, after considering
all claim elements individually and as an ordered combination, it is determined that the
claims do not include additional elements that integrate the exception into a practical
application of the exception as explained below.
In Prong Two, a claim is evaluated as a whole to determine whether the recited judicial exception is integrated into a practical application of that exception. A claim is not “directed to” a judicial exception, and thus is patent eligible, if the claim as a whole integrates the recited judicial exception into a practical application of that exception. A claim that integrates a judicial exception into a practical application will apply, rely on, or use the judicial exception in a manner that imposes a meaningful limit on the judicial exception, such that the claim is more than a drafting effort designed to monopolize the judicial exception. MPEP 2106.04(d). The claims recite an abstract idea and further the claims as a whole does not integrate the recited judicial exception into a practical application of the exception. A claim that integrates a judicial exception into a practical application will apply, rely on, or use the judicial exception in a manner that imposes a meaningful limit on the judicial exception, such that the claim is more than a drafting effort designed to monopolize the judicial exception. MPEP 2106.04(d).
Regarding Claims 1, 5, 9, 12 these claims
This limitation recites using one or more neural networks as a tool to perform an
abstract idea, which is not indicative of integration into a practical application. MPEP 2106.05(f).)
This limitation is understood to be generic computer equipment and mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.0S(f))
MPEP § 2106.05(f): Mere Instructions to Apply an Exception. Do the additional element(s) amount to merely the words “apply it” (or an equivalent)
or are mere instructions to implement an abstract idea or other exception on a computer? (Yes)
Step 2B
Based on the determination in Step 2A of the analysis that the claims are
directed to a judicial exception, it must be determined if the claims contain any element
or combination of elements sufficient to ensure that the claim amounts to significantly
more than the judicial exception (Step 2B). In this case, after considering all claim
elements individually and as an ordered combination, it is determined that the claims do
not include additional elements that are sufficient to amount to significantly more than
the judicial exception for the same reasons given above in the Step 2A, Prong 2
analysis. Furthermore, each additional element identified above as being insignificant
extra-solution activity is also well-known, routine, conventional as described below.
Claims 1, 5, 9, and 12: The claims do not include additional elements, alone or in combination, that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements amount to no more than generic computing components and field of use/technological environment which do not amount to significantly more than the abstract idea. The underlying concept merely receives information, analyzes it, and store the results of the analysis – this concept is not meaningfully different than concepts found by the courts to be abstract (see Electric Power Group, collecting information, analyzing it, and displaying certain results of the collection and analysis; see Cybersource, obtaining and comparing intangible data; see Digitech, organizing information through mathematical correlations; see Grams, diagnosing an abnormal condition by performing clinical tests and thinking about the results; see Cyberfone, using categories to organize store and transmit information; see Smartgene, comparing new and stored information and using rules to identify options). The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements when considered both individually and as a combination do not amount to significantly more than the abstract idea. For example, claim 1 and 5 recites “receiving…”, “generating…”,“generating…”, “transmitting…” and receiving…” and claim 9 and 12 recite “generating…”, “transmitting…”, “receiving…”, “generating…”, “training…”, “requesting…”, “receiving…” and “performing…” etc. These elements are recited at a high level of generality and are well-understood, routine, and conventional activities in the computer art. Generic computers performing generic computer functions, without an inventive concept, do not amount to significantly more than the abstract idea. Looking at the elements as a combination does not add anything more than the elements analyzed individually. Therefore, these claims do not amount to significantly more than the abstract idea itself.
Step 2A/2B Prong 2 Dependent Claims
Regarding to claim 2, 6
Claim 2, 6 merely recite other additional elements that define generating local leaning model which performing generic functions that when looking at the elements as a combination does not add anything more than the elements analyzed individually. Therefore, these claims also do not amount to significantly more than the abstract idea itself. These claims are not patent eligible.
Regarding to claim 3, 7
Claim 3, 7 merely recite other additional elements that define generating the client leaning model which performing generic functions that when looking at the elements as a combination does not add anything more than the elements analyzed individually. Therefore, these claims also do not amount to significantly more than the abstract idea itself. These claims are not patent eligible.
Regarding to claim 4, 8
Claim 4, 8 merely recite other additional elements that define transmitting the matrix to the server which performing generic functions that when looking at the elements as a combination does not add anything more than the elements analyzed individually. Therefore, these claims also do not amount to significantly more than the abstract idea itself. These claims are not patent eligible.
Regarding to claim 10, 13
Claim 10, 13 merely recite other additional elements that define generating the global leaning model which performing generic functions that when looking at the elements as a combination does not add anything more than the elements analyzed individually. Therefore, these claims also do not amount to significantly more than the abstract idea itself. These claims are not patent eligible.
Regarding to claim 11, 14
Claim 11, 14 merely recite other additional elements that define receiving the matrices from the clients which performing generic functions that when looking at the elements as a combination does not add anything more than the elements analyzed individually. Therefore, these claims also do not amount to significantly more than the abstract idea itself. These claims are not patent eligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 4-6, 8-14 are rejected under 35 U.S.C. 103 as being unpatentable over Hyeon-Woo et al. (Hyeon-Woo) “FEDPARA: LOW-RANK HADAMARD PRODUCT FOR COMMUNICATION-EFFICIENT FEDERATED LEARNING” arXiv:2108.06098v2, 2021
In view of Satheesh Kumar et al. (Satheesh Kumar) US 2023/0004776
In regard to claim 1, Hyeon-Woo disclose A method of performing federated learning with a client, comprising: (abstract, 1 Introduction, federated learning with clients)
receiving a global learning model from a server, (abstract, 1 Introduction, downloading a global model from a central server) the global learning model being a neural network model including a plurality of parameters; (2. Method. 2.1, “Overview of Low-Rank Parameterization” neural network and the model have parameters)
generating a local learning model based on the global learning model and a RANK-1 matrix (1 Introduction, 2. Method, updating the local model based on global model and a RANK-1 matrix) wherein the RANK-1 matrix is an outer product of a first vector and a second vector; (2. Method. 2.1, “Overview of Low-Rank Parameterization” and 2.2. “FEDPARA:ACOMMUNICATION-EFFICIENTPARAMETERIZATION” the Kronecker product is the outer product of the first vector and the second vector)
generating a client learning model by training the local learning model based on local learning data; (2. Method, generating a model by training the updated local model based on the local training data)
transmitting the client learning model to the server. (2. Method, Fig. 2, transmitting the updated local model to the server) and
transmitting the RANK-1 matrix to the server (2. Method, Fig. 2, W1 is transferred to the server)
But Hyeon-Woo fail to explicitly disclose “and receiving, from the server after the transmitting of the client learning model, a request to transmit information, and transmitting the information to the server in response to the request.”
Satheesh Kumar disclose and receiving, from the server after the transmitting of the client learning model, a request to transmit information, and transmitting the information to the server in response to the request. ([0031]-[0040] [0056]-[0057] after receiving the local model from the client node by the server, moderator node can be a part of the central server, send a request to the client to request data and the client transmit the data as requested)
It would have been obvious to one having ordinary skill in the art before the effective filing data of the claimed invention was made to incorporate Satheesh Kumar’s federated learning method into Hyeon-Woo’s invention as they are related to the same field endeavor of federated learning. The motivation to combine these arts, as proposed above, at least because Satheesh Kumar’s communication between the server and the client in federated learning would help to provide more information into Hyeon-Woo’s system. Therefore it would have been obvious to one having ordinary skill in the art before the effective filing data of the claimed invention was made that providing more information between the server and the client would help to improve accuracy of the federated learning.
In regard to claim 2, Hyeon-Woo and Satheesh Kumar disclose The method of claim 1,
Hyeon-Woo disclose wherein the generating of the local learning model based on the global learning model and the RANK-1 matrix includes generating the local learning model based on the following equation: W new =W·(r·s T), wherein the Wnew is the local, learning model, the W is the global learning model, the (r·sT) is the RANK-1 matrix, the r is the first vector, and the sT is the second vector. (2. Method, generating the local model based on an equation with rank-1 matrix, 2.3 according to pFedPara, Wper. = W1·W2 and Wglo. = W1, and W2 is the summation of r1r2 number of rank-1 matrix)
In regard to claim 4, Hyeon-Woo and Satheesh Kumar disclose The method of claim 1,
Hyeon-Woo disclose wherein the global learning model received from the server is a shared model common to a plurality of clients including the client, (abstract, 1 Introduction, a globally shared model from a central server downloaded by clients) and the local learning data has a not independent and identically distributed (Non-IID) distribution with respect to local learning data of others of the plurality of clients. (abstract, 1 Introduction, data is non-IID corresponding to other clients)
In regard to claims 5-6, 8, claims 5-6, 8 are client claims corresponding to the method claims 1-4 above and, therefore, are rejected for the same reasons set forth in the rejections of claims 1-2, 4.
In regard to claim 9, Hyeon-Woo disclose A method of performing federated learning with a server, comprising: (abstract, 1 Introduction, 2 Method, federated learning a server)
generating a first global learning model based on global learning data; (3. Experiments, 3.2 Quantitative Results, generating a global model based on the global training data)
transmitting the first global learning model to a plurality of clients; (2 Method, Fig. 2, transmitting the global model to the clients)
receiving a client learning model from each of the plurality of clients, (2. Method, Fig. 2, transmitting the updated local model from clients to the server) each of the client learning models being generated by a respective client of the plurality of clients by training, based on local learning data of the respective client, a local learning model generated based on the first global learning model and a RANK-1 matrix of the respective client (Fig. 2, 2.3. “PFEDPARA: PERSONALIZED FL APPLICATION” local models are generated by training based on the local data of the client, and the global model and the matrix of the client)
generating a second global learning model based on the plurality of client learning models. (1 Introduction, 2 Method, Fig. 2, 3. Experiments, 3.2 Quantitative Results, generating an updated global model based on the received client models)
training the second global learning model based on the global learning data; 2 Method, Fig. 2, 3. Experiments, 3.2 Quantitative Results, training the updated global model based on the global data)
transmit a plurality of RANK-1 matrices from the plurality of clients; (2. Method, Fig. 2, W1 is transferred to the server from the clients)
and performing inference on the second global learning model based on the plurality of RANK-1 matrices received from the plurality of clients. (1 Introduction, 2 Method, Fig. 2, performing inference on the updated global model based on the matrices received from the clients)
But Hyeon-Woo fail to explicitly disclose “requesting the plurality of clients to transmit information; receiving the information from the plurality of clients;”
Satheesh Kumar disclose requesting the plurality of clients to transmit information; receiving the information from the plurality of clients. ([0031]-[0040] [0056]-[0057] moderator node can be a part of the central server, send a request to the clients to request data and the clients transmit the data as requested)
It would have been obvious to one having ordinary skill in the art before the effective filing data of the claimed invention was made to incorporate Satheesh Kumar’s federated learning method into Hyeon-Woo’s invention as they are related to the same field endeavor of federated learning. The motivation to combine these arts, as proposed above, at least because Satheesh Kumar’s communication between the server and the client in federated learning would help to provide more information into Hyeon-Woo’s system. Therefore it would have been obvious to one having ordinary skill in the art before the effective filing data of the claimed invention was made that providing more information between the server and the client would help to improve accuracy of the federated learning.
In regard to claim 10, Hyeon-Woo and Satheesh Kumar disclose The method of claim 9,
Hyeon-Woo disclose wherein the generating of the second global learning model based on the plurality of client learning models includes calculating the average of the plurality of client learning models to generate the second global learning model. (1 Introduction, 2 Method, Fig. 2, 3. Experiments, 3.2 Quantitative Results, 4 Related Work, FedAvg, generating the updated global model based on the received client models is averaging the client models to generate the updated global model)
In regard to claim 11, Hyeon-Woo and Satheesh Kumar disclose The method of claim 9,
Hyeon-Woo disclose further comprising: receiving RANK-1 matrices from the plurality of clients; (2 Method, Fig. 2, 3. Experiments, 3.2 Quantitative Results, receiving Rank-1 matrices from the clients) and
performing inference on the second global learning model based on the RANK-1 matrix (1 Introduction, 2 Method, inference the model based on the Rank-1 matrix received) wherein the second global learning model includes a shared model and the plurality of RANK-1 matrices received from the plurality of clients, (Fig. 2, abstract, 1 Introduction, 2 Method, a globally shared model from a central server downloaded by clients and W1 is transferred to the server from the clients) and the second global learning model constitutes an ensemble model. (I Introduction, and Supplementary aggregate the updated model at the server side)
In regard to claims 12-14, claims 12-14 are server claims corresponding to the method claims 9-11 above and, therefore, are rejected for the same reasons set forth in the rejections of claims 9-11.
Claims 3, 7 are rejected under 35 U.S.C. 103 as being unpatentable over Hyeon-Woo et al. (Hyeon-Woo) “FEDPARA: LOW-RANK HADAMARD PRODUCT FOR COMMUNICATION-EFFICIENT FEDERATED LEARNING” arXiv:2108.06098v2, 2021
And Satheesh Kumar et al. (Satheesh Kumar) US 2023/0004776 as applied to claim 1, further In view of Yulian “Master Thesis: Federated Transfer Learning with Multimodal Data.” arXiv:2209.03137v1 https://doi.org/10.48550/arXiv.2209.03137, Sep. 8, 2022
In regard to claim 3, Hyeon-Woo and Satheesh Kumar disclose The method of claim 1,
Hyeon-Woo disclose wherein the generating of the client learning model includes training the local learning model based on the local learning data in a manner that minimizes a loss function (2. Method, generating a model by training the updated local model based on the local training data and minimizing a loss function)
But Hyeon-Woo and Satheesh Kumar fail to explicitly disclose “the loss function defined by the following equation:
PNG
media_image1.png
84
382
media_image1.png
Greyscale
Wherein the LCE is a loss function, the
PNG
media_image2.png
38
42
media_image2.png
Greyscale
PNG
media_image3.png
135
14
media_image3.png
Greyscale
is a y-th dimension vector, the xj
PNG
media_image3.png
135
14
media_image3.png
Greyscale
is a j-th dimension vector, and the j is the number of classes.”
Yulian disclose the loss function defined by the following equation:
PNG
media_image1.png
84
382
media_image1.png
Greyscale
Wherein the LCE is a loss function, the
PNG
media_image2.png
38
42
media_image2.png
Greyscale
PNG
media_image3.png
135
14
media_image3.png
Greyscale
is a y-th dimension vector, the xj
PNG
media_image3.png
135
14
media_image3.png
Greyscale
is a j-th dimension vector, and the j is the number of classes. (2.3 Supervised Learning and Self-Supervised Learning, with Lce eqation at page 7-10)
It would have been obvious to one having ordinary skill in the art before the effective filing data of the claimed invention was made to incorporate Yulian’s federated learning method into Hyeon-Woo’s invention as they are related to the same field endeavor of federated learning. The motivation to combine these arts, as proposed above, at least because Yulian’s loss calculation in federated learning would help to provide more loss functionality into Hyeon-Woo’s system. Therefore it would have been obvious to one having ordinary skill in the art before the effective filing data of the claimed invention was made that providing loss functionality in federated learning would help to improve federated learning.
In regard to claim 7, claim 7 is a client claim corresponding to the method claim 3 above and, therefore, are rejected for the same reasons set forth in the rejections of claim 3.
Response to Arguments
Applicant’s arguments with respect to claims 1-14 filed on 8/17/2026 have been considered but are moot because the arguments do not apply to the current rejection.
Conclusion
The prior art made of record and not relied upon is considered pertinent to Applicant's disclosure.
U.S. Patent Documents PATENT DATE INVENTOR(S) TITLE
US 20220158888 A1 2022-05-19 LEE et al.
METHOD TO REMOVE ABNORMAL CLIENTS IN A FEDERATED LEARNING MODEL
LEE et al. disclose a method of removing, by a server, an abnormal client in federated learning. A method of removing, by a server, an abnormal client in federated learning may include receiving, from a user equipment (UE), first weight values trained in a first local model, generating a first client model based on the first weight values, validating the first client model by using a validation data set in order to determine whether the first client model is legitimate, and removing the first weight values based on the first client model not being legitimate…. See abstract.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to XUYANG XIA whose telephone number is (571)270-3045. The examiner can normally be reached Monday-Friday 8am-4pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Welch can be reached at 571-272-7212. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
XUYANG XIA
Primary Examiner
Art Unit 2143
/XUYANG XIA/ Primary Examiner, Art Unit 2143
Prosecution Insights