DETAILED ACTION
1. The present application is being examined under the pre-AIA first to invent provisions.
Response to Amendment
2. Receipt of Applicant’s Amendment filed on 07/31/2026 is acknowledged. The amendment includes the amending of claims 1 and 11.
Claim Rejections - 35 USC § 103
3. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
4. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
5. This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
6. Claims 1, 4-6, 8-9, 11, 14-16, and 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over Augenstein et al. (Article entitled “Jointly Learning from Decentralized (Federated) and Centralized Data to Mitigate Distribution Shift”, dated 23 November 2021), in view of Sidahmed et al. (Article entitled “Efficient and Private Federated Learning with Partially Trainable Networks”, dated 08 November 2021).
7. Regarding claims 1 and 11, Augenstein teaches a computer-implemented method and system comprising:
A) training, at the client device, a client machine learning (ML) model on client training data (Page 2, Section 2, Page 10); and
B) while training the client ML model: obtaining, from a server, server model weights of a server ML model trained on server training data separately from and in parallel with the client ML model (Page 2, Section 2, Page 10);
C) the server training data different than the client training data (Page 2, Section 2, Page 10);
E) transmitting, to the server, client model weights of the client ML model (Page 2, Section 2, Page 10);
F) updating the client ML model using the server model weights (Page 2, Section 2, Page 10);
G) obtaining, from the server, updated server model weights of the server ML model, the updated server model weights updated based on the transmitted client model weights (Page 2, Section 2, Page 10); and
H) further updating the client ML model using the updated server model weights (Page 2, Section 2, Page 10).
The examiner notes that Augenstein teaches “training, at the client device, a client machine learning (ML) model on client training data” as “In synchronous parallel training, we start with a global model, and then separately in parallel perform a round of FEDAVG (with the decentralized data) and steps of training (with the centralized data). This yields two updated versions of the model, one via FL and one via central training. After every round, we take these two versions of the model, merge them together (e.g., average the weights) to form a new global model, and repeat. See Algorithm 2 for details” (Page 2, Section 2) and “for client…in parallel do…end” (Page 10). The examiner further notes that the algorithm 2 (See Page 10) depicts the synchronous parallel training that includes training a client model with client data. The examiner further notes that Augenstein teaches “while training the client ML model: obtaining, from a server, server model weights of a server ML model trained on server training data separately from and in parallel with the client ML model” as “In synchronous parallel training, we start with a global model, and then separately in parallel perform a round of FEDAVG (with the decentralized data) and steps of training (with the centralized data). This yields two updated versions of the model, one via FL and one via central training. After every round, we take these two versions of the model, merge them together (e.g., average the weights) to form a new global model, and repeat. See Algorithm 2 for details” (Page 2, Section 2) and “Generalized FEDAVG in Parallel with Central Training…Initialize central model…Perform update…Input: initial model…for client…in parallel do…end” (Page 10). The examiner further notes that the algorithm 2 (See Page 10) depicts the synchronous parallel training (which includes central training of the server model that is separate from the federated learning that updates the client model) that includes the obtaining of server model weights of a trained server model at the client model while training that client model in the broadest reasonable interpretation. The examiner further notes that Augenstein teaches “the server training data different than the client training data” as “In synchronous parallel training, we start with a global model, and then separately in parallel perform a round of FEDAVG (with the decentralized data) and steps of training (with the centralized data). This yields two updated versions of the model, one via FL and one via central training. After every round, we take these two versions of the model, merge them together (e.g., average the weights) to form a new global model, and repeat. See Algorithm 2 for details” (Page 2, Section 2) and “Input: initial model…for client…in parallel do…end” (Page 10). The examiner further notes that the algorithm 2 (See Page 10) depicts the synchronous parallel training that includes server model weights that are distinct from client model weights. The examiner further notes that Augenstein teaches “transmitting, to the server, client model weights of the client ML model” as “In synchronous parallel training, we start with a global model, and then separately in parallel perform a round of FEDAVG (with the decentralized data) and steps of training (with the centralized data). This yields two updated versions of the model, one via FL and one via central training. After every round, we take these two versions of the model, merge them together (e.g., average the weights) to form a new global model, and repeat. See Algorithm 2 for details” (Page 2, Section 2) and “Input: initial model…for client…in parallel do…end…Update Global Model…end” (Page 10). The examiner further notes that the algorithm 2 (See Page 10) depicts the synchronous parallel training that includes client model weights being transmitted to the global model for subsequent updating of the global model. The examiner further notes that Augenstein teaches “updating the client ML model using the server model weights” as “In synchronous parallel training, we start with a global model, and then separately in parallel perform a round of FEDAVG (with the decentralized data) and steps of training (with the centralized data). This yields two updated versions of the model, one via FL and one via central training. After every round, we take these two versions of the model, merge them together (e.g., average the weights) to form a new global model, and repeat. See Algorithm 2 for details” (Page 2, Section 2) and “Input: initial model…for client…in parallel do…end…Update Global Model…end” (Page 10). The examiner further notes that the algorithm 2 (See Page 10) depicts the synchronous parallel training that includes client model being updated during every iteration using the server model weights. The examiner further notes that Augenstein teaches “obtaining, from the server, updated server model weights of the server ML model, the updated server model weights updated based on the transmitted client model weights” as “In synchronous parallel training, we start with a global model, and then separately in parallel perform a round of FEDAVG (with the decentralized data) and steps of training (with the centralized data). This yields two updated versions of the model, one via FL and one via central training. After every round, we take these two versions of the model, merge them together (e.g., average the weights) to form a new global model, and repeat. See Algorithm 2 for details” (Page 2, Section 2) and “Input: initial model…for client…in parallel do…end…Update Global Model…end” (Page 10). The examiner further notes that the algorithm 2 (See Page 10) depicts the synchronous parallel training that includes client model being updated after every subsequent iteration using the updated server model weights that are updated from transmitted client weights during a previous iteration. The examiner further notes that Augenstein teaches “further updating the client ML model using the updated server model weights” as “In synchronous parallel training, we start with a global model, and then separately in parallel perform a round of FEDAVG (with the decentralized data) and steps of training (with the centralized data). This yields two updated versions of the model, one via FL and one via central training. After every round, we take these two versions of the model, merge them together (e.g., average the weights) to form a new global model, and repeat. See Algorithm 2 for details” (Page 2, Section 2) and “Input: initial model…for client…in parallel do…end…Update Global Model…end” (Page 10). The examiner further notes that the algorithm 2 (See Page 10) depicts the synchronous parallel training that includes client model being updated after every subsequent iteration using the updated server model weights.
Augenstein does not explicitly teach:
D) wherein the server model weights comprise non-trainable weights of the server ML model.
Sidahmed, however, teaches “wherein the server model weights comprise non-trainable weights of the server ML model” as “we propose the use of partially trainable networks (PTNs) to tackle the communication and computation burdens of training large models, which can make FL more accessible to various applications. Based on the generalized FedAvg algorithm, we only communicate the trainable parameters (which can represent as little as 2% of the total network parameter count in our study), and a random seed from server to clients. Clients reconstruct the full model by regenerating the frozen parameters from the random seed, perform local training on private data, and send back the updates on the trainable parameters to the server” (Section 1, Page 2).
The examiner further notes that Sidahmed teaches the concept of frozen parameters (i.e. non-trainable weights) being sent from a server to client models via a random seed. The combination would result in Augenstein sending such frozen weights to its client models.
It would have been obvious to one of ordinary skill in the art before the effective filing date of instant invention to combine the teachings of the cited references because teaching Sidahmed’s would have allowed Augenstein’s to provide a method for improving federated learning efficiency via saved communication, as noted by Sidahmed (Section 1, Page 2).
Regarding claims 4 and 14, Augenstein teaches a computer-implemented method and system comprising:
A) wherein the client ML model is trained locally on the client device using the client training data that is exclusively stored on the client device (Page 1, Section 1, Page 2, Section 2, Page 3, Section 3, Page 10).
The examiner notes that Augenstein teaches “wherein the client ML model is trained locally on the client device using the client training data that is exclusively stored on the client device” as “Federated learning (FL) [McMahan et al., 2017] is a machine learning setting where multiple ‘clients’ (typically, edge computing devices like mobile phones) collaborate in solving a machine learning problem, under the coordination of a central server. Each client caches raw training data locally, and the data are never exchanged or transferred” (Page 1, Section 1), “In synchronous parallel training, we start with a global model, and then separately in parallel perform a round of FEDAVG (with the decentralized data) and steps of training (with the centralized data). This yields two updated versions of the model, one via FL and one via central training. After every round, we take these two versions of the model, merge them together (e.g., average the weights) to form a new global model, and repeat. See Algorithm 2 for details” (Page 2, Section 2), “We use aversion of the CelebA dataset [Liu et al., 2015], made into a federated dataset1 by partitioning images up by portrait subject [Caldas et al., 2018]. I.e., we treat each celebrity as having a mobile phone with a cache of photos of themselves, which participates as a client in FL” (Page 3, Section 3), and “for client…in parallel do…end” (Page 10). The examiner further notes that the algorithm 2 (See Page 10) depicts the synchronous parallel training that includes training a client model with client data (i.e. data stored exclusively at the client).
Regarding claims 5 and 15, Augenstein teaches a computer-implemented method and system comprising:
A) wherein the client training data comprises sensitive data corresponding to the client device (Page 1, Section 1, Page 2, Section 2, Page 3, Section 3, Page 10).
The examiner notes that Augenstein teaches “wherein the client training data comprises sensitive data corresponding to the client device” as “Federated learning (FL) [McMahan et al., 2017] is a machine learning setting where multiple ‘clients’ (typically, edge computing devices like mobile phones) collaborate in solving a machine learning problem, under the coordination of a central server. Each client caches raw training data locally, and the data are never exchanged or transferred” (Page 1, Section 1), “In synchronous parallel training, we start with a global model, and then separately in parallel perform a round of FEDAVG (with the decentralized data) and steps of training (with the centralized data). This yields two updated versions of the model, one via FL and one via central training. After every round, we take these two versions of the model, merge them together (e.g., average the weights) to form a new global model, and repeat. See Algorithm 2 for details” (Page 2, Section 2), “We use aversion of the CelebA dataset [Liu et al., 2015], made into a federated dataset1 by partitioning images up by portrait subject [Caldas et al., 2018]. I.e., we treat each celebrity as having a mobile phone with a cache of photos of themselves, which participates as a client in FL” (Page 3, Section 3), and “for client…in parallel do…end” (Page 10). The examiner further notes that the algorithm 2 (See Page 10) depicts the synchronous parallel training that includes training a client model with client data (i.e. data stored exclusively at the client). Such client data includes examples of “sensitive” data (See example of celebrity photos stored on their device).
Regarding claims 6 and 16, Augenstein teaches a computer-implemented method and system comprising:
A) wherein the operations further comprise obtaining, from the server, the server model weights at a predetermined interval (Page 2, Section 2, Page 10).
The examiner notes that Augenstein teaches “wherein the operations further comprise obtaining, from the server, the server model weights at a predetermined interval” as “In synchronous parallel training, we start with a global model, and then separately in parallel perform a round of FEDAVG (with the decentralized data) and steps of training (with the centralized data). This yields two updated versions of the model, one via FL and one via central training. After every round, we take these two versions of the model, merge them together (e.g., average the weights) to form a new global model, and repeat. See Algorithm 2 for details” (Page 2, Section 2) and “Input: initial model…for client…in parallel do…end…Update Global Model…end” (Page 10). The examiner further notes that synchronous parallel training depicted in Algorithm 2 is performed at predetermined intervals.
Regarding claims 8 and 18, Augenstein teaches a computer-implemented method and system comprising:
A) wherein the predetermined interval comprises a number of training steps (Page 2, Section 2, Page 10).
The examiner notes that Augenstein teaches “wherein the predetermined interval comprises a number of training steps” as “In synchronous parallel training, we start with a global model, and then separately in parallel perform a round of FEDAVG (with the decentralized data) and steps of training (with the centralized data). This yields two updated versions of the model, one via FL and one via central training. After every round, we take these two versions of the model, merge them together (e.g., average the weights) to form a new global model, and repeat. See Algorithm 2 for details” (Page 2, Section 2) and “Input: initial model…for t…for client…in parallel do…end…Update Global Model…end” (Page 10). The examiner further notes that synchronous parallel training depicted in Algorithm 2 is performed at predetermined intervals of a number of training steps.
Regarding claims 9 and 19, Augenstein teaches a computer-implemented method and system comprising:
A) wherein the operations further comprise causing the server to update the server ML model by transmitting, to the server, client ML model weights of the client ML model (Page 2, Section 2, Page 10).
The examiner notes that Augenstein teaches “wherein the operations further comprise causing the server to update the server ML model by transmitting, to the server, client ML model weights of the client ML model” as “In synchronous parallel training, we start with a global model, and then separately in parallel perform a round of FEDAVG (with the decentralized data) and steps of training (with the centralized data). This yields two updated versions of the model, one via FL and one via central training. After every round, we take these two versions of the model, merge them together (e.g., average the weights) to form a new global model, and repeat. See Algorithm 2 for details” (Page 2, Section 2) and “Input: initial model…for client…in parallel do…end…Update Global Model…end” (Page 10). The examiner further notes that the global model is updated from the client model weights that are transmitted as depicted in Algorithm 2.
8. Claims 2 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Augenstein et al. (Article entitled “Jointly Learning from Decentralized (Federated) and Centralized Data to Mitigate Distribution Shift”, dated 23 November 2021), in view of Sidahmed et al. (Article entitled “Efficient and Private Federated Learning with Partially Trainable Networks”, dated 08 November 2021) as applied to claims 1, 4-6, 8-9, 11, 14-16, and 18-19 above, and further in view of Anwar et al. (U.S. PGPUB 2022/0156633).
9. Regarding claims 2 and 12, Augenstein and Sidahmed do not explicitly teach a computer-implemented method and system comprising:
A) wherein the client ML model is randomly initialized.
Anwar, however, teaches “wherein the client ML model is randomly initialized” as “The local model 22 can be randomly initialised” (Paragraph 39).
The examiner further notes that although Sidahmed sends random seeds to its client ML model, there is no explicit teaching of a random initialization of such a client ML model. Nevertheless, Anwar teaches the concept of randomly initializing a local model in a federated environment. The combination would result in the local models of Augenstein to be randomly initialized.
It would have been obvious to one of ordinary skill in the art before the effective filing date of instant invention to combine the teachings of the cited references because teaching Anwar’s would have allowed Augenstein’s and Sidahmed’s to provide a method for improving federated learning efficiency, as noted by Anwar (Paragraph 14).
10. Claims 3 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Augenstein et al. (Article entitled “Jointly Learning from Decentralized (Federated) and Centralized Data to Mitigate Distribution Shift”, dated 23 November 2021), in view of Sidahmed et al. (Article entitled “Efficient and Private Federated Learning with Partially Trainable Networks”, dated 08 November 2021) as applied to claims 1, 4-6, 8-9, 11, 14-16, and 18-19 above, and further in view of Niu et al. (Article entitled “Billion-Scale Federated Learning on Mobile Clients: A Submodel Design with Tunable Privacy”, dated 25 September 2020).
11. Regarding claims 3 and 13, Augenstein and Sidahmed do not explicitly teach a computer-implemented method and system comprising:
A) wherein the server ML model is randomly initialized.
Niu, however, teaches “wherein the server ML model is randomly initialized” as “At the initial stage, the cloud server randomly initializes the global model (Line 1)” (Page 410, Section 4.2.1).
The examiner further notes that Niu teaches the concept of randomly initializing a global model in a federated environment. The combination would result in the global model of Augenstein to be randomly initialized.
It would have been obvious to one of ordinary skill in the art before the effective filing date of instant invention to combine the teachings of the cited references because teaching Niu’s would have allowed Augenstein’s and Sidahmed’s provide a method for improving federated learning efficiency, as noted by Niu (Page 416, Section 7).
12. Claims 7 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Augenstein et al. (Article entitled “Jointly Learning from Decentralized (Federated) and Centralized Data to Mitigate Distribution Shift”, dated 23 November 2021), in view of Sidahmed et al. (Article entitled “Efficient and Private Federated Learning with Partially Trainable Networks”, dated 08 November 2021) as applied to claims 1, 4-6, 8-9, 11, 14-16, and 18-19 above, and further in view of Balakrishnan et al. (U.S. PGPUB 2021/0119901).
13. Regarding claims 7 and 17, Augenstein and Sidahmed do not explicitly teach a computer-implemented method and system comprising:
A) wherein the predetermined interval comprises a time period.
Balakrishnan, however, teaches “wherein the predetermined interval comprises a time period” as “The federated learning may use a multi-access edge computing (MEC) server that learns a global machine learning model from multiple clients; each of which learn a local machine learning model from the dataset available to them. Instead of the clients sending all the data to a central MEC server to train a machine learning algorithm-such as a deep neural network, recurrent neural network (RNN) etc.—federated learning enables the clients to train a model on their data locally and only share the model weights to the MEC server. The MEC server receives model updates from several clients and calculates the average (or other statistical combination) of the model weights to form the global model. For the next round of client training, the MEC server propagates the global model weights to clients. A federated learning epoch is the time between the beginning of the round and the end of the round. Any data received after the epoch is not part of the round, though the data may be used in a subsequent round” (Paragraph 33).
The examiner further notes that Balakrishnan teaches the concept of performing federated learning for a predetermined epoch (i.e. the claimed time period). The combination would result in the federated learning of Augenstein to be performed for a predetermined interval of a period of time.
It would have been obvious to one of ordinary skill in the art before the effective filing date of instant invention to combine the teachings of the cited references because teaching Balakrishnan’s would have allowed Augenstein’s and Sidahmed’s to provide a method for avoiding overhead in federated learning, as noted by Balakrishnan (Paragraph 18).
14. Claims 10 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Augenstein et al. (Article entitled “Jointly Learning from Decentralized (Federated) and Centralized Data to Mitigate Distribution Shift”, dated 23 November 2021), in view of Sidahmed et al. (Article entitled “Efficient and Private Federated Learning with Partially Trainable Networks”, dated 08 November 2021) as applied to claims 1, 4-6, 8-9, 11, 14-16, and 18-19 above, and further in view of Beaufays et al. (WO 2022/019885).
15. Regarding claims 10 and 20, Augenstein does not explicitly teach a computer-implemented method and system comprising:
A) wherein the client ML model comprises a local hotword detection model.
Beaufays, however, teaches “wherein the client ML model comprises a local hotword detection model” as “As some non-limiting example, the hotword detection engine 222 can utilize a combined hotword detection model 222A” (Paragraph 52).
The examiner further notes that Beaufays teaches the concept pf a local hotword detection model in a federated environment. The combination would result in the local models of Augenstein to be hotword detection models.
It would have been obvious to one of ordinary skill in the art before the effective filing date of instant invention to combine the teachings of the cited references because teaching Beaufays’s would have allowed Augenstein’s and Sidahmed’s to provide a method for training models for specific uses, as noted by Beasufays (Paragraph 67).
Response to Arguments
16. Applicant’s arguments with respect to claims 1-20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument (See newly applied art of Sidahmed).
Applicant's arguments filed 07/31/2026 have been fully considered but they are not persuasive.
Applicants argue on Page 05 that “Augenstein does not maintain a separate, persisting server ML model "trained separately from and in parallel with the client ML model" whose weights are obtained by the client; rather, Augenstein derives both sub-models from, and merges them back into, one global model each round”. However, the examiner wishes to refer to Augenstein which states “In synchronous parallel training, we start with a global model, and then separately in parallel perform a round of FEDAVG (with the decentralized data) and steps of training (with the centralized data). This yields two updated versions of the model, one via FL and one via central training. After every round, we take these two versions of the model, merge them together (e.g., average the weights) to form a new global model, and repeat. See Algorithm 2 for details” (Page 2, Section 2) and “Generalized FEDAVG in Parallel with Central Training…Initialize central model…Perform update…Input: initial model…for client…in parallel do…end” (Page 10). The examiner further notes that the algorithm 2 (See Page 10) depicts the synchronous parallel training (which includes central training of the server model that is separate from the federated learning that updates the client model) that includes the obtaining of server model weights of a trained server model at the client model while training that client model in the broadest reasonable interpretation.
Conclusion
17. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
U.S. PGPUB 2022/0398500 issued to Singhal et al. on 15 December 2022. The subject matter disclosed therein is pertinent to that of claims 1-20 (e.g., methods to perform federated learning).
U.S. PGPUB 2022/0293093 issued to Beaufays et al. on 15 September 2022. The subject matter disclosed therein is pertinent to that of claims 1-20 (e.g., methods to perform federated learning).
U.S. PGPUB 2023/0359907 issued to Augenstein et al. on 09 November 2023. The subject matter disclosed therein is pertinent to that of claims 1-20 (e.g., methods to perform federated learning).
18. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Contact Information
19. Any inquiry concerning this communication or earlier communications from the examiner should be directed to Mahesh Dwivedi whose telephone number is (571) 272-2731. The examiner can normally be reached on Monday to Friday 8:20 am – 4:40 pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Charles Rones can be reached (571) 272-4085. The fax number for the organization where this application or proceeding is assigned is (571) 273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see 20. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Mahesh Dwivedi
Primary Examiner
Art Unit 2168
August 25, 2026
/MAHESH H DWIVEDI/Primary Examiner, Art Unit 2168
Prosecution Insights