DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is in response to communications filed on 06/03/2023. Claims 1-10 are pending and have been examined.
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 3-7, and 9-10 are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al. (“Friends to Help: Saving Federated Learning from Client Dropout”, arXiv:2205.13222v1, May 26 2022, 11 pages) in view of Eykholt et al. (US 20220180157 A1) and Froelicher et al. (US 20230188319 A1).
As per independent claim 1, Wang teaches a federated learning method of protecting data digest (e.g. section 1, “Federated learning (FL) is a distributed machine learning paradigm where a set of clients with decentralized data work collaboratively to learn a model under the coordination of a centralized server” and section 2, “We consider a typical FL algorithm working in the client dropout setting. Clients upload their local model updates to the server. Instead of uploading the local model itself, clients can simply upload the local model updates”) comprising:
sending a general model to each of a plurality of client devices by a moderator (e.g. section 2, “Global model download. Each client downloads the global model from the server”);
executing a digest producer by each of the plurality of client devices (e.g. section 2, “Local model update. Each client uses the initial model to train a new local model”);
performing a training procedure by each of the plurality of client devices, wherein the training procedure comprises: updating the general model to generate a client model according to the plurality of raw data and a present client loss function (e.g. section 2 and section 4.2, “Global model download. Each client downloads the global model from the server. Local model update. Each client uses the initial model to train a new local model, typically by using mini-batch stochastic gradient descent (SGD) as follows: [EQ2]” and equation 3 loss);
sending an update parameter of the client model to the moderator (e.g. section 2, “Clients upload their local model updates to the server. Instead of uploading the local model itself, client k can simply upload the local model update, which is defined as the accumulative model parameter difference as follows: [EQ3]”);
determining an absent client and a present client among the plurality of client devices by the moderator, generating a replacement model according to the general model, the digest of the absent client and an absent client loss function by the moderator (e.g. section 1, “The main insight derived from this analysis is that the FL convergence performance depends on the total substitution error [i.e. loss] (i.e., the difference between the true update and its substitute) over the entire course of learning”, section 3, “we consider a larger class of FL problems that include client dropout as a special case. Specifically, imagine that a dropout client k, instead of contributing nothing, uses a substitute when submitting its local model update” and section 4, “we develop a new algorithm to improve the convergence bound of FL with client dropout. Our key idea is to find a better substitute when client k drops out in round t. This is possible by noticing that σ2ij are different across client pairs and the local model updates are more similar when the clients’ data distributions are more similar. Thus, when a client i drops out, one can use the local model update as a replacement if j shares a similar data distribution with i, or in our terminology, j is a friend of i… Clients. For any dropout client k, the server looks up the similarity score between k and every non-dropout client, finds the one with the highest similarity score, and uses the local model update as a substitute when computing the global update”);
performing an aggregation to generate an aggregation model according to the update parameter of the client model of the present client and an update parameter of the replacement model of the absent client by the moderator and training the aggregation model to update the general model according to a moderator loss function by the moderator (e.g. section 2, “We consider a server and a set of K clients, indexed by K = {1, . . ., K}, who work together to train a machine learning model by solving a distributed optimization problem: [EQ1]… The server updates the global model by using the aggregated local model updates of the clients in [EQ4]” and section 3, “we consider a larger class of FL problems that include client dropout as a special case. Specifically, imagine that a dropout client k, instead of contributing nothing, uses a substitute when submitting its local model update… rather than completely ignoring the dropout clients, we write the aggregate model update in a different way to include all clients in the equation: [EQ5, i.e. loss]. In other words, although the dropout clients did not participate in the round t’s learning, it is equivalent to the case where a dropout client uses a substitute of its true local update”),
but does not specifically teach to generate a plurality of encoded features according to a plurality of raw data, updating according to the plurality of encoded features, a plurality of labels corresponding to the plurality of encoded features, and selecting at least two of the plurality of encoded features to compute a feature weighted sum, computing a sum of the feature weighted sum and noise, selecting at least two of the plurality of labels to compute a label weighted sum, and sending the sum and the label weighted sum to the moderator as a digest when receiving a digest request.
However, Eykholt teaches generate a plurality of encoded features according to a plurality of raw data, updating according to the plurality of encoded features and a plurality of labels corresponding to the plurality of encoded features, selecting at least two of the plurality of encoded features to compute a feature weighted sum, selecting at least two of the plurality of labels to compute a label weighted sum, and sending a sum and the label weighted sum to the moderator as a digest when receiving a digest request (e.g. in paragraphs 20, 22-24, and 33, “DNN has been described in literature by a function g: X [Wingdings font/0xE0] Y, where X is an input space, and Y is an output space representing a categorical set. Each f, represents a layer, and FL is the last output layer. The last output layer creates a mapping from a hidden space to the output space (class labels) through a softmax function that outputs a vector of real numbers in the range [0, 1] that add up to 1… The DNN 100 is trained using a training data set, thereby resulting in generation of a set of weights corresponding to the trained DNN. The output of the network is a prediction g(xi) associated with the ith sample. To train the DNN, the difference between a predicted output g(xi) and its true label, yi, is modeled with a loss function, J(g(xi), yi), which is back-propagated into the network to update the model parameters…Typically, a neural network model such as depicted in FIG. 1 accepts numeric values as inputs. To work with categorical data, however, such data typically needs to be encoded in some manner. One-hot encoding is a known technique that converts category data into integers or a vector of ones and zeros”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Wang to include the teachings of Eykholt because one of ordinary skill in the art would have recognized the benefit of facilitating an adversarially robust network,
but does not specifically teach computing a sum of the feature weighted sum and noise.
However, Froelicher teaches computing a sum of feature weighted sum and noise (e.g. in paragraphs 29 and 92, “to standardize the dataset, each DP, for each feature, locally computes the sum of its value, the sum of its squared values, and the number of samples. These values are encrypted and then aggregated among all DPs. Then, the DPs collectively and obliviously add randomly sampled noise from a deviation”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of the combination to include the teachings of Froelicher because one of ordinary skill in the art would have recognized the benefit of facilitating differential privacy.
As per claim 3, the rejection of claim 1 is incorporated and the combination further teaches wherein the general model comprises a first feature extractor, a second feature extractor and a classifier (e.g. Eykholt, in paragraphs 20-21, “a DNN has been described in literature by a function g: X [Wingdings font/0xE0] Y, where X is an input space, and Y is an output space representing a categorical set. Each f, represents a layer, and FL is the last output layer. The last output layer creates a mapping from a hidden space to the output space (class labels) through a softmax function that outputs a vector of real numbers in the range [0, 1] that add up to 1…in a DNN 100 typically the neurons are aggregated in layers, and different layers may perform different transformations on their inputs. As depicted, signals (typically real numbers) travel from the first layer (the input layer) 102 to the last layer (the output layer) 104, via traversing one or more intermediate (the hidden layers) 106. Hidden layers 106 provide the ability to extract features from the input layer 102. As depicted in FIG. 1, there are two hidden layers, but this is not a limitation”), and updating the general model to generate the client model according to the plurality of raw data, the plurality of encoded features, the plurality of labels corresponding to the plurality of encoded features, and the present client loss function comprises: inputting the plurality of raw data to the first feature extractor to generate a first feature, inputting the plurality of encoded features to the second feature extractor to generate a second feature (e.g. Wang, section 2, “We consider a server and a set of K clients, indexed by K = {1, . . ., K}, who work together to train a machine learning model by solving a distributed optimization problem [EQ1]… The server updates the global model by using the aggregated local model updates of the clients in (EQ4)”), inputting a concatenation of the first feature and the second feature to the classifier to generate a predicted result (e.g. Eykholt, in paragraphs 18 and 22, “Each network unit receives an input vector from its connected neurons and outputs a value that will be passed to the following layers… During training, parameters related to each layer arerandomly initialized, and input samples (xi, yi) are fed through the network. The output of the network is a prediction g(xi) associated with the ith sample”);; and inputting the predicted result and an actual result to the present client loss function, and adjusting a weight of at least one of the first feature extractor, the second feature extractor, and the classifier according to an output of the present client loss function (e.g. Eykholt, in paragraph 22, “The DNN 100 is trained using a training data set, thereby resulting in generation of a set of weights corresponding to the trained DNN. The output of the network is a prediction g(xi) associated with the ith sample. To train the DNN, the difference between a predicted output g(xi) and its true label, yi, is modeled with a loss function, J(g(xi), yi), which is back-propagated into the network to update the model parameters”).
As per claim 4, the rejection of claim 1 is incorporated and the combination further teaches wherein the general model comprises a first feature extractor, a second feature extractor and a classifier (e.g. Eykholt, in paragraphs 20-21, “a DNN has been described in literature by a function g: X [Wingdings font/0xE0] Y, where X is an input space, and Y is an output space representing a categorical set. Each f, represents a layer, and FL is the last output layer. The last output layer creates a mapping from a hidden space to the output space (class labels) through a softmax function that outputs a vector of real numbers in the range [0, 1] that add up to 1…in a DNN 100 typically the neurons are aggregated in layers, and different layers may perform different transformations on their inputs. As depicted, signals (typically real numbers) travel from the first layer (the input layer) 102 to the last layer (the output layer) 104, via traversing one or more intermediate (the hidden layers) 106. Hidden layers 106 provide the ability to extract features from the input layer 102. As depicted in FIG. 1, there are two hidden layers, but this is not a limitation”), and generating the replacement model according to the digest of the absent client and the absent client loss function comprises: inputting the digest of the absent client to a guidance producer to generate a piece of guidance (e.g. Wang, section 4, “we develop a new algorithm to improve the convergence bound of FL with client dropout. Our key idea is to find a better substitute when client k drops out in round t. This is possible by noticing that σ2ij are different across client pairs and the local model updates are more similar when the clients’ data distributions are more similar. Thus, when a client i drops out, one can use the local model update as a replacement if j shares a similar data distribution with i, or in our terminology, j is a friend of i” and section 4.1, “Clients. For any dropout client k, the server looks up the similarity score between k and every non-dropout client, finds the one with the highest similarity score, and uses the local model update as a substitute when computing the global update”); inputting the piece of guidance to the first feature extractor to generate a first feature, inputting the digest of the absent client to the second feature extractor to generate a second feature, inputting a concatenation of the first feature and the second feature to the classifier to generate a predicted result (e.g. Eykholt, in paragraphs 18 and 22 “Each network unit receives an input vector from its connected neurons and outputs a value that will be passed to the following layers… During training, parameters related to each layer are randomly initialized, and input samples (xi, yi) are fed through the network. The output of the network is a prediction g(xi) associated with the ith sample”); and inputting the predicted result and an actual result to the absent client loss function, and adjusting a weight of at least one of the first feature extractor, the second feature extractor, and the classifier according to an output of the absent loss function; wherein the replacement model is the general model with an updated weight (e.g. Wang, section 3, “Let
∆
-
t = 1/K
∑
k
ϵ
K
∆
k
t
be the average local model update assuming all clients participated in round t and submitted their true local updates. Thus, et :=
∆
~
kt - ∆kt represents aggregate global update error due to client dropout and local update substitution in round t. Furthermore, let ekt :=
∆
~
kt - ∆kt
∀
k
ϵ
K
/
S
t
be the individual substitution error for an individual dropout client k in round t” and section 4.1, “In each FL round t, in addition to the regular steps in the FL algorithm described in Section 3, FL-FDMS performs different actions depending on whether or not a client drops out. For non-dropout clients, FL-FDMS calculates pairwise similarity scores to learn the similarity between clients. For dropout clients, FL-FDMS uses the historical similarity scores to find non-dropout friend clients and use their local model updates as substitutes”; Eykholt, in paragraph 22, “The DNN 100 is trained using a training data set, thereby resulting in generation of a set of weights corresponding to the trained DNN. The output of the network is a prediction g(xi) associated with the ith sample. To train the DNN, the difference between a predicted output g(xi) and its true label, yi, is modeled with a loss function, J(g(xi), yi), which is back-propagated into the network to update the model parameters”).
As per claim 5, the rejection of claim 1 is incorporated and the combination further teaches wherein performing the aggregation to generate the aggregation model according to the update parameter of the client model of the present client and the update parameter of the replacement model of the absent client comprises: computing a first weighted sum of the update parameter of the client model of the present client and a first weight; computing a second weighted sum of the update parameter of the replacement model and a second weight; and summing a parameter of the general model, the first weighted sum and the second weighted sum to generate a parameter of the aggregation model (e.g. Wang, section 2, “We consider a server and a set of K clients, indexed by K = {1, . . ., K}, who work together to train a machine learning model by solving a distributed optimization problem [EQ1]… The server updates the global model by using the aggregated local model updates of the clients in [EQ4]”, section 3, “we consider a larger class of FL problems that include client dropout as a special case. Specifically, imagine that a dropout client k, instead of contributing nothing, uses a substitute when submitting its local model update”. Section 3 para. 1 recites “rather than completely ignoring the dropout clients, we write the aggregate model update in a different way to include all clients in the equation: [EQ5]. In other words, although the dropout clients did not participate in the round t’s learning, it is equivalent to the case where a dropout client uses a substitute of its true local update”, and section 5, “We compare FL-FDMS with three variants of FedAvg since in this paper we describe FL-FDMS in the context of FedAvg”).
As per claim 6, the rejection of claim 1 is incorporated and the combination further teaches wherein training the aggregation model to update the general model according to the moderator loss function by the moderator comprises: inputting the digest of each of the plurality of client devices to a guidance producer to generate a piece of guidance (e.g. Wang, section 2, “We consider a typical FL algorithm working in the client dropout setting. Clients upload their local model updates to the server. Instead of uploading the local model itself, clients can simply upload the local model updates”); inputting the piece of guidance and the digest of each of the plurality of client devices to the aggregation model to generate a predicted result and inputting the predicted result and an actual result to the moderator loss function, and adjusting a parameter of the aggregation model according to an output of the moderator loss function (e.g. Wang, section 2, “We consider a server and a set of K clients, indexed by K = {1, . . ., K}, who work together to train a machine learning model by solving a distributed optimization problem” (i.e., the moderator loss function). Wang section 3 para. 1 recites “rather than completely ignoring the dropout clients, we write the aggregate model update in a different way to include all clients in the equation [EQ5]. In other words, although the dropout clients did not participate in the round t’s learning, it is equivalent to the case where a dropout client uses a substitute of its true local update”).
Claims 7 and 9-10 are the system claims corresponding to method claims 1 and 3-4, and are rejected under the same reasons set forth and the combination further teaches a plurality of client devices, wherein each of the plurality of client devices comprises: a first processor and a first communication circuit electrically connected to the first processor and a moderator communicably connected to each of the plurality of client devices, wherein the moderator comprises: a second communication circuit and a second processor electrically connected to the second communication circuit (e.g. Wang, section 2, “We consider a server and a set of K clients, indexed by K = {1, . . ., K}, who work together to train a machine learning model by solving a distributed optimization problem”; Eykholt, in paragraph 40, “FIG. 5 depicts an exemplary distributed data processing system in which the deployed system or any other computing task associated with the techniques herein may be implemented. Data processing system 500 includes communications fabric 502, which provides communications between processor unit 504, memory 506, persistent storage 505, communications unit 510, input/output (I/O) unit 512, and display 514”).
Claims 2 and 8 are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al. (“Friends to Help: Saving Federated Learning from Client Dropout”, arXiv:2205.13222v1, May 26 2022, 11 pages) in view of Eykholt et al. (US 20220180157 A1) and Froelicher et al. (US 20230188319 A1) as applied above, and further in view of Dwork et al. (US 20090182797 A1).
As per claim 2, the rejection of claim 1 is incorporated, but the combination does not specifically teach generating the noise according to a Laplace mechanism or Gaussian mechanism of differential privacy. However, Dwork teaches generating noise according to a Laplace mechanism or Gaussian mechanism of differential privacy (e.g. in paragraph 25, “Laplace noise may be added to preserve differential privacy. For example, adding Laplace noise with variance 2.sigma..sup.2 to a function f preserves (.DELTA.f/.sigma.)-differential privacy”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of the combination to include the teachings of Dwork because one of ordinary skill in the art would have recognized the benefit of preserving differential privacy.
Claim 8 is the system claim corresponding to method claim 2 and is rejected under the same reasons set forth.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
For example,
Ben-Itzhak et al. (US 20220083917 A1) teaches “training, by a plurality of nodes, a global multi-layer ML model on the nodes' local training datasets by: (1) training the primary model in a distributed/federated fashion on the entireties (or sub-samplings) of those local training datasets, (2) computing, by each node using the trained version of the primary model, a “training value” measure for each labeled training data instance in the node's local training dataset, where this training value measure indicates how useful/valuable the labeled training data instance is for training purposes, (3) identifying, by each node, a subset of its local training dataset based on the computed training value measures (e.g., the labeled training data instances deemed most valuable for training), and (4) training the secondary model in a distributed/federated fashion on the identified subsets” (e.g. in paragraph 10).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WILLIAM WONG whose telephone number is (571)270-1399. The examiner can normally be reached Monday-Friday 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, TAMARA KYLE can be reached at (571)272-4241. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/W.W/Examiner, Art Unit 2144 05/31/2026
/TAMARA T KYLE/Supervisory Patent Examiner, Art Unit 2144
Prosecution Insights