DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is in response to the Application filed on 08/20/2026. Claims 1, 3, 5-9, 11, 13-16 and 21-23 and 25-27 are pending in the case.
Applicant Response
3. In Applicant’s response dated 08/20/2026, Applicant amended independent claims 1 and 9, and 21 dependent claims 22, 25, and 26, and canceled claim 24; and argued against all objections and rejections previously set forth in the Office Action dated 05/21/2026.
Examiner Comments
4. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Continued Examination under 37 CFR 1.114
5. A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 08/20/2026 has been entered.
Claim Rejections - 35 USC § 103
6. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
7. Claims 1, 3, 5-9, 11, 13-16 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Cosman (Pub. No. US 20210192078 B1, Pub. Date 2021-06-24) in view of Li (NPL: Title: Practical Federated Gradient Boosting Decision Trees; Dae: 11-2019) in further view of Ong ( NPL: Title: ADAPTIVE HISTOGRAM-BASED GRADIENT BOOSTED TREES
FOR FEDERATED LEARNING; Date: December,11 2020, Publisher: arXiv:2012.06670v1 [cs.LG] )
Cosman teaches a computer-implemented method comprising:
training a global machine learning model using federated learning between a plurality of parties (see Cosman: Fig.4A, [0055], “Method 400 can additionally include to privatize the aggregate model updates on the learning server using central differential privacy (block 407). The privatized model updates can then be used to update the server machine learning model (block 408) (i.e. The server machine learning model (global machine learning model) is trained and updated by the privatized model data sent/ transmitted form plurality of parties (client devices) using federated learning as shown in Fig.3A);
distributing the global machine learning model to each of the parties (see Cosman: Fig.4B, [0057], “a machine learning model M0 131 may be received by client device 310 from a server 130 (411). The machine learning model M0 131 may be a previously trained model with privatized federated learning (e.g. a previously updated machine learning model M1 235) or an initial machine learning model.”), […]
receiving personalized model updates from each of the parties (see Cosman: Fig.4A, [0055], “The individual model updates can then be privatized on the set of client devices using separated differential privacy (block 404). The privatized model updates are then sent from the set of client devices to a central learning server (block 405).”), wherein the personalized model updates are generated from updated models boosted locally and produced by each of the parties using their respective local data (see Cosman: Fig.5A, [0071], method 510 includes an operation (block 511) to obtain a weight vector that represents a difference between a previous and recently trained model. This difference represents the model update that is to be transmitted to the learning server to update the current server model. Method 510 additionally includes an operation (block 512) to decompose the weight vector into a unit vector (e.g., unit direction U=W/∥W∥.sub.2) and a magnitude (e.g., radius R=∥W∥.sub.2). Method 500 additionally includes an operation (block 513) to privatize the unit vector. The unit vector can be privatized using any desired mechanism.).”)
As shown above, Cosman teaches private federated learning, to create or update a machine learning model to predict user behavior with items accessible with a device. In some embodiments, the resulting updated machine learning model predicts a desired usage pattern with a content item and/or an application accessible on a user device.
Cosman does not teach the computer-implemented method of comprising:
wherein the parties performing local boosting steps by computing gradients, and hessian statistics relating to the global machine learning model and their private datasets;
rebuilding the global machine learning model using the boosted decision tree;
fusing the personalized model updates to produce a boosted decision tree to update the global machine learning model, wherein an aggregator utilizes an XGBoost algorithm for a retraining of the global machine learning model in an iterative manner, and wherein the XGBoost algorithm is utilized to perform a federated quantile sketch method for the fusing of the model updates to produce a global histogram;
starting with the updated global machine learning model, retraining the global machine learning model in an iterative manner, with the global machine learning model from the personalized model updates acting as an initial model transmitted to the plurality of parties for a next iteration, until a predetermined stopping criterion is achieved.
However, Li teaches the computer-implemented method comprising:
the parties performing local boosting steps by computing gradients, and hessian statistics relating to the global machine learning model and their private datasets (see Li: Fig.1, Pg. 4643, describing first order gradients g and second order Hessians h for Gradient Boosting Decision Trees and use g and h to build trees as shown below.”)
PNG
media_image1.png
334
340
media_image1.png
Greyscale
fusing to produce boosted decision (see Li Fig.1, Sec. 4.2, Pg. 4645, “Since the process of building a tree is similar between different parties, we only describe the process of building a tree in Party Pm, which is shown in Algorithm 2. At first, the parties update the gradients of the local instances. Then, for each instance of Pm, the other parties compute and send the aggregated gradients of the similar instances. Instead of sending each gradient directly, such aggregation on the local party can reduce the communication cost and protect the individual gradients. After all the aggregated gradients are computed and send to Pm, the weighted gradients can be easily computed by summing (fusing) the aggregated gradients. Then, we can build a tree based on the weighted gradients.”)
rebuilding the global machine learning model using the boosted decision tree (see Li Fig.1, Sec. 4.2, Pg. 4645, “Since the process of building a tree is similar between different parties, we only describe the process of building a tree in Party Pm, which is shown in Algorithm 2. At first, the parties update the gradients of the local instances. Then, for each instance of Pm, the other parties compute and send the aggregated gradients of the similar instances. Instead of sending each gradient directly, such aggregation on the local party can reduce the communication cost and protect the individual gradients.”)
Because both Cosman and Li are in the same/similar field of endeavor of selecting machine learning training data, accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify the teaching of Cosman to include a system that fuse to produce boosted decision tree and rebuilding the global machine learning model using the boosted decision tree as taught by Li. One would have been motivated to make such a combination in provide significantly improves the predictive accuracy compared with training with data in individual parties alone and significantly improve the predictive accuracy, and achieve comparable accuracy to the original GBDT with and is close to the model with joint data from all parties. (see Li: Abstract)
Cosman and Li does not teach the computer-implemented method of comprising:
fusing the personalized model updates to produce a boosted decision tree to update the global machine learning model, wherein an aggregator utilizes an XGBoost algorithm for a retraining of the global machine learning model in an iterative manner, and wherein the XGBoost algorithm is utilized to perform a federated quantile sketch method for the fusing of the model updates to produce a global histogram;
starting with the updated global machine learning model, retraining the global machine learning model in an iterative manner, with the global machine learning model from the personalized model updates acting as an initial model transmitted to the plurality of parties for a next iteration, until a predetermined stopping criterion is achieved.
However, Ong teaches the computer-implemented method comprising:
fusing the personalized model updates to produce a boosted decision tree to update the global machine learning model (see Ong: Section 4.2, Page 6, Paragraph 9, “Given the collected results from each party, we perform a fusion operation to merge the final histogram representation used towards boosting the decision tree model. This entails reconstructing the local party’s histogram by re-aligning the bin size from each of the collected histograms accordingly to a particular value. This is determined by a heuristic based on the defined downstream objective loss function, l (line 21-24). If the loss function task is based on a classification loss such as binary cross-entropy or categorical cross-entropy loss, we consider the smallest (i.e. large bin size) computed from each of the local party’s data distribution.”)
wherein an aggregator utilizes an XGBoost algorithm for a retraining of the global machine learning model in an iterative manner (see Ong: Section 4.2, Page 6, Paragraph 4, “To train a gradient boosted decision tree within a FL-setting, the aggregator first initializes a global null model, f(A) (line 1). The aggregator also defines the global hyperparameter (A), which denotes the error tolerance budget for the training process for XGBoost. This parameter sets the upper-bound histogram approximation error and equivalently the number of maximum bins used in training.” … ( see Page 6, Paragraph 8, “After computing the surrogate histogram representation of the data, we then initiate the iterative federated learning process.”),
wherein the XGBoost algorithm is utilized to perform a federated quantile sketch method for the fusing of the model updates to produce a global histogram (see Ong: Section 4.1, Page 5, Paragraph 8, “To mitigate this performance issue, algorithms such as XGBoost [1] and LightGBM [18] utilize a quantile approximation strategy to reduce the overall search space of the split finding procedure as a histogram representation of the raw data distribution. Empirically, quantile approximations of data can provide optimal splits just as well as the exact greedy solution [1, 18, 19]. One observation here is that by quantizing or reducing the resolution of the raw data distribution, we can generate a surrogate representation of the raw data. However, applying this directly into FL settings may result in models that do not adapt to the deviations of each party’s data.”… ( see Page 6, 1st paragraph “adjusting the quantile sketch approximation error hyperparameter 4, used by XGBoost for each individual party. Intuitively, the inverse of the parameter, or 1=, is roughly equivalent to the bin size of the histogram. Therefore, this approximated surrogate representation of the party’s data can be transmitted to the aggregator without directly exposing the precise representation of the party’s local raw dataset.”)
starting with the updated global machine learning model, retraining the global machine learning model in an iterative manner (see Ong: Section 3.1, Page 3, Last Paragraph, “adjusting the quantile sketch approximation error hyperparameter 4, used by XGBoost for each individual party. Intuitively, the inverse of the parameter, or 1=, is roughly equivalent to the bin size of the histogram. Therefore, this approximated surrogate representation of the party’s data can be transmitted to the aggregator without directly exposing the precise representation of the party’s local raw dataset.”), with the global machine learning model from the personalized model updates acting as an initial model transmitted to the plurality of parties for a next iteration, until a predetermined stopping criterion is achieved (see Ong: Section 4.2, Page 8, 1st Paragraph “the merged gradients, G(A) m , hessians, H(A) m , and feature-value split candidates, D (A) m which is used for the boosting process (line 25-26). With a new f(A) t generated, we repeat our training process for T rounds, or until some other stopping criteria depending on whether early-stopping or other heuristics are considered (line 27).”
Because both Cosman, Li and Ong are in the same/similar field of endeavor of federated leering system, accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify the teaching of Cosman to include the iterative federated learning system that distribute global learning model to personal models to update the global model and personal model until a certain criterion is achieved as taught by Ong. One would have been motivated to make such a combination in improve training efficiency, model accuracy and model by improving the efficiency of federated learning systems is needed for improved scalability and usability.
Regarding Claim 3,
As shown above, Cosman, Li and Ong teaches all the limitations of claim 2. Cosman further teaches the system wherein training the global machine learning model comprises:
establishing a connection between an aggregator and the plurality of parties (see Cosman: Fig.1, [0040], “system 100 includes a server 130 that can receive data from a set of client devices 110a-110n, 111a-111n, 112a-112n over a network 120.”)
computing, by the aggregator, party-centric epsilon parameters for each party of the plurality of parties based on a global epsilon parameter (see Cosman: Fig.1, [0037], “Using separated differential privacy, a private federated learning system can be enabled that provides comparable utility to a federated learning system that does not provide privacy safeguards. Privacy is enabled by obfuscating the individual updates to the server. In one embodiment a relaxed privacy parameter ε is used, user data is still protected against reconstruction by individuals (e.g., internal employees) that may have access to privatized updates. In one embodiment, fully ε-differentially-private techniques (a global epsilon parameter) are used to enable privatization of the magnitude. In another embodiment, relative noise mechanisms are used to privatize the magnitude.”);
transmitting a party-centric epsilon parameter to each of the plurality of parties (see Cosman: Fig.1, [0038], “The use of central differential privacy provides additional protection for updated learning models on the server against external adversaries that may have access to the model and any other information except the user data that the adversary wishes to decode.”)
receiving, from the parties, histogram data distributions representing raw training data maintained by the parties (see Cosman: Fig.2, [0048], “generate privatized model updates 212a-212c (e.g., privatized model update 212a from client device 210a, privatized model update 212b from client device 210b, privatized model update 212c from client device 210c), which can be transmitted to the server 130 via the network 120. The privatized model updates 212a-212c can be stripped of their IP addresses or other information that can be used to identify the client devices 210 prior to entering an ingestor 232 on the server 130.”)
generating a global machine learning model based on the histogram data distributions (see Cosman: Fig.2, [0048], “The ingestor 232 can collect the data from the client devices 210 and remove metadata and forwards the data to an aggregator 233. The aggregator takes the privatized model updates and aggregates them to form a single update to the current server model, which in the initial round is machine learning model 131 (e.g., model M0).)”;
transmitting the global machine learning model to the parties (see Cosman: Fig.4B, [0057], “a machine learning model M0 131 may be received by client device 310 from a server 130 (411). The machine learning model M0 131 may be a previously trained model with privatized federated learning (e.g. a previously updated machine learning model M1 235) or an initial machine learning model.”);
fusing the model updates to produce a global histogram to update the global machine learning model (see Cosman: Fig.2, [0048], “The privatized model updates 212a-212c can be stripped of their IP addresses or other information that can be used to identify the client devices 210 prior to entering an ingestor 232 on the server 130. The ingestor 232 can collect the data from the client devices 210 and remove metadata and forwards the data to an aggregator 233. The aggregator takes the privatized model updates and aggregates them to form a single update to the current server model, which in the initial round is machine learning model 131 (e.g., model M0).”
receiving model updates from each of the parties, wherein each of the model updates includes a histogram, a gradient, and the hessian statistics based on applying the global machine learning model to training data stored by the parties’ datasets (see Li: Fig.1, Pg. 4643, describing first order gradients g and second order Hessians h for Gradient Boosting Decision Trees and use g and h to build trees as shown above.)
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify the teaching of Cosman to include a system that receive model updates includes a histogram, a gradient, and hessian statistics based on applying the global machine learning model to training data stored by the parties’ datasets as taught by Li. One would have been motivated to make such a combination in provide significantly improves the predictive accuracy compared with training with data in individual parties alone and significantly improve the predictive accuracy, and achieve comparable accuracy to the original GBDT with and is close to the model with joint data from all parties. (see Li: Abstract)
Regarding Claim 5,
As shown above, Cosman, Li and Ong teaches all the limitations of claim 1. Cosman further teaches the system wherein:
the party-centric epsilon parameter is specific to each party of the plurality of parties based on sizes of the training data provided by each party (see Cosman: Fig.4A, [0083], “a decoder may reconstruct the sequence of layers from the last hidden state representation. Upon comparison of the reconstructed layer surface data with a layer surface data of the layer being printed, the encoder-decoder based model may identify a deviation between the reconstructed layer surface data and the layer surface data of the layer being printed. Such a deviation is indicated as a predicted anomaly score.”);
Regarding Claim 6,
As shown above, Cosman, Li and Ong teaches all the limitations of claim 1. Cosman further teaches the system wherein:
the personalized model updates are produced using gradient descent on a per-party loss (see Cosman: Fig.16, [0152], “act 1602 can include accessing, by a device (e.g., 114) operatively coupled to a processor, a first set of data candidates (e.g., 106) and a second set of data candidates (e.g., 108), wherein a machine learning model (e.g., 104) is trained on the first set of data candidates.”)
Regarding Claim 7,
As shown above, Cosman, Li and Ong teaches all the limitations of claim 1. Cosman further teaches the system wherein:
model-agnostic meta-learning is used to train the machine learning model (see Cosman: Fig.3, [0050], “The receive module 351 can remove latent identifiers such as IP addresses or other data that might identify the client device 310. The ingestor/aggregator can include components of the ingestor 232 and aggregator 233 shown in FIG. 2 and can perform similar operations, such as removing metadata, session identifiers, and other identifying information, and aggregating the privatized information to generate an aggregated model update 331.’)
Regarding Claim 8,
As shown above, Cosman, Li and Ong teaches all the limitations of claim 1. Cosman further teaches the system wherein:
the updated global machine learning model mimics trees obtained from local boosting by the parties using their local distribution to add personalization (see Li: Fig.1, Sec.4, Pg. 4644, “each party can use the global hash tables for tree building without accessing
other party’s raw data. In the training stage, all parties together train a number of trees one by one using the similarity information. Once a tree is built in a party, it will be sent to all the other parties for the update of gradients.”)
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify the teaching of Cosman to include a system that updated global machine learning model mimics trees obtained from local boosting by the parties using their local distribution to add personalization as taught by Li. One would have been motivated to make such a combination in provide smaller, more efficient training datasets and a lag-free and efficient machine learning model training system.
Regarding independent claim 9
Claim 9 is directed to a computer implemented method claim and has similar/same claim limitation as Claim 1 and is rejected under same rationale.
Regarding Claim 11, and 13-16,
Claims 11 and 13-16 are directed to a computer implemented method claim and have similar/same claim limitation as Claims 2-8, respectively and are rejected under same rationale.
Regarding independent Claim 21,
Claim 21 is directed to a system claim and has similar/same claim limitation as Claim 1 and is rejected under same rationale.
8. Claims 22-23 and 25-27 are rejected under 35 U.S.C. 103 as being unpatentable over Cosman in view of Li, Ong as applied to claims 1,3,5-9,11,13-16 and 21 as shown above and in further view of Kamkar (Pub. No. US 20220164877 A1, Pub. Date 2022-05-26) in further view of Jiang ( NPL: Title: IMPROVING FEDERATED LEARNING ERSONALIZATION VIA MODEL AGNOSTIC META LEARNING: Pub Date 09/27/2019)
Regarding Claim 22,
As shown above, Cosman, Li and Ong teaches all the limitations of claim 3. Cosman, Li and Ong does not teach claim 22 limitation.
Kamkar teaches the computer-implemented method wherein:
the aggregator utilizes a […] framework and includes a machine learning model, a fusing component, an epsilon computation unit, and a personalization component, (see Kamkar: Fig.1C, [0082]“the model training system 110 (ML model), Fig.3A, [0078], adding the new tree sub-model to the tree ensemble S330.( fusing component) , [0081], “computing a gradient value (first-order derivative) and a Hessian value (second-order derivative) using the custom loss function S322 (an epsilon computation unit), wherein the aggregator is configured to train the global machine learning model using an XGBoost algorithm and added personalization each round of the training (see Kamkar: Fig.2, [0070], “after accessing the training input, the model training system 110 automatically trains the model to fit the training data (at S230). In some implementations, the model training system 110 automatically trains the model (at S230) by executing a training function (e.g., for the XGBoost python package, calling xgboost.train, or another such method as required by the machine learning library) that accepts as input the accessed training data rows and the optional model training parameters.”)
Jiang teaches a Model-Agnostic Meta-Learning (MAML) framework (see Jiang: Abstract : In this work, we point out that the setting of Model Agnostic Meta Learning (MAML), where one optimizes for a fast, gradient-based, few-shot adaptation to a heterogeneous distribution of tasks, has a number of similarities with the objective of personalization for FL.”) and added personalization each round (see Jiang: Section 2, Pg.2, “Algorithm 1 presents a conceptual algorithm with nested structure (left column), of which the MAMLmeta-training algorithm, Reptile (middle column), and FL-training algorithm, FedAvg (right column), are particular instances. We assume that L is a loss function common to all of the following arguments. In each iteration, a MAML algorithm trains across a random batch of tasks fTig. For each task Ti, it conducts an inner-loop update, and aggregates gradients from each sampled task with an outer-loop update.”)
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify the teaching of Cosman to include a system that teach XGBoost gradient boosted tree training as taught by Kamkar and MAML based personalization for federated learning as taught by Jiang because all references are directed to improving federated machine learning using iterative model updated from multiple client devices. One would have been motivated to make such a combination to significantly improves the model accuracy and personalization when applied across distributed client datasets.
Regarding Claim 23,
As shown above, Cosman, Li, Ong, in view of Kamkar and Jiang teaches all the limitations of claim 22. Kamkar further teaches the computer-implemented method wherein:
the XGBoost algorithm is an end-to-end system including a sparsity-aware algorithm, a weighted quantile sketch procedure, and a cache-aware block structure (see Kamkar: Fig.1c, [0091], “a solver module (e.g., 118 shown in FIG. 1C) included in the model training system 110 (e.g., a fairness-enabled XGBoost package) functions to generate the new tree model (e.g., 117 shown in FIG. 1C) by using the computed gradient and Hessian values at S323. In an example, the solver 118 is a conventional tree-boosting solver (e.g., a tree solver that performs an XGBoost process for generating regression trees, etc.).”
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify the teaching of Cosman to include a system that teach the XGBoost algorithm is an end-to-end system including a sparsity-aware algorithm, a weighted quantile sketch procedure, and a cache-aware block structure as taught by Kamkar because both the references are directed to improving federated machine learning using iterative model updated from multiple client devices. One would have been motivated to make such a combination to significantly improves the model accuracy and personalization when applied across distributed client datasets.
Regarding Claim 25,
As shown above, Cosman, Li, and Ong teaches all the limitations of claim 1.
Cosman, Li and Ong does not teach claim 25 limitation.
Kamkar further teaches the computer-implemented method wherein:
the performing of the local boosting steps by each of the parties further comprises: building recursively upon, by each of the parties, the boosted decision tree of the global machine learning model using an XGBoost algorithm (see Kamakar: Fig.3A, [0087], “the Hessian for the custom loss function (determined at S325) is the partial derivative of the gradient above. In variations P is set to 0 in the gradient equation prior to computing the Hessian and providing the Hessian to the gradient boosting learning algorithm. In examples, the custom loss function and associated gradient and Hessian are provided to the gradient boosted decision tree learning algorithm so that the gradient boosted decision tree algorithm can grow an ensemble of trees based on the provided custom loss, gradient and Hessian”), wherein a split candidate is added to the boosted decision tree at a location in which a previous boosted decision tree had largest errors or residuals (see Kamakar: Fig.3A, [0086], “the loss function for the tree ensemble is a mean squared error and this regression loss is combined similarly as above, with the adversarial classification loss.”)
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify the teaching of Cosman to include a system that teach building recursively upon, by each of the parties, the boosted decision tree of the global machine learning model using an XGBoost algorithm as taught by Kamkar because both the references are directed to improving federated machine learning using iterative model updated from multiple client devices. One would have been motivated to make such a combination to significantly improves the model accuracy and personalization when applied across distributed client datasets.
Regarding Claim 26,
As shown above, Cosman, Li, and Ong teaches all the limitations of claim 1.
Cosman, Li and Ong does not teach claim 25 limitation.
Kamkar further teaches the computer-implemented method wherein:
an aggregator utilizes is configured to transmit the global machine learning model and individualized epsilon hyperparameters to each of the plurality of parties (see Kamakar: Fig.5, [0086], “The process 500 then generates a second different tree-based machine learning model (block 508). The second tree-based machine learning model is preferably trained based on the accuracy of the first tree-based machine learning model and the fairness of the first tree-based machine learning model. The process 500 then combines the first tree-based machine learning model and the second tree-based machine learning model to produce a gradient-boosted machine learning model (block 510).”), wherein an epsilon computation component is configured to compute the individualized epsilon hyperparameters for each of the plurality of parties, wherein each of the individualized epsilon hyperparameters represents a bin size indicator (see Kamakar: Fig.5, [0125], “first tree-based machine learning model is preferably trained to predict if the financial loan will be repaid (block 502). The process 500 then determines an accuracy of the first tree-based machine learning model (block 504). The process 500 then determines a fairness of the first tree-based machine learning model (block 506). The fairness is preferably associated with at least one of gender, race, ethnicity, age, marital status, military status, sexual orientation, and disability status”
an aggregator utilizes a Model-Agnostic Meta-Learning (MAML) framework (see Jiang: Section 1, Pg.2, “Both tasks for MAML, and clients for FL, are heterogeneous. For each task in MAML and client in FL, existing algorithms use a variant of gradient descent locally, and send an overall update to a coordinator to update the global model. If we present the FL training process as meta-training in the MAML language”)
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify the teaching of Cosman to include a system that teach an aggregator utilizes is configured to transmit the global machine learning model and individualized epsilon hyperparameters to each of the plurality of parties as taught by Kamkar and MAML based personalization for federated learning as taught by Jiang because all references are directed to improving federated machine learning using iterative model updated from multiple client devices. One would have been motivated to make such a combination to significantly improves the model accuracy and personalization when applied across distributed client datasets.
Regarding Claim 27,
As shown above, Cosman, Li, Ong, in view of Kamkar and Jiang teaches all the limitations of claim 26. Li further teaches the computer-implemented method wherein the personalized model updates received from each of the parties are aggregated by a personalization component of the aggregator, and wherein the personalized model updates provided by each of the parties are fused to update the global machine learning model by a fusing component of the aggregator (see Li Fig.1, Sec. 4.2, Pg. 4645, “Since the process of building a tree is similar between different parties, we only describe the process of building a tree in Party Pm, which is shown in Algorithm 2. At first, the parties update the gradients of the local instances. Then, for each instance of Pm, the other parties compute and send the aggregated gradients of the similar instances. Instead of sending each gradient directly, such aggregation on the local party can reduce the communication cost and protect the individual gradients. After all the aggregated gradients are computed and send to Pm, the weighted gradients can be easily computed by summing(fusing) the aggregated gradients. Then, we can build a tree based on the weighted gradients.”)
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify the teaching of Cosman to include a system that personalized model updates received from each of the parties are aggregated by a personalization component of the aggregator as taught by Li. One would have been motivated to make such a combination in provide significantly improves the predictive accuracy compared with training with data in individual parties alone and significantly improve the predictive accuracy, and achieve comparable accuracy to the original GBDT with and is close to the model with joint data from all parties. (see Li: Abstract)
Response to Arguments
Claim Rejections - 35 U.S.C. § 112(b),
The original rejection to the claims as being indefinite under - 35 U.S.C. § 112(b), has been withdrawn based on applicant amendment.
Claim Rejections - 35 U.S.C. § 103,
Applicant’s arguments with respect to claim amendments have been considered but are moot considering the new combination of references being used in the current rejection. The new combination of references was necessitated by Applicant’s claim amendments. Therefore, the claims are rejected under the new combination of references as indicated above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
PGPUB
NUMBER:
INVENTOR-INFORMATION:
TITLE / DESCRIPTION
US 20210166157 A1
Bhowmick; Abhishek
Title: PRIVATE FEDERATED LEARNING WITH PROTECTION AGAINST RECONSTRUCTION
Description: Embodiments described herein relate generally to federated machine learning using distributed computing systems. More specifically, embodiments relate to a private federated learning system with protections against reconstruction attacks.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ZELALEM W SHALU whose telephone number is (571)272-3003. The examiner can normally be reached M- F 0800am- 0500pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Cesar Paula can be reached at (571) 272-4128. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Zelalem Shalu/Examiner, Art Unit 2145
/CESAR B PAULA/Supervisory Patent Examiner, Art Unit 2145
Prosecution Insights