DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is in response to the Application filed on 01/21/2026. Claims 1, 3, 5-9, 11, 13-16 and 21-27 are pending in the case. This action is Final.
Claim Rejections - 35 USC § 112
3. The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
4. Claim 1, 9 and 21 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1:
Claim 1 states “… with the global machine learning model from the personalized model updates acting as an initial model transmitted to the plurality of parties for a next iteration, until a predetermined stopping criterion is achieved” is unclear because the claim appears to state that the “global machine learning model” is generated from the personalized model updates, included within the personalized model updates, within the personalized model updates, or itself includes one of the personalized model updates. In addition the phrase “acting as an initial model “did not clarify which model is transmitted for the next iteration. Accordingly, the meets and bounds of the claim can not be reasonably determined by a person of ordinary skill in the art.
Independent Claim 9 and 21 are also rejected 35 USC § 112(b) for the same reason stated above.
Examiner Comments
5. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claim Rejections - 35 USC § 103
6. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
7. Claims 1, 3, 5-9, 11, 13-16 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Cosman (Pub. No. US 20210192078 B1, Pub. Date 2021-06-24) in view of Li (NPL: Title: Practical Federated Gradient Boosting Decision Trees; Dae: 11-2019) in further view of Sidahmed ( Pub. No. US 20230214642 A1, Pub. Date 2023-07-06)
Cosman teaches a computer-implemented method comprising:
training a global machine learning model using federated learning between a plurality of parties (see Cosman: Fig.4A, [0055], “Method 400 can additionally include to privatize the aggregate model updates on the learning server using central differential privacy (block 407). The privatized model updates can then be used to update the server machine learning model (block 408) (i.e. The server machine learning model (global machine learning model) is trained and updated by the privatized model data sent/ transmitted form plurality of parties (client devices) using federated learning as shown in Fig.3A);
distributing the global machine learning model to each of the parties (see Cosman: Fig.4B, [0057], “a machine learning model M0 131 may be received by client device 310 from a server 130 (411). The machine learning model M0 131 may be a previously trained model with privatized federated learning (e.g. a previously updated machine learning model M1 235) or an initial machine learning model.”), […]
receiving personalized model updates from each of the parties (see Cosman: Fig.4A, [0055], “The individual model updates can then be privatized on the set of client devices using separated differential privacy (block 404). The privatized model updates are then sent from the set of client devices to a central learning server (block 405).”), wherein the personalized model updates are generated from updated models boosted locally and produced by each of the parties using their respective local data (see Cosman: Fig.5A, [0071], method 510 includes an operation (block 511) to obtain a weight vector that represents a difference between a previous and recently trained model. This difference represents the model update that is to be transmitted to the learning server to update the current server model. Method 510 additionally includes an operation (block 512) to decompose the weight vector into a unit vector (e.g., unit direction U=W/∥W∥.sub.2) and a magnitude (e.g., radius R=∥W∥.sub.2). Method 500 additionally includes an operation (block 513) to privatize the unit vector. The unit vector can be privatized using any desired mechanism.).”)
fusing the personalized model updates to produce […] update the global machine learning model (see Cosman: Fig.4A, [0055], “The privatized model updates can then be used to update the server machine learning model (block 408). Additional details are provided below for the operations of method 400.”)
As shown above, Cosman teaches private federated learning, to create or update a machine learning model to predict user behavior with items accessible with a device. In some embodiments, the resulting updated machine learning model predicts a desired usage pattern with a content item and/or an application accessible on a user device.
Cosman does not teach the computer-implemented method of comprising:
parties performing local boosting steps by computing gradients, and hessian statistics relating to the global machine learning model and their private datasets;
fusing to produce boosted decision tree;
rebuilding the global machine learning model using the boosted decision tree;
starting with the updated global machine learning model, retraining the global machine learning model in an iterative manner, with the global machine learning model from the personalized model updates acting as an initial model transmitted to the plurality of parties for a next iteration, until a predetermined stopping criterion is achieved.
However, Li teaches the computer-implemented method comprising:
the parties performing local boosting steps by computing gradients, and hessian statistics relating to the global machine learning model and their private datasets (see Li: Fig.1, Pg. 4643, describing first order gradients g and second order Hessians h for Gradient Boosting Decision Trees and use g and h to build trees as shown below.
PNG
media_image1.png
334
340
media_image1.png
Greyscale
fusing to produce boosted decision (see Li Fig.1, Sec. 4.2, Pg. 4645, “Since the process of building a tree is similar between different parties, we only describe the process of building a tree in Party Pm, which is shown in Algorithm 2. At first, the parties update the gradients of the local instances. Then, for each instance of Pm, the other parties compute and send the aggregated gradients of the similar instances. Instead of sending each gradient directly, such aggregation on the local party can reduce the communication cost and protect the individual gradients. After all the aggregated gradients are computed and send to Pm, the weighted gradients can be easily computed by summing(fusing) the aggregated gradients. Then, we can build a tree based on the weighted gradients.”)
rebuilding the global machine learning model using the boosted decision tree (see Li Fig.1, Sec. 4.2, Pg. 4645, “Since the process of building a tree is similar between different parties, we only describe the process of building a tree in Party Pm, which is shown in Algorithm 2. At first, the parties update the gradients of the local instances. Then, for each instance of Pm, the other parties compute and send the aggregated gradients of the similar instances. Instead of sending each gradient directly, such aggregation on the local party can reduce the communication cost and protect the individual gradients.”)
Because both Cosman and Li are in the same/similar field of endeavor of selecting machine learning training data, accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify the teaching of Cosman to include a system that fuse to produce boosted decision tree and rebuilding the global machine learning model using the boosted decision tree as taught by Li. One would have been motivated to make such a combination in provide significantly improves the predictive accuracy compared with training with data in individual parties alone and significantly improve the predictive accuracy, and achieve comparable accuracy to the original GBDT with and is close to the model with joint data from all parties. (see Li: Abstract)
Cosman and Li does not teach the computer-implemented method of comprising:
starting with the updated global machine learning model, retraining the global machine learning model in an iterative manner, with the global machine learning model from the personalized model updates acting as an initial model transmitted to the plurality of parties for a next iteration, until a predetermined stopping criterion is achieved.
However, Sidahmed teaches the computer-implemented method comprising:
starting with the updated global machine learning model (see Sidahmed: Fig.4, [0133], “At operation 412, the method can include modifying one or more global parameters of the global model based on the aggregation of the updates to the one or more parameters that are respectively received from the plurality of client computing devices.”), retraining the global machine learning model in an iterative manner (see Sidahmed: Fig.3, [0116], “Any number of iterations of local and global updates can be performed. That is, method (300) can be performed iteratively to update the global model based on locally stored training data over time.” See also Fig.5, [0141]), “When the performance value does exceed the threshold value, the operation 504 continues to operation 506 to train more parameters in order to improve the accuracy of the model. When the performance value does exceed the threshold value, then less parameters of the global model may be frozen in the next iteration in order to improve the performance value (e.g., improve accuracy percentage of the global model).”), with the global machine learning model from the personalized model updates acting as an initial model transmitted to the plurality of parties for a next iteration (see Sidahmed: Fig.1, [0151] “determining a new set of training parameters from the plurality of parameters of the global model. The new set of training parameters can have less parameters than the first set of training parameters. In some instances, method 600 can include determining additional parameters from the first set of parameters to freeze. For example, an updated set of frozen parameters can include the additional parameters from the first set of parameters that have been determined to be frozen.”… [0152], “transmitting, respectively to the plurality of client computing devices, the new set of training parameters and a new random seed. The new random seed can be generated from the updated set of frozen parameters by the random seed generator.”), until a predetermined stopping criterion is achieved (see Sidahmed: Fig.5, [0141], “determining whether the performance value exceeds a threshold value. In some instances, the performance value exceeds the threshold value when an accuracy percentage of the global model is reduced by a specific margin after the modification of the one or more global parameters of the global model, which may result in performance degradation.”)
Because both Cosman and Sidahmed are in the same/similar field of endeavor of federated leering system, accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify the teaching of Cosman to include the iterative federated learning system that distribute global learning model to personal models to update the global model and personal model until a certain criterion is achieved as taught by Sidahmed. One would have been motivated to make such a combination in improve training efficiency, model accuracy and model by improving the efficiency of federated learning systems is needed for improved scalability and usability.( see Sidahmed [0003])
Regarding Claim 3,
As shown above, Cosman, Li and Sidahmed and teaches all the limitations of claim 2. Cosman further teaches the system wherein training the global machine learning model comprises:
establishing a connection between an aggregator and the plurality of parties (see Cosman: Fig.1, [0040], “system 100 includes a server 130 that can receive data from a set of client devices 110a-110n, 111a-111n, 112a-112n over a network 120.”)
computing, by the aggregator, party-centric epsilon parameters for each party of the plurality of parties based on a global epsilon parameter (see Cosman: Fig.1, [0037], “Using separated differential privacy, a private federated learning system can be enabled that provides comparable utility to a federated learning system that does not provide privacy safeguards. Privacy is enabled by obfuscating the individual updates to the server. In one embodiment a relaxed privacy parameter ε is used, user data is still protected against reconstruction by individuals (e.g., internal employees) that may have access to privatized updates. In one embodiment, fully ε-differentially-private techniques (a global epsilon parameter) are used to enable privatization of the magnitude. In another embodiment, relative noise mechanisms are used to privatize the magnitude.”);
transmitting a party-centric epsilon parameter to each of the plurality of parties (see Cosman: Fig.1, [0038], “The use of central differential privacy provides additional protection for updated learning models on the server against external adversaries that may have access to the model and any other information except the user data that the adversary wishes to decode.”)
receiving, from the parties, histogram data distributions representing raw training data maintained by the parties (see Cosman: Fig.4A, [0055], (see Cosman: Fig.2, [0048], “generate privatized model updates 212a-212c (e.g., privatized model update 212a from client device 210a, privatized model update 212b from client device 210b, privatized model update 212c from client device 210c), which can be transmitted to the server 130 via the network 120. The privatized model updates 212a-212c can be stripped of their IP addresses or other information that can be used to identify the client devices 210 prior to entering an ingestor 232 on the server 130.”)
generating a global machine learning model based on the histogram data distributions (see Cosman: Fig.2, [0048], “The ingestor 232 can collect the data from the client devices 210 and remove metadata and forwards the data to an aggregator 233. The aggregator takes the privatized model updates and aggregates them to form a single update to the current server model, which in the initial round is machine learning model 131 (e.g., model M0).)”;
transmitting the global machine learning model to the parties (see Cosman: Fig.4B, [0057], “a machine learning model M0 131 may be received by client device 310 from a server 130 (411). The machine learning model M0 131 may be a previously trained model with privatized federated learning (e.g. a previously updated machine learning model M1 235) or an initial machine learning model.”);
fusing the model updates to produce a global histogram to update the global machine learning model (see Cosman: Fig.2, [0048], “The privatized model updates 212a-212c can be stripped of their IP addresses or other information that can be used to identify the client devices 210 prior to entering an ingestor 232 on the server 130. The ingestor 232 can collect the data from the client devices 210 and remove metadata and forwards the data to an aggregator 233. The aggregator takes the privatized model updates and aggregates them to form a single update to the current server model, which in the initial round is machine learning model 131 (e.g., model M0).”
receiving model updates from each of the parties, wherein each of the model updates includes a histogram, a gradient, and the hessian statistics based on applying the global machine learning model to training data stored by the parties’ datasets (see Li: Fig.1, Pg. 4643, describing first order gradients g and second order Hessians h for Gradient Boosting Decision Trees and use g and h to build trees as shown above.)
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify the teaching of Cosman to include a system that receive model updates includes a histogram, a gradient, and hessian statistics based on applying the global machine learning model to training data stored by the parties’ datasets as taught by Li. One would have been motivated to make such a combination in provide significantly improves the predictive accuracy compared with training with data in individual parties alone and significantly improve the predictive accuracy, and achieve comparable accuracy to the original GBDT with and is close to the model with joint data from all parties. (see Li: Abstract)
Regarding Claim 5,
As shown above, Cosman and Li and teaches all the limitations of claim 1. Cosman further teaches the system wherein:
the party-centric epsilon parameter is specific to each party of the plurality of parties based on sizes of the training data provided by each party (see Cosman: Fig.4A, [0083], “a decoder may reconstruct the sequence of layers from the last hidden state representation. Upon comparison of the reconstructed layer surface data with a layer surface data of the layer being printed, the encoder-decoder based model may identify a deviation between the reconstructed layer surface data and the layer surface data of the layer being printed. Such a deviation is indicated as a predicted anomaly score.”);
Regarding Claim 6,
As shown above, Cosman, Li and Sidahmed and teaches all the limitations of claim 1. Cosman further teaches the system wherein:
the personalized model updates are produced using gradient descent on a per-party loss (see Cosman: Fig.16, [0152], “act 1602 can include accessing, by a device (e.g., 114) operatively coupled to a processor, a first set of data candidates (e.g., 106) and a second set of data candidates (e.g., 108), wherein a machine learning model (e.g., 104) is trained on the first set of data candidates.”)
Regarding Claim 7,
As shown above, Cosman, Li and Sidahmed and teaches all the limitations of claim 1. Cosman further teaches the system wherein:
model-agnostic meta-learning is used to train the machine learning model (see Cosman: Fig.3, [0050], “The receive module 351 can remove latent identifiers such as IP addresses or other data that might identify the client device 310. The ingestor/aggregator can include components of the ingestor 232 and aggregator 233 shown in FIG. 2 and can perform similar operations, such as removing metadata, session identifiers, and other identifying information, and aggregating the privatized information to generate an aggregated model update 331.’)
Regarding Claim 8,
As shown above, Cosman, Li and Sidahmed and teaches all the limitations of claim 1. Cosman further teaches the system wherein:
the updated global machine learning model mimics trees obtained from local boosting by the parties using their local distribution to add personalization (see Li: Fig.1, Sec.4, Pg. 4644, “each party can use the global hash tables for tree building without accessing
other party’s raw data. In the training stage, all parties together train a number of trees one by one using the similarity information. Once a tree is built in a party, it will be sent to all the other parties for the update of gradients.”)
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify the teaching of Cosman to include a system that updated global machine learning model mimics trees obtained from local boosting by the parties using their local distribution to add personalization as taught by Li. One would have been motivated to make such a combination in provide smaller, more efficient training datasets and a lag-free and efficient machine learning model training system.
Regarding independent claim 9
Claim 9 is directed to a computer implemented method claim and has similar/same claim limitation as Claim 1 and is rejected under same rationale.
Regarding Claim 11, and 13-16,
Claims 11 and 13-16 are directed to a computer implemented method claim and have similar/same claim limitation as Claims 2-8, respectively and are rejected under same rationale.
Regarding independent Claim 21,
Claim 21 is directed to a system claim and has similar/same claim limitation as Claim 1 and is rejected under same rationale.
8. Claims 22-27 are rejected under 35 U.S.C. 103 as being unpatentable over Cosman in view of Li, Sidahmed as applied to claims 1,3,5-9,11,13-16 and 21 as shown above and in further view of Kamkar (Pub. No. US 20230214642 A1, Pub. Date 2022-05-26) in further view of Jiang ( NPL: Title: IMPROVING FEDERATED LEARNING ERSONALIZATION VIA MODEL AGNOSTIC META LEARNING Pub Date 09/27/2019)
Regarding Claim 22,
As shown above, Cosman, Li and Sidahmed and teaches all the limitations of claim 3. Cosman, Li and Sidahmed does not teach claim 22 limitation.
Kamkar teaches the computer-implemented method wherein:
the aggregator utilizes a […] framework and includes a machine learning model, a fusing component, an epsilon computation unit, and a personalization component, (see Kamkar: Fig.1C, [0082]“the model training system 110 (ML model), Fig.3A, [0078], adding the new tree sub-model to the tree ensemble S330.( fusing component) , [0081], “computing a gradient value (first-order derivative) and a Hessian value (second-order derivative) using the custom loss function S322 (an epsilon computation unit), wherein the aggregator is configured to train the global machine learning model using an XGBoost algorithm and added personalization each round of the training (see Kamkar: Fig.2, [0070], “after accessing the training input, the model training system 110 automatically trains the model to fit the training data (at S230). In some implementations, the model training system 110 automatically trains the model (at S230) by executing a training function (e.g., for the XGBoost python package, calling xgboost.train, or another such method as required by the machine learning library) that accepts as input the accessed training data rows and the optional model training parameters.”)
Jiang teaches a Model-Agnostic Meta-Learning (MAML) framework (see Jiang: Abstract : In this work, we point out that the setting of Model Agnostic Meta Learning (MAML), where one optimizes for a fast, gradient-based, few-shot adaptation to a heterogeneous distribution of tasks, has a number of similarities with the objective of personalization for FL.”) and added personalization each round (see Jiang: Section 2, Pg.2, “Algorithm 1 presents a conceptual algorithm with nested structure (left column), of which the MAMLmeta-training algorithm, Reptile (middle column), and FL-training algorithm, FedAvg (right column), are particular instances. We assume that L is a loss function common to all of the following arguments. In each iteration, a MAML algorithm trains across a random batch of tasks fTig. For each task Ti, it conducts an inner-loop update, and aggregates gradients from each sampled task with an outer-loop update.”)
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify the teaching of Cosman to include a system that teach XGBoost gradient boosted tree training as taught by Kamkar and MAML based personalization for federated learning as taught by Jiang because all references are directed to improving federated machine learning using iterative model updated from multiple client devices. One would have been motivated to make such a combination to significantly improves the model accuracy and personalization when applied across distributed client datasets.
Regarding Claim 23,
As shown above, Cosman, Li, Sidahmed in view of Kamkar and Jiang and teaches all the limitations of claim 22. Kamkar further teaches the computer-implemented method wherein:
the XGBoost algorithm is an end-to-end system including a sparsity-aware algorithm, a weighted quantile sketch procedure, and a cache-aware block structure (see Kamkar: Fig.1c, [0091], “a solver module (e.g., 118 shown in FIG. 1C) included in the model training system 110 (e.g., a fairness-enabled XGBoost package) functions to generate the new tree model (e.g., 117 shown in FIG. 1C) by using the computed gradient and Hessian values at S323. In an example, the solver 118 is a conventional tree-boosting solver (e.g., a tree solver that performs an XGBoost process for generating regression trees, etc.).”
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify the teaching of Cosman to include a system that teach the XGBoost algorithm is an end-to-end system including a sparsity-aware algorithm, a weighted quantile sketch procedure, and a cache-aware block structure as taught by Kamkar because both the references are directed to improving federated machine learning using iterative model updated from multiple client devices. One would have been motivated to make such a combination to significantly improves the model accuracy and personalization when applied across distributed client datasets.
Regarding Claim 24
As shown above, Cosman, Li, and Sidahmed teaches all the limitations of claim 1.
Cosman, Li and Sidahmed does not teach claim 24 limitation.
Kamkar further teaches the computer-implemented method wherein:
an aggregator utilizes an XGBoost algorithm for the retraining of the global machine learning model in the iterative manner, and wherein the XGBoost algorithm is utilized to perform a federated quantile sketch method for the fusing of the model updates to produce a global histogram (see Kamkar: Fig.1C, [0077], “the model is a tree ensemble, and executing the training function (e.g., “xgboost.train( )”) at S230 includes performing a tree boosting training process (e.g., a gradient tree boosting training process) that includes iteratively adding tree sub-models to a tree ensemble until output generated by the tree ensemble for each of a plurality of training data rows satisfies training stopping criteria.”)
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify the teaching of Cosman to include a system that teach an aggregator utilizes an XGBoost algorithm for the retraining of the global machine learning model in the iterative manner as taught by Kamkar because both the references are directed to improving federated machine learning using iterative model updated from multiple client devices. One would have been motivated to make such a combination to significantly improves the model accuracy and personalization when applied across distributed client datasets.
Regarding Claim 25,
As shown above, Cosman, Li, and Sidahmed teaches all the limitations of claim 1.
Cosman, Li and Sidahmed does not teach claim 25 limitation.
Kamkar further teaches the computer-implemented method wherein:
the performing of the local boosting steps by each of the parties further comprises: building recursively upon, by each of the parties, the boosted decision tree of the global machine learning model using an XGBoost algorithm (see Kamakar: Fig.3A, [0087], “the Hessian for the custom loss function (determined at S325) is the partial derivative of the gradient above. In variations P is set to 0 in the gradient equation prior to computing the Hessian and providing the Hessian to the gradient boosting learning algorithm. In examples, the custom loss function and associated gradient and Hessian are provided to the gradient boosted decision tree learning algorithm so that the gradient boosted decision tree algorithm can grow an ensemble of trees based on the provided custom loss, gradient and Hessian”), wherein a split candidate is added to the boosted decision tree at a location in which a previous boosted decision tree had largest errors or residuals (see Kamakar: Fig.3A, [0086], “the loss function for the tree ensemble is a mean squared error and this regression loss is combined similarly as above, with the adversarial classification loss.”)
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify the teaching of Cosman to include a system that teach building recursively upon, by each of the parties, the boosted decision tree of the global machine learning model using an XGBoost algorithm as taught by Kamkar because both the references are directed to improving federated machine learning using iterative model updated from multiple client devices. One would have been motivated to make such a combination to significantly improves the model accuracy and personalization when applied across distributed client datasets.
Regarding Claim 26,
As shown above, Cosman, Li, and Sidahmed teaches all the limitations of claim 1.
Cosman, Li and Sidahmed does not teach claim 25 limitation.
Kamkar further teaches the computer-implemented method wherein:
an aggregator utilizes is configured to transmit the global machine learning model and individualized epsilon hyperparameters to each of the plurality of parties (see Kamakar: Fig.5, [0086], “The process 500 then generates a second different tree-based machine learning model (block 508). The second tree-based machine learning model is preferably trained based on the accuracy of the first tree-based machine learning model and the fairness of the first tree-based machine learning model. The process 500 then combines the first tree-based machine learning model and the second tree-based machine learning model to produce a gradient-boosted machine learning model (block 510).”), wherein an epsilon computation component is configured to compute the individualized epsilon hyperparameters for each of the plurality of parties, wherein each of the individualized epsilon hyperparameters represents a bin size indicator (see Kamakar: Fig.5, [0125], “first tree-based machine learning model is preferably trained to predict if the financial loan will be repaid (block 502). The process 500 then determines an accuracy of the first tree-based machine learning model (block 504). The process 500 then determines a fairness of the first tree-based machine learning model (block 506). The fairness is preferably associated with at least one of gender, race, ethnicity, age, marital status, military status, sexual orientation, and disability status”
an aggregator utilizes a Model-Agnostic Meta-Learning (MAML) framework (see Jiang: Section 1, Pg.2, “Both tasks for MAML, and clients for FL, are heterogeneous. For each task in MAML and client in FL, existing algorithms use a variant of gradient descent locally, and send an overall update to a coordinator to update the global model. If we present the FL training process as meta-training in the MAML language”)
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify the teaching of Cosman to include a system that teach an aggregator utilizes is configured to transmit the global machine learning model and individualized epsilon hyperparameters to each of the plurality of parties as taught by Kamkar and MAML based personalization for federated learning as taught by Jiang because all references are directed to improving federated machine learning using iterative model updated from multiple client devices. One would have been motivated to make such a combination to significantly improves the model accuracy and personalization when applied across distributed client datasets.
Regarding Claim 27,
As shown above, Cosman, Li, Sidahmed in view of Kamkar and Jiang and teaches all the limitations of claim 26. Li further teaches the computer-implemented method wherein the personalized model updates received from each of the parties are aggregated by a personalization component of the aggregator, and wherein the personalized model updates provided by each of the parties are fused to update the global machine learning model by a fusing component of the aggregator (see Li Fig.1, Sec. 4.2, Pg. 4645, “Since the process of building a tree is similar between different parties, we only describe the process of building a tree in Party Pm, which is shown in Algorithm 2. At first, the parties update the gradients of the local instances. Then, for each instance of Pm, the other parties compute and send the aggregated gradients of the similar instances. Instead of sending each gradient directly, such aggregation on the local party can reduce the communication cost and protect the individual gradients. After all the aggregated gradients are computed and send to Pm, the weighted gradients can be easily computed by summing(fusing) the aggregated gradients. Then, we can build a tree based on the weighted gradients.”)
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify the teaching of Cosman to include a system that personalized model updates received from each of the parties are aggregated by a personalization component of the aggregator as taught by Li. One would have been motivated to make such a combination in provide significantly improves the predictive accuracy compared with training with data in individual parties alone and significantly improve the predictive accuracy, and achieve comparable accuracy to the original GBDT with and is close to the model with joint data from all parties. (see Li: Abstract)
Response to Arguments
Claim Rejections - 35 U.S.C. § 101,
Regarding the 35 U.S.C. 101 rejection for being directed non-statutory subject matter has been updated and withdrawn based on applicant amendments and. Therefore, the 35 U.S.C. 101 rejection has been withdrawn.
Claim Rejections - 35 U.S.C. § 112(b),
The original rejection to the claims as being indefinite under - 35 U.S.C. § 112(b), has been withdrawn based on applicant amendment. However, an new - 35 U.S.C. § 112(b) rejection has been applied to the update office action based on the applicant amendment.
Claim Rejections - 35 U.S.C. § 103,
Applicant’s arguments with respect to claim amendments have been considered but are moot considering the new combination of references being used in the current rejection. The new combination of references was necessitated by Applicant’s claim amendments. Therefore, the claims are rejected under the new combination of references as indicated above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
PGPUB
NUMBER:
INVENTOR-INFORMATION:
TITLE / DESCRIPTION
US 20220391781 A1
LIU; Jian
Title: Architecture-agnostic federated learning system
Description: As the foregoing illustrates, there is a need to develop a technology that supports federated learning between clients with different architectures that implement different local model architectures.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ZELALEM W SHALU whose telephone number is (571)272-3003. The examiner can normally be reached M- F 0800am- 0500pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Cesar Paula can be reached at (571) 272-4128. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Zelalem Shalu/Examiner, Art Unit 2145
/CESAR B PAULA/Supervisory Patent Examiner, Art Unit 2145
Prosecution Insights