DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This final office action is in response to the amendment filed 24 March 2026.
Claims 3-7, 10-14, and 17-28 are pending. Claims 22-25 are withdrawn as being directed toward a non-elected invention. Claims 26-28 are newly added. Claims 1, 8, and 15 are cancelled.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 3-5, 8, 10-12, 15, and 17-19 are rejected under 35 U.S.C. 103 as being unpatentable over Wenbo et al. (CN112749812, published 4 May 2021, hereafter Wenbo) and further in view of Girgis et al. (Shuffled Model of Differential Privacy in Federated Learning, April 13-15, 2021, hereafter Girgis) and further in view of Jiang et al. (US 2022/0044447, filed 7 August 2020, hereafter Jiang) and further in view of Ghazi et al. (WO 2020/257264, published 24 December 2020, hereafter Ghazi) and further in view of Shaloudegi et al. (US 2022/0237508, filed 28 January 2021, hereafter Shaloudegi) and further in view of Wang et al. (US 2022/0121999, filed 17 October 2020, hereafter Wang).
As per independent claim 26, Wenbo discloses a method of federated learning with reduced information leakage, the method comprising:
generating, by a plurality of different agents, a plurality of different trained local models based on local data and at least one model update received from at least one aggregator (page 1, paragraph 2: Here, federated learning is a model training solution including a cloud device and a plurality of terminal devices that jointly train a model in combination. The cloud device includes an aggregator that obtains training results uploaded by the plurality of end sides (client device)
mapping by a mapper, for each of the trained local models, the plurality of different parameters to a plurality of different aggregators (page 2, paragraphs 1-2: Here a model is partitioned into a plurality of aggregator nodes and training nodes. Data from the aggregation nodes is used by an aggregator to train the model (page 1, paragraph 2))
receiving, by each of the aggregators, a plurality of the partitions according to the mapping (page 6, lines 9-10: Here, the aggregated result (fused model updates) is sent to each training node to update the local training model (fused local model))
fusing, by the aggregators, each of the received partitions into an aggregated partition (page 6, paragraphs 6-9: Here, each of the terminal devices communicate data to the first aggregation node. The aggregation node fuses the data for aggregation and updating the current round of training model)
receiving from the aggregators, by each of the agents at the local level, the fused aggregated partitions (page 6, paragraphs 6-9: Here, each of the terminal devices communicate data to the first aggregation node. The aggregation node fuses the data for aggregation and updating the current round of training model)
merging, by the agents at the local level and based on the mappings, the partitions back together to create updated merged local models (page 6, paragraphs 6-9: Here, based upon completing the round of training, the training node updates the local training models)
Wenbo fails to specifically disclose:
each trained local model including a parameter vector including a plurality of different parameters
grouping, for each of the trained local models, the plurality of different parameters into a set of different partitions according to the mapping, each of the different partitions including a user-designated proportion of the plurality of parameters
shuffling at a local level by each of the agents, for each of the different partitions, the parameters for each partition to change an order of the parameters according to a same permutation algorithm that is shared across the agents
reverse shuffling, by the agents at the local level based on the same permutation algorithm, the fused aggregated partitions
However, Girgis, which is analogous to the claimed invention because it is directed toward maintaining security in a federated learning system, discloses shuffling at a local level by each of the agents, for each of the different partitions, the parameters for each partition to change an order of the parameters according to a same permutation algorithm that is shared across the agents (Section 3: Here, a shuffler is used to randomly shuffle permutations). It would have been obvious to one of ordinary skill in the art at the time of the applicant’s effective filing date to have combined Girgis with Wenbo, with a reasonable expectation of success, as it would have allowed for maintaining privacy of shuffled items within a federated learning framework (Girgis: Section 3).
Further, Jiang, which is analogous to the claimed invention because it is directed toward reverse-shuffling based upon mapping, discloses reverse shuffling, by the agents at the local level based on the same permutation algorithm, the fused aggregated partitions (Figure 1D; paragraph 0032: here, the remapping/inverse process of the first mapping process is used to return elements to their original positions). It would have been obvious to one of ordinary skill in the art at the time of the applicant’s effective filing date to have combined Jiang with Wenbo-Girgis, with a reasonable expectation of success, as it would have allowed for reconstructing the original data. This would have provided the user with the advantage of processing the original data for training the model.
Ghazi, which is analogous to the claimed invention because it is directed toward private distributed aggregation of data, discloses grouping, the plurality of different parameters into a set of different partitions (Figures 2 and 4; paragraphs 0051 and 0055-0060: Here, a plurality of local models are passed to the shuffler. Federated learning is applied to aggregate the data from each of the local models at the shuffler). It would have been obvious to one of ordinary skill in the art at the time of the applicant’s effective filing date to have combined Ghazi with Wenbo-Girgis-Jiang, with a reasonable expectation of success, as it would have allowed for separating the models to improve privacy within the aggregate data (Ghazi: paragraph 0059).
Additionally, Shaloudegi, which is analogous to the claimed invention because it is directed toward federated learning, discloses each trained local model including a parameter vector including a plurality of different parameters (paragraph 0026: Here, a local Hessian-vector product based upon the Hessian matrix of the local model and the parameter vector). It would have been obvious to one of ordinary skill in the art at the time of the applicant’s effective filing date to have combined Shaloudegi with Wenbo-Girgis-Jiang-Ghazi, with a reasonable expectation of success, as it would have allowed for integrating local models with the global federated learning model (Shaloudegi: paragraph 0026).
Further, Wang, which is analogous to the claimed invention because it is directed toward federated learning, discloses each of the different partitions including a user-designated proportion of the plurality of parameters (claim 1: here, a Hessian matrix specifies a user-defined objective of the local models used for federated learning). It would have been obvious to one of ordinary skill in the art at the time of the applicant’s effective filing date to have combined Wang with Wenbo-Girgis-Jiang-Ghazi- Shaloudegi, with a reasonable expectation of success, as it would have allowed a user to specify objectives of the federated learning (Wang: claim 1).
As per dependent claim 3, Wenbo, Girgis, Jiang, Ghazi, Shaloudegi, and Wang disclose the limitations similar to those in claim 26, and the same rejection is incorporated herein. Wenbo discloses repeating the operation dynamically (page 6, paragraph 11: Here, the aggregation and updating a model is performed dynamically upon receipt of new data).
Wenbo fails to specifically disclose applying a permutation operation. However, Girgis, which is analogous to the claimed invention because it is directed toward maintaining security in a federated learning system, discloses applying a permutation operation to one or more elements within each of the partitions to generate shuffled partitions (Section 3: Here, a shuffler is used to randomly shuffle permutations). It would have been obvious to one of ordinary skill in the art at the time of the applicant’s effective filing date to have combined Girgis with Wenbo, with a reasonable expectation of success, as it would have allowed for maintaining privacy of shuffled items within a federated learning framework (Girgis: Section 3).
As per dependent claim 4, Wenbo, Girgis, Jiang, Ghazi, Shaloudegi, and Wang disclose the limitations similar to those in claim 3, and the same rejection is incorporated herein. Wenbo discloses wherein the operation is repeated at every training iteration (page 6, paragraph 11: Here, the training iteration is performed each time aggregated data is added).
Wenbo fails to specifically disclose applying a permutation algorithm operation to one or more elements within each of the partitions to generate shuffled partitions. However, Girgis, which is analogous to the claimed invention because it is directed toward maintaining security in a federated learning system, discloses applying a permutation algorithm operation to one or more elements within each of the partitions to generate shuffled partitions (Section 3: Here, a shuffler is used to randomly shuffle permutations). It would have been obvious to one of ordinary skill in the art at the time of the applicant’s effective filing date to have combined Girgis with Wenbo, with a reasonable expectation of success, as it would have allowed for maintaining privacy of shuffled items within a federated learning framework (Girgis: Section 3).
As per dependent claim 5, Wenbo, Girgis, Jiang, Ghazi, Shaloudegi, and Wang disclose the limitations similar to those in claim 26, and the same rejection is incorporated herein. Wenbo discloses wherein the permutation algorithm operation is based on a secret mutually-agreed by the participating agents, and wherein the mapper is shared by all the participating agents (page 1, paragraph 2: Here, the secret sharing manner is used to encrypt data where the cloud device and terminals receive the public key for decrypting data).
With respect to claim 27, the applicant discloses the limitations substantially similar to those in claim 26, and the rejection is incorporated herein.
Additionally, Wenbo discloses a hardware processor (page 12, paragraph 1) and computer memory holding computer program instructions executed by the hardware (page 12, paragraph 3) to provide federated learning with reduced information leakage, wherein model updates provided by participating parties are fused (page 1, paragraphs 2-3).
With respect to claims 10-12, the applicant discloses the limitations substantially similar to those in claims 3-5, respectively. Claims 10-12 are similarly rejected.
With respect to claim 28, the applicant discloses the limitations substantially similar to those in claim 26, and the rejection is incorporated herein.
Additionally, Wenbo discloses a computer program product in a non-transitory computer readable medium for use in a data processing system (page 12, paragraph 3) to provide federated learning with reduced information leakage, wherein model updates provided by participating parties are fused (page 1, paragraphs 2-3).
With respect to claims 17-19, the applicant discloses the limitations substantially similar to those in claims 3-5, respectively. Claims 17-19 are similarly rejected.
Claims 6-7, 13-14, and 20-21 are rejected under 35 U.S.C. 103 as being unpatentable over Wenbo, Girgis, Jiang, Ghazi, Shaloudegi, and Wang and further in view of Yu et al. (US 2022/0269977, filed 22 February 2021, hereafter Yu).
As per dependent claim 6, Wenbo, Girgis, Jiang, Ghazi, Shaloudegi, and Wang disclose the limitations similar to those in claim 26, and the same rejection is incorporated herein. Wenbo fails to specifically disclose wherein the elements are one of model parameters and model gradients.
However, Yu, which is analogous to the claimed invention because it is directed toward training a federated machine learning model, discloses wherein the elements are one of model parameters and model gradients (Figure 3; paragraphs 0073-0074 : Here, model parameters are updated based upon receiving updates from participants). It would have been obvious to one of ordinary skill in the art at the time of the applicant’s effective filing date to have combined Yu with Wenbo-Girgis-Jiang-Ghazi-Shaloudegi-Wang, with a reasonable expectation of success, as it would have allowed for identifying and filtering potentially malicious edits before propagating the changes to all local models (Yu: paragraph 0074).
As per dependent claim 7, Wenbo, Girgis, Jiang, Ghazi, Shaloudegi, and Wang disclose the limitations similar to those in claim 26, and the same rejection is incorporated herein. Wenbo fails to specifically disclose wherein applying the operation adjusts just a single model parameter. However, Yu, which is analogous to the claimed invention because it is directed toward training a federated machine learning model, discloses wherein the elements are one of model parameters and model gradients (Figure 3; paragraphs 0073-0074 : Here, model parameters are updated based upon receiving updates from participants). It would have been obvious to one of ordinary skill in the art at the time of the applicant’s effective filing date to have combined Yu with Wenbo-Girgis, with a reasonable expectation of success, as it would have allowed for identifying and filtering potentially malicious edits before propagating the changes to all local models (Yu: paragraph 0074).
Further, the examiner takes official notice that it was notoriously well-known in the art at the time of the applicant’s effective filing date that an update may include just a single parameter. It would have been obvious to one of ordinary skill in the art at the time of the applicant’s effective filing date to have combined the well-known with Wenbo-Girgis-Yu, with a reasonable expectation of success, as it would have allowed for updating a single parameter. This would have allowed for creating focused updates that allow for testing the effectiveness of modifying the single parameter. This would have allowed a user to specifically identify whether the updated parameter leads to improvement within the model.
With respect to claims 13-14, the applicant discloses the limitations substantially similar to those in claims 6-7, respectively. Claims 13-14 are similarly rejected.
With respect to claims 20-21, the applicant discloses the limitations substantially similar to those in claims 6-7, respectively. Claims 20-21 are similarly rejected.
Response to Arguments
Applicant’s arguments have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Wenbo, Girgis, Jiang, Ghazi, Shaloudegi, and Wang.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Choudhury et al. (US 11188791): Discloses training a global federated learning model using an aggregator (Abstract)
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KYLE R STORK whose telephone number is (571)272-4130. The examiner can normally be reached 8am - 2pm; 4pm - 6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Omar Fernandez Rivas can be reached at 571/272-2589. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KYLE R STORK/Primary Examiner, Art Unit 2128
Prosecution Insights