DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claim 7,
Step 1: Is the claim to a process, machine, manufacture, or composition of matter?
Yes, the claim is directed to a method.
Step 2A Prong One: Does the claim recite an abstract idea, law of nature, or natural phenomenon?
The limitations of:
clustering, [by a system operatively coupled to a processor], one or more participants in a federated learning system based on distributions of data classification labels for data sets of the one or more participants into one or more clusters of participants; (mental evaluation, a human can look at data and then separate it or cluster it based on any arbitrary criteria)
selecting, [by the system, participants equitably from across the one or more clusters of participants for a round of federated learning.
Step 2A Prong Two: Does the claim recite additional elements that integrate the judicial exception into a practical application?
The limitations of:
[…] by a system operatively coupled to a processor […] (generic computer components to carry out the abstract idea, MPEP 2106.05(f))
Step 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception?
The limitations of:
[…] by a system operatively coupled to a processor […] (generic computer components to carry out the abstract idea, MPEP 2106.05(f))
Dependent claim 8 recites establishing communication channels, applying the abstract idea to a particular technological environment, MPEP 2106.05(h).
Dependent claim 9 recites receiving labels for the data, insignificant extra-solution activity, MPEP 2106.05(g), transmitting data is well-understood, routine, and conventional in the art, MPEP 2106.05(d)(II)(i).
Dependent claim 10 recites clustering within the trusted environment, applying the abstract idea to a particular technological environment, MPEP 2106.05(h).
Dependent claim 11 recites storing the data in a trusted environment, insignificant extra-solution activity MPEP 2106.05(g), storing data is well-understood, routine, and conventional in the art, MPEP 2106.05(d)(II)(iv).
Dependent claim 12 recites selecting underrepresented participants, mental evaluation.
Dependent claim 13 recites k-means clustering, applying the abstract idea to a particular field of use, MEPP 2106.05(h).
Independent claims 1 and 14 recite the same substantial subject matter as independent claim 7, only differing in embodiment. The difference in embodiments, a system and computer-readable medium does not meaningfully change the above analysis and therefore the claims are subject to the same rejection.
Dependent claims 15-20 correspond to dependent claims 8-13. Dependent claims 2-6 correspond to dependent claims 8-12.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claim(s) 1-5, 7-11, 13-18, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ghosh, Avishek, et al. "Robust federated learning in a heterogeneous environment." in view of Soltani, Behnaz, et al. "A survey on participant selection for federated learning in mobile networks.".
Regarding claims 1, 7, and 14, Ghosh teaches “a system comprising: a memory that stores computer executable components; and a processor, operatively coupled to the memory, that executes the computer executable components stored in the memory, wherein the computer executable components comprise” (abstract “Federated Learning, where the worker machines are end users’ own devices.” own devices i.e. computer, mobile devices, which inherently have processor and memories):
“a clustering component that clusters one or more participants in a federated learning system based on distributions of data classification labels for data sets of the one or more participants into one or more clusters of participants” (pg. 4 §3.2 “The second step of the modular algorithms deals with clustering the compute nodes based on their local ERMs” ERM is analogous to data classification labels and pg. 6 algorithm 2, the trimmed k-means);
While all federated learning inherently includes participant/client selection, Soltani more specifically teaches “a selection component that selects participants equitably from across the one or more clusters of participants for a round of federated learning” (Soltani pg. 4 Group-based selection “Clients are divided into several groups and then those within the same group are selected for each communication round. In [23], a grouping based participant selection mechanism is introduced in which participants are split into various groups based on group earth mover’s distance (GEMD) to balance the label distribution of the clients. This new metric evaluates similarity between global distribution and local data distributions. A smaller GEMD means that the training data of the selected clients are closer to IID distribution. Therefore, selecting a group of clients with the smallest GEMD can improve the performance of FL” which by grouping similar clients and then picking from each group, the selection is equitable (i.e. not random))
It would have been obvious to one having ordinary skill in the art at the time that the invention was effectively filed to combine the teachings of Ghosh with that of Soltani since “selecting a group of clients with the smallest GEMD can improve the performance of FL” Soltani Group-based selection. This shows that by combining the techniques, the federated learning system operates more efficiently.
Note that independent claims 7 and 14 recite the same substantial subject matter as independent claim 1, only differing in embodiments. The different embodiments, a method and computer-readable medium are obvious variations of another and therefore the claims are subject to the same rejection.
Regarding claims 2, 8, and 15, the Ghosh and Soltani references have been addressed above. Soltani further teaches “wherein the computer executable components further comprise: a communication component that establishes one or more secure communication channels between the one or more participants and a trusted execution environment” (Soltani abstract “Federated Learning (FL) is an efficient distributed machine learning paradigm that employs private datasets in a privacy preserving manner”)
Regarding claims 3, 9, and 16, the Ghosh and Soltani references have been addressed above. Soltani further teaches “wherein the communication component further receives the distributions of data classification labels for the data sets from the one or more participants over the one or more secure communication channels” (Soltani pg. 1 §1 ¶2 “A decentralized machine learning approach called Federated Learning (FL) has been proposed by Google that enables cooperative learning on devices without sharing the local data [2]. Clients train the model on-device in a privacy preserving manner using their local datasets and transfer the local model parameters to the FL server for aggregation. As a result, FL enables user privacy preservation, low communication costs, and transmission latency reduction owing to transmitting only model parameters to the server for aggregation”)
Regarding claims 4, 10, and 17, the Ghosh and Soltani references have been addressed above. Ghosh further teaches “wherein the clustering component operates within the trusted execution environment” (abstract “we propose a general statistical model for this problem which takes both the cluster structure of the users and the Byzantine machines into account. Then, leveraging the statistical model, we solve the robust heterogeneous Federated Learning problem optimally” everything operates within the federated learning system, i.e. trusted environment)
Regarding claims 5, 11, and 18, the Ghosh and Soltani references have been addressed above. Soltani further teaches “wherein the distributions of data classification labels for the data sets are stored in the trusted execution environment” (Soltani pg. 1 §1 ¶2 “A decentralized machine learning approach called Federated Learning (FL) has been proposed by Google that enables cooperative learning on devices without sharing the local data [2]. Clients train the model on-device in a privacy preserving manner using their local datasets and transfer the local model parameters to the FL server for aggregation. As a result, FL enables user privacy preservation, low communication costs, and transmission latency reduction owing to transmitting only model parameters to the server for aggregation”)
Regarding claims 13 and 20, the Ghosh and Soltani references have been addressed above. Ghosh further teaches “wherein the clustering comprises K-means clustering” (abstract “we prove statistical guarantees for an outlier-robust clustering algorithm, which can be considered as the Lloyd algorithm with robust estimation” wherein Lloyd algorithm is k-means clustering)
Claim(s) 6, 12, and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ghosh and Soltani further in view of Fraboni, Yann, et al. "Clustered sampling: Low-variance and improved representativity for clients selection in federated learning."
Regarding claims 6, 12, and 19, the Ghosh and Soltani references have been addressed above. While they teach participant selection, Fraboni more specifically teaches “wherein the selection component further: selects underrepresented participants from across the one or more clusters of participants for a second round of federated learning” (Fraboni abstract “we introduce clustered sampling for clients selection. We prove that clustered sampling leads to better clients representativity and to reduced variance of the clients stochastic aggregation weights in FL”)
It would have been obvious to one having ordinary skill in the art at the time that the invention was effectively filed to combine the teachings of Ghosh and Soltani with that of Fraboni since “We prove that clustered sampling leads to better clients representatitivity and to reduced variance of the clients stochastic aggregation weights in FL” Fraboni abstract. This shows that by combining the above techniques, one again the federated learning system becomes more robust and optimal.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KEVIN W FIGUEROA whose telephone number is (571)272-4623. The examiner can normally be reached Monday-Friday, 10AM-6PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, MIRANDA HUANG can be reached at (571)270-7092. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
KEVIN W FIGUEROA
Primary Examiner
Art Unit 2124
/Kevin W Figueroa/ Primary Examiner, Art Unit 2124
Prosecution Insights