DETAILED ACTION
10/27Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 10/10/2023. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more Claim s particularly pointing out and distinctly Claim ing the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more Claim s particularly pointing out and distinctly Claim ing the subject matter which the applicant regards as his invention.
Claims 2, 6-10,12, and 16-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly Claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
As to Claim 2 and Analogous Claim 12
The limitation “and the adaptation is that which most benefits, as among other possible adaptations “makes the scope of the Claim unclear on what most benefits denotes.
As to Claim 6 and Analogous Claim 16
The limitation “splitting is performed based on loss values obtained by evaluation of candidate ML models at the nodes”. There is insufficient antecedent basis for this limitation in the Claim . The recitation of “at the nodes” makes the scope of the Claim unclear on which nodes , splitting is being performed.
Claim s 7-10 are further rejected on virtue of their dependencies to Claim 6.
Claim s 17-20 are further rejected on virtue of their dependencies to Claim 16.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 1-20 is rejected under 35 USC § 101 because Claim ed invention is directed to the abstract idea without significantly more.
As to Claim 1:
Step 1 Analysis: Is the Claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Claim 1 is a method Claim , therefore it falls under one of four categories of statutory subject matter.
Step 2A Prong One Analysis: Does the Claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
The limitation” determining subsets of the edge nodes” is an abstract idea of
Mental process.(i.e. evaluation). See MPEP § 2106.04(a)(2)(III).
The limitation “by the central node, splitting a distillation process for each of the subsets of the edge nodes to generate distilled datasets” is an abstract idea of
Mental process.(i.e. evaluation). See MPEP § 2106.04(a)(2)(III).
“Adapt a base machine learning (ML) model for use at a newly deployed edge node that lacks adequate data to adapt the base ML model” is an abstract idea of
Mental process.(i.e. evaluation). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the Claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
The limitation “by a central node configured to communicate with edge nodes” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2).
The limitation “by the central node, leveraging the distilled datasets” ” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.05(f).
The limitation “ and deploying, by the central node, the base ML model, after adaptation, to the newly deployed edge node” is an additional element that amounts to adding insignificant extra-solution activity of mere data output to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g).
Step 2B Analysis: Does the Claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
The limitation “by a central node configured to communicate with edge nodes” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f).
The limitation “by the central node, leveraging the distilled datasets” is an additional element that does not add significantly more since the intended practical application is well-understood, routine, and conventional and stated as generic level (i.e. , “ apply it “, see MPEP 2106.05(f).
The limitation “ and deploying, by the central node, the base ML model, after adaptation, to the newly deployed edge node” is an additional element that amounts to adding insignificant extra-solution activity of mere data output to the judicial exception. 2106.05(g). Furthermore, the elements is directed to output the model to nodes which the courts have recognized as well‐understood, routine, and conventional when they are Claim ed in a generic manner. See MPEP § 2106.05(d)(II).
Therefore, in examining elements as recited by the limitations individually and as an ordered combination, as a whole the independent Claim limitations do not recite what have the courts have identified as “significantly more”.
As to Claim 11 :
Step 1 Analysis: Is the Claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Claim 11 is drawn to a storage medium consisting of Hardware components. i.e (article of manufacture), therefore Claim 11 falls under one of four categories of statutory subject matter.
Step 2A Prong Two Analysis: Does the Claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
The limitation “A non-transitory storage medium having stored therein instructions that are executable by one or more hardware processors to perform operation” is additional element is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2)
Step 2B Analysis: Does the Claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
The limitation “A non-transitory storage medium having stored therein instructions that are executable by one or more hardware processors to” is the additional Claim elements of one or more processors; a memory coupled to at least one of the processors; a stored instructions stored in the memory and executed by at least one of the processors, are not sufficient to amount to significantly more than the judicial exception since these additional Claim elements are recited at a high level of generality (i.e. using a generic processor and generic memory.
And for all other Claim elements of Claim 11 they are rejected using the PEG analysis of Claim 1 since they are analogous Claim s.
As to Claim 2 :
Step 2A Prong One Analysis: Does the Claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
The limitation “wherein adapting the base ML model comprises choosing an adaptation to be applied to the base ML model using one of the distilled datasets, and the adaptation is that which most benefits, as among other possible adaptations, the ML base model.” is an abstract idea of Mental process.(i.e. evaluation). See MPEP § 2106.04(a)(2)(III).
As to Claim 3 :
Step 2A Prong One Analysis: Does the Claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
The limitation “wherein the splitting comprises performing a federated dataset distillation process to generate the distilled datasets, and each of the distilled datasets is associated with a respective one of the subsets of the edge nodes. ”is an abstract idea of a of a mental process .See MPEP § 2106.04(a)(2)(III).
As to Claim 4 :
Step 2A Prong Two Analysis: Does the Claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
The limitation “ wherein privacy of data respectively associated with the edge nodes is maintained at all times.” is additional element that amounts to adding the words “apply it” (or an equivalent), such as mere instruction to implement an abstract idea. See MPEP 2106.05(f)(2)
Step 2B Analysis: Does the Claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
The limitation “ wherein privacy of data respectively associated with the edge nodes is maintained at all times.” is an additional element that does not add significantly more since the practical application is well-understood, routine, and conventional and stated at a generic level. (i.e., “apply it” See MPEP 2106.05(f).
As to Claim 5 :
Step 2A Prong One Analysis: Does the Claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
The limitation “ wherein a robust aggregation process is performed, for each of the subsets, to generate an update for the distilled dataset corresponding to that subset” is an abstract idea of mathematical concept. (i.e. performing mathematical operation between variables or numbers). See MPEP § 2106.04(a)(2)(I)(A).
As to Claim 6 :
Step 2A Prong Two Analysis: Does the Claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
The limitation “wherein the splitting is performed based on loss values obtained by evaluation of candidate ML models at the nodes, and the base ML model is one of the candidate ML models” is an abstract idea of Mental process.(i.e. evaluation). See MPEP § 2106.04(a)(2)(III).
As to Claim 7 :
Step 2A Prong Two Analysis: Does the Claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
The limitation.” , wherein each of the subsets is associated with a respective candidate ML model “is additional element that amounts to adding the words “apply it” (or an equivalent), such as mere instruction to implement an abstract idea. See MPEP 2106.05(f)(2)
Step 2B Analysis: Does the Claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
The limitation” wherein each of the subsets is associated with a respective candidate ML model” is an additional element that does not add significantly more since the practical application is well-understood, routine, and conventional and stated at a generic level. (i.e., “apply it” See MPEP 2106.05(f).
As to Claim 8 :
Step 2A Prong One Analysis: Does the Claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
The limitation “ wherein a learning rate is determined for each of the candidate ML model.” is an abstract idea of Mental process.(i.e. evaluation). See MPEP § 2106.04(a)(2)(III
As to Claim 9 :
Step 2A Prong One Analysis: Does the Claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
As to Claim 10 :
Step 2A Prong Two Analysis: Does the Claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
The limitation “wherein each of the distilled datasets is used to train one or more of the candidate ML models“ is additional element that amounts to adding the words “apply it” (or an equivalent), such as mere instruction to implement an abstract idea. See MPEP 2106.05(f)(2)
Step 2B Analysis: Does the Claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
The limitation “wherein each of the distilled datasets is used to train one or more of the candidate ML models” is an additional element that does not add significantly more since the practical application is well-understood, routine, and conventional and stated at a generic level. (i.e., “apply it” See MPEP 2106.05)
As to Claim 12 :
Step 2A Prong One Analysis: Does the Claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
The limitation “wherein adapting the base ML model comprises choosing an adaptation to be applied to the base ML model using one of the distilled datasets, and the adaptation is that which most benefits, as among other possible adaptations, the ML base model.” is an abstract idea of Mental process.(i.e. evaluation). See MPEP § 2106.04(a)(2)(III).
As to Claim 13 :
Step 2A Prong One Analysis: Does the Claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
The limitation “wherein the splitting comprises performing a federated dataset distillation process to generate the distilled datasets, and each of the distilled datasets is associated with a respective one of the subsets of the edge nodes. ”is an abstract idea of a of a mental process .See MPEP § 2106.04(a)(2)(III).
As to Claim 14 :
Step 2A Prong Two Analysis: Does the Claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
The limitation “ wherein privacy of data respectively associated with the edge nodes is maintained at all times.” is additional element that amounts to adding the words “apply it” (or an equivalent), such as mere instruction to implement an abstract idea. See MPEP 2106.05(f)(2)
Step 2B Analysis: Does the Claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
The limitation “ wherein privacy of data respectively associated with the edge nodes is maintained at all times.” is an additional element that does not add significantly more since the practical application is well-understood, routine, and conventional and stated at a generic level. (i.e., “apply it” See MPEP 2106.05(f).
As to Claim 15 :
Step 2A Prong One Analysis: Does the Claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
The limitation “ wherein a robust aggregation process is performed, for each of the subsets, to generate an update for the distilled dataset corresponding to that subset” is an abstract idea of mathematical concept. (i.e. performing mathematical operation between variables or numbers). See MPEP § 2106.04(a)(2)(I)(A).
As to Claim 16 :
Step 2A Prong Two Analysis: Does the Claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
The limitation “wherein the splitting is performed based on loss values obtained by evaluation of candidate ML models at the nodes, and the base ML model is one of the candidate ML models” is an abstract idea of Mental process.(i.e. evaluation). See MPEP § 2106.04(a)(2)(III).
As to Claim 17 :
Step 2A Prong Two Analysis: Does the Claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
The limitation.” , wherein each of the subsets is associated with a respective candidate ML model “is additional element that amounts to adding the words “apply it” (or an equivalent), such as mere instruction to implement an abstract idea. See MPEP 2106.05(f)(2)
Step 2B Analysis: Does the Claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
The limitation” wherein each of the subsets is associated with a respective candidate ML model” is an additional element that does not add significantly more since the practical application is well-understood, routine, and conventional and stated at a generic level. (i.e., “apply it” See MPEP 2106.05(f).
As to Claim 18 :
Step 2A Prong One Analysis: Does the Claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
The limitation “ wherein a learning rate is determined for each of the candidate ML model.” is an abstract idea of Mental process.(i.e. evaluation). See MPEP § 2106.04(a)(2)(III
As to Claim 19 :
Step 2A Prong One Analysis: Does the Claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
As to Claim 20 :
Step 2A Prong Two Analysis: Does the Claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
The limitation “wherein each of the distilled datasets is used to train one or more of the candidate ML models“ is additional element that amounts to adding the words “apply it” (or an equivalent), such as mere instruction to implement an abstract idea. See MPEP 2106.05(f)(2)
Step 2B Analysis: Does the Claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
The limitation “wherein each of the distilled datasets is used to train one or more of the candidate ML models” is an additional element that does not add significantly more since the practical application is well-understood, routine, and conventional and stated at a generic level. (i.e., “apply it” See MPEP 2106.05)
Claim Rejections – 35 USC § 103
The following is a quotation of 35 U.S.C. 103, which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a Claim ed invention may not be obtained, notwithstanding that the Claim ed invention is not identically disclosed as set forth in section 102, if the differences between the Claim ed invention and the prior art are such that the Claim ed invention as a whole would have been obvious before the effective filing date of the Claim ed invention to a person having ordinary skill in the art to which the Claim ed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claims 1-4, 6, 8, 10-14, 16, 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Song,et. al. “Federated Learning via Decentralized Dataset Distillation in Resource-Constrained Edge Environments” (“Song”) in view of Wang et. al. “Dataset Distillation (“Wang”) and in view of Zhou et. al “Distilled One-Shot Federated Learning” (“Zhou”)
As to Claim 1:
Song teaches “by a central node configured to communicate with edge nodes, determining subsets of the edge nodes” (Song, pg -1
PNG
media_image1.png
200
400
media_image1.png
Greyscale
Examiner notes: Under BRI, Server/ Cloud is interpreted as central node that is configured to communicate with clients , interpreted as edge nodes. Central node determines the subsets of the two dataset CIFAR 10 and MNIST.
2. by the central node, splitting a distillation process for each of the subsets of the edge nodes to generate distilled datasets; (Song, abs, pg-2, “[abs] FedD3 allows the connected clients to distill the local datasets independently, and then aggregates those decentralized distilled datasets (typically in the form a few unrecognizable images, which are normally smaller than a model) across the network only once to form the final model.
PNG
media_image2.png
728
936
media_image2.png
Greyscale
Examiner notes : Fig1 is further interpreted as splitting a distillation process, where it discloses “We distill 1 image per class in each of 10 clients.” Clients are interpreted as edge nodes where distilled dataset is generated for each of the subsets of edge nodes.”).
Under BRI Song teaches central node interpreted as server. (Song, fig-1,)
Song does not explicitly teach:
[by the central node] leveraging the distilled datasets to adapt a base machine learning (ML) model for use at a newly deployed edge node that lacks adequate data to adapt the base ML model.
and deploying, [by the central node], the base ML model, after adaptation, to the newly deployed edge node.
Wang teaches “[by the central node] leveraging the distilled datasets to adapt a base machine learning (ML) model for use at a newly deployed edge node that lacks adequate data to adapt the base ML model;” (Wang, “[pg-6], “[pg-6] Such learned distilled data essentially fine-tune weights pre-trained on one dataset to perform well for a new dataset, thus bridging the gap between the two domains. Domain mismatch and dataset bias represent a challenging problem in machine learning today (Torralba & Efros, 2011). Extensive prior work has been proposed to adapt models to new tasks and datasets [to adapt a base machine learning (ML) model] (Daume III, 2007; Saenko et al., 2010). In this work, we characterize the domain mismatch via distilled data [leveraging the distilled datasets] In Section 4.2, we show that a small number of distilled images are sufficient to quickly adapt (CNN models to new datasets and tasks. [for use at a newly deployed edge node that lacks adequate data]”).
Examiner notes: Song in fig 1 teaches (central node), interpreted as server (See fig 1)
Wang and Song are related to the same field of endeavor (i.e. Dataset Distillation). In view of the teachings of Wang it would have been obvious for a person of ordinary skill in the art to apply the teachings of Wang to Song before the effective filing date of the Claim ed invention in order to Improve
accuracy and optimize the performance of the models. ( Wang, “[pg-2] We distill the domain difference between SVHN and MNIST into 100 images. These images can quickly fine-tune pre-trained SVHN networks to achieve high accuracy on MNIST… To further boost performance, we propose an iterative version, where we obtain a sequence of distilled images and these distilled images can be trained with multiple epochs (passes).”).
Song in view of Wang teaches:
[by the central node] and [to the newly deployed edge node] (See, fig 1: from Song)
Song in view of Wang does not explicitly teach:
and deploying, [by the central node], the base ML model, after adaptation, [to the newly deployed edge node]
Zhou teaches “and deploying, by [the central node], the base ML model, after adaptation, [to the newly deployed edge node]. “(Zhou, [pg-2]
PNG
media_image3.png
705
1111
media_image3.png
Greyscale
Examiner notes: Song teaches (central node) interpreted as a server in fig:1: (See above). Also, Song in fig 1 : teaches newly deployed edge nodes interpreted as clients. Zhou teaches deploying Ml model after adaptation , after adaption the final model to all the edge nodes i.e. corresponding to the Claim .”).
Zhou and Song are related to the same field of endeavor (i.e. Dataset Distillation). In view of the teachings of Zhou it would have been obvious for a person of ordinary skill in the art to apply the teachings of Zhou to Song before the effective filing date of the Claim ed invention in order to reduce the communication cost significantly and improve accuracy of models in Federated learning.(Zhou, “[ abs] DOSFL serves as an inexpensive method to quickly converge on a performant pre-trained model with less than 0.1% communication cost of traditional methods.”). Additionally, person of ordinary skill in the art would be motivated to combine the teachings of Song to Zhou to protect the privacy of data at all times (i.e. both central node and other edge nodes does not have the access to the data.) ((Zhou, “[pg-1] The server averages these to obtain a global model. Since learning processes happen at local level, neither the server nor other clients directly observe a client's data.”).
As to Claim 11:
Song teaches “A non-transitory storage medium having stored therein instructions that are executable by one or more hardware processors to perform operation”(Song, “[pg-11], In this section, we report details on datasets, federated system parameters, and hyperparameters in our framework as well as in baselines in our experiments, which run on a computer cluster A non-[transitory storage medium having stored therein instructions that are ]with 4× NVIDIA-A100-PCIE-40GB GPUs [executable by one or more hardware processors to perform operation] and 4× 32-Core-AMD-EPYC-7513 CPUs.
And for all the other limitation of Claim 12, it is rejected under same basis as Claim 1. As the Claim are Analogous.
As to Claim 2 Analogous Claim 12 :
Song, as modified by Wang and Zhou, teaches the method of Claim 1.
Wang further teaches “wherein adapting the base ML model comprises choosing an adaptation to be applied to the base ML model using one of the distilled datasets, and the adaptation is that which most benefits, as among other possible adaptations, the ML base model.” (Wang, “[pg-6]Distillation with pre-trained weights. Such learned distilled data essentially fine-tune weights pre-trained on one dataset to perform well for a new dataset, thus bridging the gap between the two domains. Domain mismatch and dataset bias represent a challenging problem in machine learning today (Torralba & Efros, 2011). Extensive prior work has been proposed to adapt models to new tasks and datasets [choosing an adaptation to be applied to the base ML model] (Daume III, 2007; Saenko et al., 2010). In this work, we characterize the domain mismatch via distilled data [using one of the distilled datasets] In Section 4.2, we show that a small number of distilled images are sufficient to quickly adapt (CNN models to new datasets and tasks. [and the adaptation is that which most benefits, as among other possible adaptations, the ML base model.]
PNG
media_image4.png
217
848
media_image4.png
Greyscale
Song, Wang , and Zhou are combinable for the same rationale as set forth above with respect to Claim 1
As to Claim 3 Analogous Claim 13 :
Song, as modified by Wang and Zhou, teaches the method of Claim 1.
Song Further teaches “wherein the splitting comprises performing a federated dataset distillation process to generate the distilled datasets, and each of the distilled datasets is associated with a respective one of the subsets of the edge nodes”. (Song” [pg-2]Note that our method keeps one of the biggest advantages of federated learning: privacy[14,26,40,45]. It anonymously maps distilled datasets from the original client data without any exposure, which is analogous to the shared model parameters in previous federated learning methods, but substantially more efficient and effective.
PNG
media_image2.png
728
936
media_image2.png
Greyscale
Examiner notes: under BRI, FIG 1 IS interpreted as federated dataset distillation process to generate distilled datasets, Where CIFAR10 and MNIST datasets are splitted to generate a new dataset, and each client interpreted as edge nodes has its own distilled data sets which is interpreted as each of the distilled datasets is associated with a respective one of the subsets of the edge nodes.
Song, Wang , and Zhou are combinable for the same rationale as set forth above with respect to Claim 1
As to Claim 4 Analogous Claim 14:
Song, as modified by Wang and Zhou, teaches the method of Claim 1.
Song Further teaches “wherein privacy of data respectively associated with the edge nodes is maintained at all times.” ((Song “[Pg-2], Note that our method keeps one of the biggest advantages of federated learning: privacy [14, 26, 40, 45]. It anonymously maps distilled datasets from the original client data [data respectively associated with the edge nodes] without any exposure, [wherein privacy is maintained at all times] which is analogous to the shared model parameters in previous federated learning methods, but substantially more efficient and effective.
Song, Wang , and Zhou are combinable for the same rationale as set forth above with respect to Claim 1
As to Claim 6 and Analogous Claim 16 :
Song, as modified by Wang and Zhou, teaches the method of Claim 1.
Song Further teaches “wherein the splitting is performed based on loss values obtained by evaluation of candidate ML models at the nodes, and the base ML model is one of the candidate ML models.” ( Song- “pg[3] Instead of distilling models, our framework distills input data into smaller synthetic datasets in clients, , [wherein the splitting is performed] which can be used for training a more general model by only relying on one-shot communication. In a federated learning scenario, given a set of clients indexed by k, machine learning models [candidate ML models at the nodes and the base ML model is one of the candidate ML models] with weights wk are trained individually on local client datasets Dk = {(xi)|i = 1, 2, ..., nk}, where xi is one data point with its label yi in client k and nk is the number of the local data points. The goal of local training in client k is to minimize
PNG
media_image5.png
86
470
media_image5.png
Greyscale
where fi (w) is the loss function on one data point xi [loss values obtained by evaluation]with the label yi. Finally, the goal is to minimize aggregated local goals Fk(w) in (1):
PNG
media_image6.png
80
457
media_image6.png
Greyscale
Song, Wang , and Zhou are combinable for the same rationale as set forth above with respect to Claim 1
As to Claim 8 and Analogous Claim 18 :
Song, as modified by Wang and Zhou, teaches the method of Claim 6.
Wang Further teaches “wherein a learning rate is determined for each of the candidate ML models. “(Wang, “pg[7-8] We train the distilled images on 2000 random pre-trained models [each of the candidate ML models.]and evaluate them on unseen models.
PNG
media_image7.png
350
774
media_image7.png
Greyscale
Figure 2: Dataset distillation for fixed initializations: MNIST distilled images use one GD step and three epochs (10 images in total). CIFAR10 distilled images use ten GD steps and three epochs (100 images in total). For CIFAR10, only selected steps are shown. At left, we report the corresponding learning rates [wherein a learning rate is determined] for all three epochs.
Song, Wang , and Zhou are combinable for the same rationale as set forth above with respect to Claim 1
As to Claim 10 and Analogous Claim 20 :
Song, as modified by Wang and Zhou, teaches the method of Claim 6.
Song further teaches “wherein each of the distilled datasets is used to train one or more of the candidate ML models.” (Song, [pg-3]) Instead of distilling models, our framework distills input data into smaller synthetic datasets in clients [wherein each of the distilled datasets], which can be used for training[ is used to train] a more general model [one or more of the candidate ML models.] by only relying on one-shot communication.
Song, Wang , and Zhou are combinable for the same rationale as set forth above with respect to Claim 1
Claims 5 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over (“Song”) ”) in view (“Wang”) and in view of (“Zhou”) and in further view of Goetz et.al, “Federated Learning via Synthetic Data.”(“Goetz”)
As to Claim 5 and Analogous Claim 15:
Song, as modified by Wang and Zhou, teaches the method of Claim 1.
Song, as modified by Wang and Zhou, does not explicitly teach:
wherein a robust aggregation process is performed, for each of the subsets, to generate an update for the distilled dataset corresponding to that subset.
Goetz teaches “wherein a performing aggregation for robust aggregation process is performed, for each of the subsets, to generate an update for the distilled dataset corresponding to that subset.” (Goetz ,pg-3
PNG
media_image8.png
213
761
media_image8.png
Greyscale
Examiner notes: Dm is interpreted as subsets performing the robust aggregation to generate an update and gm is interpreted as updated distilled dataset corresponding to the subset Dm.
{ Furthermore, (Song pg-4, Algorithm-1, lines 4-5) discloses robust aggregation where ~D is interpreted as updated distilled dataset generated from the aggregation process where D~1, D~1 …. D~M is interpreted as the updated distilled dataset corresponding to that subset.}
Goetz and Song are related to the same field of endeavor (i.e. Dataset Distillation). In view of the teachings of Goetz it would have been obvious for a person of ordinary skill in the art to apply the teachings of Goetz to Song before the effective filing date of the Claimed invention to perform an aggregation to transform distributed or fragmented information into a formative, reusable distilled dataset in order to reduce the number of training samples and minimize memory and compute usage while allowing models learn effectively from fewer samples.(Goetz, “[pg-1], We will build on this method to present a procedure which can reduce the upload communication costs by one or two orders of magnitude, while still producing good server models.”).
Claims 7,9, 17 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over (“Song”) in view of (“Wang”) and in view of (“Zhou”) and in further view of Asad et.al, “Evaluating the Communication Efficiency in Federated Learning Algorithms.”(“Asad”)
As to Claim 7 and Analogous Claim 17 :
Song, as modified by Wang and Zhou, teaches the method of Claim 6.
Song, as modified by Wang and Zhou does not explicitly teach:
wherein each of the subsets is associated with a respective candidate ML model
Asad teaches “wherein each of the subsets is associated with a respective candidate ML model.” (Asad, “[pg-556]. In addition, the learning tasks and model classification for each algorithm is summarised in Table III. On the other hand, we have considered two ways of dividing the CIFAR-10 and MNIST datasets over the clients: 1) 11D, where the data is first shuffled and then divided among 100 clients, and 2) non-11D, where the data is first sorted according to their labels and then evenly divided among 100 clients.
PNG
media_image9.png
242
1011
media_image9.png
Greyscale
Table III: Hyper-Parameters and Models on Non-11D Data:-learning rate kept constant throughout training process
Examiner notes: Under BRI, Subsets are interpreted as Datasets of CIFAR & MNIST associated with respected model CNN .. Corresponding to the Claim .
Additionally, pg-556,”[ Algorithm -1], Subset β is associated with respective ML model WGM.
Asad and Song are related to the same field of endeavor (i.e. Dataset Distillation). In view of the teachings of Asad it would have been obvious for a person of ordinary skill in the art to apply the teachings of Asad to Song before the effective filing date of the Claim ed invention to link subsets and Ml model in order to preserve Data and privacy of users and optimize the performance of the model . (Asad, “[pg-553] Thirdly, privacy: the participants are not sending their raw data to the centralized server which ultimately guarantees each user privacy, and with this guaranteed privacy, the maximum number of users is able to participate in collaborative model training, and hence, the built model becomes better.”).
As to Claim 9 and Analogous Claim 19 :
Song, as modified by Wang and Zhou, teaches the method of Claim 6.
Song Further teaches “[wherein a first structure is employed] to annotate relations between edge nodes and subsets” (Song, pg-3 In a federated learning scenario, given a set of clients [edge nodes] indexed by k, machine learning models with weights wk are trained individually on local client datasets Dk = {(xi)|i = 1, 2, ..., nk}, [and subsets] where xi is one data point with its label yi [annotate relations] in client k and nk is the number of the local data points.
Song does not explicitly teach:
wherein a first structure is employed
Asad teaches “wherein a first structure is employed ” (Asad, “[pg-556]
PNG
media_image10.png
936
789
media_image10.png
Greyscale
Examiner notes: Song, pg[3] discloses the relation between edge nodes and subsets. Asad, discloses first structure interpreted as Client Update performing splitting task. Furthermore, Asad also discloses the edge nodes interpreted as Clients and Subsets interpreted as β. “).
Asad teaches “and a second structure is employed to track relations between the candidate ML models and the edge nodes.” (Asad, “[pg-556].
PNG
media_image11.png
769
783
media_image11.png
Greyscale
Examiner notes: Under BRI, Ml model is interpreted as WGM and Edge nodes is interpreted as Clients and Client training update is interpreted as Second Structure and For loop (i.e for every participant w ∈ St ..). is interpreted as second structure employed to track relations between the candidate models and edge nodes.
Song, Wang , Zhou and Asad are combinable for the same rationale as set forth above with respect to Claim 6.
Prior Art of Record
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Sucholutsky et al. (“Soft-Label Dataset Distillation and Text Dataset Distillation”) Is directed to leaning from small number of datasets to identify networks.( Sucholutsky, “[pg -4] Studying the distilled images produced by dataset distillation may enable us to identify what allows neural networks to generalize so quickly from so few of them.”). Additionally, Sucholutsky discloses
The use of respective distilled datasets to train the model and compare the model accuracy.( Sucholutsky, “[pg -1] For example, training a LeNet model with 10 distilled images (one per class) results in over 96% accuracy on MNIST, and almost 92% accuracy when trained on just 5 distilled images”).
Zhao et al. (“Dataset Condensation with Gradient Matching”) is directed to generating small set of distilled datasets that Performs similar to the network trained on large dataset. (Zhao, pg -2]. Additionally, it also discloses Data privacy in federated learning.[ Zhao pg-20]
and is directed to reduce the number of real samples needed by generating synthetic ones that match the gradient dynamics of the original dataset.[ Zhao algorithm-1]
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NIROJ KOIRALA whose telephone number is (571)270-0748. The examiner can normally be reached Monday -Friday 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, MICHAEL HUNTLEY can be reached on (303) 297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/N.K./Examiner, Art Unit 2129
/MICHAEL J HUNTLEY/Supervisory Patent Examiner, Art Unit 2129
Prosecution Insights