DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is in response to amendments filed January 20th, 2026, in which claims 1, 3-8, and 10-14 have been amended. No claims have been cancelled nor added. The amendments have been entered, and claims 1-14 are currently pending in the case. Claims 1 and 8 are independent claims.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-14 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claim 1:
Step 1: Claim 1 is directed to [a] learning system, therefore it falls under the statuary category of a machine.
Step 2A Prong 1: The claim recites, in part:
“select a mini-batch from the local data” this encompasses the mental selection of a mini-batch from observed local data.
“calculate an integrated parameter using the local data information acquired from each of the local devices” this limitation is a mathematical concept.
Step 2A Prong 2: The judicial exception is not integrated into a practical application; the remaining limitations of the claim are as follows: “each of the local devices comprises…a hardware processor configured to”, “train the local model using the mini-batch”, “generate local data information relating to the local data included in the mini-batch and indicating information different from a label”, “the server comprises…a hardware processor configured to”, “update the global model using the integrated parameter and the local model parameter acquired from each of the local devices” the limitations are an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP § 2106.05(f)(2).
“a storage storing local data and a local model comprising a neural network”, “transmit a local model parameter relating to the local model and the local data information to the server, the local model parameter including weight coefficients of the local model”, “a storage storing global data and a global model comprising a neural network” the limitation is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP § 2106.05(g).
“a server”, “a plurality of local devices operably coupled to the server”, “at least two of the local models stored among the plurality of local devices differ from each other in terms of structure” the limitation is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(h).
Step 2B: The additional elements “each of the local devices comprises…a hardware processor configured to”, “train the local model using the mini-batch”, “generate local data information relating to the local data included in the mini-batch and indicating information different from a label”, “the server comprises…a hardware processor configured to”, “update the global model using the integrated parameter and the local model parameter acquired from each of the local devices”, “a server”, “a plurality of local devices operably coupled to the server”, “at least two of the local models stored among the plurality of local devices differ from each other in terms of structure”, taken individually and in combination, do not provide an inventive concept of significantly more than the abstract idea itself for the reasons set forth in step 2A prong 2 above. Further, “a storage storing local data and a local model comprising a neural network”, “transmit a local model parameter relating to the local model and the local data information to the server, the local model parameter including weight coefficients of the local model”, “a storage storing global data and a global model comprising a neural network” these limitations are an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP § 2106.05(g). Furthermore the additional element is directed to receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d or the additional element is directed to storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015). See MPEP § 2106.05(d)/(II). Therefore, the claim is ineligible.
Regarding claim 2, the rejection of claim 1 is incorporated and further:
Step 2A Prong 1: The claim recites, in part:
“the local data information is at least one of a frequency distribution of a loss for the local model, a frequency distribution of a loss for the global model, a statistical value, or a frequency distribution of attribute information relating to the local data included in the mini-batch” a continuation of the abstract idea identified in the parent claim.
Step 2A Prong 2: The claim does not recite any additional limitations, thus does not further recite any additional elements that integrates the judicial exception into a practical application or amount to significantly more.
Regarding claim 3, the rejection of claim 1 is incorporated and further:
Step 2A Prong 1: The claim recites, in part:
“generates the integrated parameter by performing weighted-averaging of the local model parameters based on the local data information” this limitation is a mathematical concept.
Step 2A Prong 2: The claim does not recite any additional limitations, thus does not further recite any additional elements that integrates the judicial exception into a practical application or amount to significantly more.
Regarding claim 4, the rejection of claim 1 is incorporated and further:
Step 2A Prong 1: The claim recites, in part:
“generate selection request information for controlling a direction that training of each local model will take based on a history of the local data information” this encompasses the mental generation of selection request information based on an observed history of local data information.
Step 2A Prong 2: The judicial exception is not integrated into a practical application; the remaining limitations of the claim are as follows: “transmit the selection request information to the plurality of local devices” the limitation is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP § 2106.05(g).
Step 2B: The additional elements, taken individually and in combination, do not provide an inventive concept of significantly more than the abstract idea itself for the reasons set forth in step 2A prong 2 above. Further, “transmit the selection request information to the plurality of local devices” the limitation is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP § 2106.05(g). Furthermore the additional element is directed to receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d. Therefore, the claim is ineligible.
Regarding claim 5, the rejection of claim 4 is incorporated and further:
Step 2A Prong 1: The claim recites, in part:
“selects a mini-batch based on the selection request information transmitted thereto” this encompasses the mental selection of an observed mini-batch based on other information.
Step 2A Prong 2: The judicial exception is not integrated into a practical application; the remaining limitations of the claim are as follows: “in each of the local devices, the processor of the local device…selects” the limitation is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP § 2106.05(f)(2).
Step 2B: The additional elements, taken individually and in combination, do not provide an inventive concept of significantly more than the abstract idea itself for the reasons set forth in step 2A prong 2 above. Therefore, the claim is ineligible.
Regarding claim 6, the rejection of claim 1 is incorporated and further:
Step 2A Prong 1: The claim recites, in part:
“calculates the local data information based on a computing cost required for training” this limitation is a mathematical concept.
Step 2A Prong 2: The judicial exception is not integrated into a practical application; the remaining limitations of the claim are as follows: “the hardware processor of the local device trains the local model…”, “the hardware processor of the local device…calculates” the limitations are an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP § 2106.05(f)(2). “the local model is a scalable neural network capable of adjusting a computing cost” the limitation is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(h). “stored in the storage thereof” the limitation is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP § 2106.05(g).
Step 2B: The additional elements “the hardware processor of the local device trains the local model…”, “the hardware processor of the local device…calculates”, “the local model is a scalable neural network capable of adjusting a computing cost”, taken individually and in combination, do not provide an inventive concept of significantly more than the abstract idea itself for the reasons set forth in step 2A prong 2 above. Further, “stored in the storage thereof” the limitation is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP § 2106.05(g). Furthermore the additional element is directed to storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015). See MPEP § 2106.05(d)/(II). Therefore, the claim is ineligible.
Regarding claim 7, the rejection of claim 6 is incorporated and further:
Step 2A Prong 1: a continuation of the abstract idea identified in the parent claim.
Step 2A Prong 2: The judicial exception is not integrated into a practical application; the remaining limitations of the claim are as follows: “the hardware processor of the server is further configured to”, “the hardware processor of the predetermined local device trains the local model stored in the storage thereof based on the request information” the limitations are an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP § 2106.05(f)(2). “transmit request information relating to the computing cost to a predetermined local device among the local devices” the limitation is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP § 2106.05(g).
Step 2B: The additional elements “the hardware processor of the server is further configured to”, “the hardware processor of the predetermined local device trains the local model stored in the storage thereof based on the request information”, taken individually and in combination, do not provide an inventive concept of significantly more than the abstract idea itself for the reasons set forth in step 2A prong 2 above. Further, “transmit request information relating to the computing cost to a predetermined local device among the local devices” the limitation is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. Furthermore the additional element is directed to receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d. See MPEP § 2106.05(g). Therefore, the claim is ineligible.
Regarding claim 8:
Step 1: Claim 8 is directed to [a] learning method, therefore it falls under the statuary category of a process.
Step 2A Prong 1: The claim recites, in part:
“selecting a mini-batch from the local data” this encompasses the mental selection of a mini-batch from observed local data.
“calculating an integrated parameter using the local data information acquired from each of the local devices” this limitation is a mathematical concept.
Step 2A Prong 2: The judicial exception is not integrated into a practical application; the remaining limitations of the claim are as follows: “each of the local devices comprises…a hardware processor configured to”, “training the local model using the mini-batch”, “generating local data information relating to the local data included in the mini-batch and indicating information different from a label”, “the server comprises…a hardware processor configured to”, “update the global model using the local model parameter and the integrated parameter acquired from each of the local devices” the limitations are an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP § 2106.05(f)(2).
“a storage storing local data and a local model comprising a neural network”, “transmitting a local model parameter relating to the local model and the local data information to the server, the local model parameter including weight coefficients of the local model”, “a storage storing global data and a global model comprising a neural network” the limitation is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP § 2106.05(g).
“a server”, “a plurality of local devices operably coupled to the server”, “at least two of the local models stored among the plurality of local devices differ from each other in terms of structure” the limitation is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(h).
Step 2B: The additional elements “each of the local devices comprises…a hardware processor configured to”, “train the local model using the mini-batch”, “generate local data information relating to the local data included in the mini-batch and indicating information different from a label”, “the server comprises…a hardware processor configured to”, “update the global model using the local model parameter and the integrated parameter acquired from each of the local devices”, “a server”, “a plurality of local devices operably coupled to the server”, “at least two of the local models stored among the plurality of local devices differ from each other in terms of structure”, taken individually and in combination, do not provide an inventive concept of significantly more than the abstract idea itself for the reasons set forth in step 2A prong 2 above. Further, “a storage storing local data and a local model comprising a neural network”, “transmitting a local model parameter relating to the local model and the local data information to the server, the local model parameter including weight coefficients of the local model”, “a storage storing global data and a global model comprising a neural network” these limitations are an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP § 2106.05(g). Furthermore the additional element is directed to receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d or the additional element is directed to storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015). See MPEP § 2106.05(d)/(II). Therefore, the claim is ineligible.
Regarding claims 9-14:
The rejection of claim 8 is further incorporated, the rejection of claims 2-7 are applicable to claims 9-14, respectively.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1 and 8 are rejected under 35 U.S.C. 103 as being unpatentable over Lee et al. (US 2022/0222578 A1) (hereinafter “Lee”) in further view of Diao et al. (“HeteroFL: Computation And Communication Efficient Federated Learning For Heterogeneous Clients”, Diao et al., 14 Dec 2021) (hereinafter “Diao”).
Regarding claim 1:
Lee teaches [a] learning system comprising:
a server; and a plurality of local devices and a operably coupled to the server (Lee, ¶4 “More specifically, each client may collect data, train the local model, and upload information on the trained local model to a server.”),
wherein: each of the local devices comprising comprises: a storage storing local data and a local model comprising a neural network; and a hardware processor configured to (Lee, ¶37-38 “The server 100 and the clients 200 may include a processor that performs deep learning operations, a storage device that stores a learning model, training data, etc., and a communication device that enables communication therebetween, and the like, and one or more clients 200 may be connected to the server 100 to perform federated learning.
In an example, a user's smart device, personal computer, etc. may be used as the client 200.” further, the training data is local data collected by the model, Lee, ¶4 “More specifically, each client may collect data, train the local model, and upload information on the trained local model to a server.”):
select a mini-batch from the local data (Lee, ¶8 “generating, by the client, a learning mini-batch by adjusting a ratio between samples classified into the two categories and included in the learning mini-batch to a preset ratio” further, the training data is local data collected by the model, Lee, ¶4 “More specifically, each client may collect data, train the local model, and upload information on the trained local model to a server.”);
train the local model using the mini-batch (Lee, ¶8 “and training, by the client, a learning model by implementing the mini-batch with the adjusted sample ratio.”);
generate local data information relating to the local data included in the mini-batch and indicating information different from a label (Lee, ¶64 “Subsequently, the client 200 may perform learning using the mini-batch with the adjusted sample ratio (operation S230). That is, the client 200 may perform the learning by configuring the mini-batch to contain a certain proportion of forgettable samples while configuring the mini-batch randomly.” Here the proportion of forgettable samples can be considered local data information, it can be considered to indicate information different from a label as it relates to a proportion of forgettable samples rather than the samples actual labels); and
transmit a local model parameter relating to the local model and the local data information to the server (Lee, ¶65 “When the learning is completed, the client 200 may proceed to the above-described operation S300 of FIG. 2, and transmit information of the updated learning model to the server 100.”), the local model parameter including weight coefficients of the local model (Lee, ¶17 “The information on the learning model and the information on the trained learning model may be weights for the learning model.”),
the server comprises: a storage storing global data and a global model comprising a neural network (Lee, ¶40 “First, the server 100 may generate an initial learning model (global model) using training data held in the server 100” here, the training data held in the server can be considered the global data); and
a hardware processor configured to: calculate an integrated parameter using the local data information acquired from each of the local devices (Lee, ¶40 “When the learning is performed in this manner, each client 200 may have an individually updated learning model (local model), and each client 200 may transmit information on the learning model updated in this manner to the server 100 (operation S300).” Here, the information on the learning model can be considered the integrated parameter from local data information); and
update the global model using the integrated parameter and the local model parameter acquired from each of the local devices (Lee, ¶40 “The server 100 may update the initial learning model (global model) based on the information collected from the clients 200 (operation S400) and distribute the updated learning model to the clients 200 (operation S500).”),
Lee does not teach “at least two of the local models stored among the plurality of local devices differ from each other in terms of structure”
However, Diao teaches at least two of the local models stored among the plurality of local devices differ from each other in terms of structure (Diao, page 6, ¶3-4 “To study the effectiveness of our proposed HeteroFL framework, we construct five different computation complexity levels {a,b,c,d,e} with the hidden channel shrinkage ratio r = 0.5. We have tried various shrinkage ratios, and we found that it is most illustrative to use the discrete complexity levels 0.5, 0.25, 0.125, and 0.0625 (relative to the most complex model). For example, model ‘a’ has all the model parameters, while models ‘b’ to ‘e’ have the effective shrinkage ratios 0.5, 0.25, 0.125, and 0.0625. We note that the complexity of ‘e’ is close to a logistic regression model. Our experiments indicated that the ratio can be arbitrary between (0,1] and dynamically change. In practice, using a dictionary of discrete complexity levels are convenient for coordination purposes. Each local client is assigned an initial computation complexity level” Diao, further page 6, ¶4 “When 10% clients use the model ’a’ and 90% use the model ’e’, the average number of model parameters is 0.1 × (size of model ‘a’) + 0.9 × (size of model ‘e’).” here, the different model architectures {a,b,c,d,e} used at different clients can be considered at least two models with different structure).
Lee and Diao are analogous art because both references concern methods for federated learning. Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Lee/Diao’s federated learning system to incorporate the differing model structures taught by Diao. The motivation for doing so would have been to address heterogeneous clients equipped with very different computation and communication capabilities as stated in Diao, ¶2 “In this work, we propose a new federated learning framework named HeteroFL to address heterogeneous clients equipped with very different computation and communication capabilities…For the first time, our method challenges the underlying assumption of existing work that local models have to share the same architecture as the global model.”
Regarding claim 8:
Lee teaches [a] learning method implemented by a learning system, the learning system comprising a server and plurality of local devices operably coupled to the server (Lee, ¶4 “More specifically, each client may collect data, train the local model, and upload information on the trained local model to a server.”), and the learning method comprising: at each of the plurality of local devices (Lee, ¶4 “More specifically, each client may collect data, train the local model, and upload information on the trained local model to a server.”):
storing, in a storage of the local device, local data and a local model comprising a neural network (Lee, ¶37-38 “The server 100 and the clients 200 may include a processor that performs deep learning operations, a storage device that stores a learning model, training data, etc., and a communication device that enables communication therebetween, and the like, and one or more clients 200 may be connected to the server 100 to perform federated learning.
In an example, a user's smart device, personal computer, etc. may be used as the client 200.” further, the training data is local data collected by the model, Lee, ¶4 “More specifically, each client may collect data, train the local model, and upload information on the trained local model to a server.”):
selecting a mini-batch from the local data (Lee, ¶8 “generating, by the client, a learning mini-batch by adjusting a ratio between samples classified into the two categories and included in the learning mini-batch to a preset ratio” further, the training data is local data collected by the model, Lee, ¶4 “More specifically, each client may collect data, train the local model, and upload information on the trained local model to a server.”);
training the local model using the mini-batch (Lee, ¶8 “and training, by the client, a learning model by implementing the mini-batch with the adjusted sample ratio.”);
generating local data information relating to the local data included in the mini-batch and indicating information different from a label(Lee, ¶64 “Subsequently, the client 200 may perform learning using the mini-batch with the adjusted sample ratio (operation S230). That is, the client 200 may perform the learning by configuring the mini-batch to contain a certain proportion of forgettable samples while configuring the mini-batch randomly.” Here the proportion of forgettable samples can be considered local data information, it can be considered to indicate information different from a label as it relates to a proportion of forgettable samples rather than the samples actual labels); and
transmitting a local model parameter relating to the local model and the local data information to the server (Lee, ¶65 “When the learning is completed, the client 200 may proceed to the above-described operation S300 of FIG. 2, and transmit information of the updated learning model to the server 100.”), the local model parameter including weight coefficients of the local model (Lee, ¶17 “The information on the learning model and the information on the trained learning model may be weights for the learning model.”),
at the server: storing, in a storage of the server, global data and a global model comprising a neural network (Lee, ¶40 “First, the server 100 may generate an initial learning model (global model) using training data held in the server 100” here, the training data held in the server can be considered the global data);
calculating an integrated parameter using the local data information acquired from each of the local devices (Lee, ¶40 “When the learning is performed in this manner, each client 200 may have an individually updated learning model (local model), and each client 200 may transmit information on the learning model updated in this manner to the server 100 (operation S300).” Here, the information on the learning model can be considered the integrated parameter from local data information); and
updating the global model using the local model parameter and the integrated parameter acquired from each of the local devices (Lee, ¶40 “The server 100 may update the initial learning model (global model) based on the information collected from the clients 200 (operation S400) and distribute the updated learning model to the clients 200 (operation S500).”).
Lee does not teach “wherein at least two of the local models stored among the plurality of local devices differ from each other in terms of structure”
However, Diao teaches wherein at least two of the local models stored among the plurality of local devices differ from each other in terms of structure (Diao, page 6, ¶3-4 “To study the effectiveness of our proposed HeteroFL framework, we construct five different computation complexity levels {a,b,c,d,e} with the hidden channel shrinkage ratio r = 0.5. We have tried various shrinkage ratios, and we found that it is most illustrative to use the discrete complexity levels 0.5, 0.25, 0.125, and 0.0625 (relative to the most complex model). For example, model ‘a’ has all the model parameters, while models ‘b’ to ‘e’ have the effective shrinkage ratios 0.5, 0.25, 0.125, and 0.0625. We note that the complexity of ‘e’ is close to a logistic regression model. Our experiments indicated that the ratio can be arbitrary between (0,1] and dynamically change. In practice, using a dictionary of discrete complexity levels are convenient for coordination purposes. Each local client is assigned an initial computation complexity level” Diao, further page 6, ¶4 “When 10% clients use the model ’a’ and 90% use the model ’e’, the average number of model parameters is 0.1 × (size of model ‘a’) + 0.9 × (size of model ‘e’).” here, the different model architectures {a,b,c,d,e} used at different clients can be considered at least two models with different structure).
Lee and Diao are analogous art because both references concern methods for federated learning. Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Lee/Diao’s federated learning system to incorporate the differing model structures taught by Diao. The motivation for doing so would have been to address heterogeneous clients equipped with very different computation and communication capabilities as stated in Diao, ¶2 “In this work, we propose a new federated learning framework named HeteroFL to address heterogeneous clients equipped with very different computation and communication capabilities…For the first time, our method challenges the underlying assumption of existing work that local models have to share the same architecture as the global model.”
Claims 2, 3, 9 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Lee in view of Diao in further view of Guttmann et al. (US 2020/0202243 A1) (as cited in the IDS, hereinafter “Guttmann”).
Regarding claim 2:
Lee in view of Diao teaches [t]he system according to claim 1,
Lee in view of Diao does not teach “wherein the local data information is at least one of a frequency distribution of a loss for the local model, a frequency distribution of a loss for the global model, a statistical value, or a frequency distribution of attribute information relating to the local data included in the mini-batch”
However, Guttmann teaches wherein the local data information is at least one of a frequency distribution of a loss for the local model, a frequency distribution of a loss for the global model, a statistical value, or a frequency distribution of attribute information relating to the local data included in the mini-batch (Guttmann, ¶127 “In some embodiments, information related to a distribution of a plurality of training examples may be obtained. For example, Step 720 may comprise obtaining information related to the distribution of the first plurality of training examples of Step 710. In another example, Step 740 may comprise obtaining information related to the distribution of the second plurality of training examples of Step 730. In yet another example, Step 840 may comprise obtaining information related to the distribution of the third plurality of training examples of Step 830. In an additional example, Step 860 may comprise obtaining information related to the distribution of the fourth plurality of training examples of Step 850.” Here, each plurality of training examples can be considered a mini-batch, and the information related to the distribution can be considered frequency distribution of attribute information. It is noted the claim recites alternative language, and Guttmann teaches at least one of the alternatives.).
Lee in view of Diao and Guttmann are analogous art because both references concern methods for federated learning. Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Lee/Diao’s federated learning system to incorporate the balanced distribution information taught by Guttmann. The motivation for doing so would have been to balance the federated learning as stated in Guttman, ¶2 “More particularly, the disclosed embodiments relate to systems and methods for balanced federated learning.”
Regarding claim 3:
Lee in view of Diao teaches [t]he system according to claim 1,
Lee in view of Diao does not teach “wherein the hardware processor of the server calculates the integrated parameter by performing weighted-averaging of the local model parameters based on the local data information”
However, Guttmann teaches wherein the hardware processor of the server calculates the integrated parameter by performing weighted-averaging of the local model parameters based on the local data information (Guttmann, ¶12 “In some examples, the information related to the distribution of the first plurality of training examples may be used to determine a weight for the first update information, and the information related to the distribution of the second plurality of training examples may be used to determine a weight for the second update information. The first update information, the weight for the first update information, the second update information and the weight for the second update information may be used to determine the global update to the inference model.”).
Lee in view of Diao and Guttmann are analogous art because both references concern methods for federated learning. Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Lee/Diao’s federated learning system to incorporate the distribution information taught by Guttmann. The motivation for doing so would have been to better balance the federated learning as stated in Guttman, ¶2 “More particularly, the disclosed embodiments relate to systems and methods for balanced federated learning.”.
Regarding claims 9-10:
Claims 9-10 are rejected under the same rationale as claims 2-3, respectively.
Claims 4, 5, 11 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Lee in view of Diao in further view of Verma et al. (US 2020/0219014 A1) (hereinafter “Verma”).
Regarding claim 4:
Lee in view of Diao teaches [t]he system according to claim 1,
Lee in view of Diao does not teach “wherein the hardware processor of the server is further configured to:
generate selection request information for controlling a direction that training of each local model will take based on a history of the local data information; and
transmit the selection request information to the plurality of local devices”
However, Verma teaches wherein the hardware processor of the server is further configured to:
generate selection request information for controlling a direction that training of each local model will take based on a history of the local data information (Verma, ¶34 “The agents share these statistics with the fusion server, which in turn detects class imbalances at each system node. Using the class imbalance distribution, the fusion server determines which classes of data need to be shared by the agents in order for each system node to store each class of data. The fusion server also determines the permutation order for distribution of data based upon the aggregated statistical characteristics at the system nodes.” Here, the statistics can be considered the historical local data information and the classes of data to be shared and permutation information can be considered the selection request); and
transmit the selection request information to the plurality of local devices (Verma, ¶42 “Referring to FIG. 3, the fusion server creates a set of control and coordination instructions for each agent at the system nodes 302. Such a set of control and coordination instructions may include a permutation order for moving the neural network models between system nodes for training” here, the permutation order which can be considered the selection request information is transmitted to the local devices).
Lee in view of Diao and Verma are analogous art because both references concern methods for federated learning. Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Lee/Diao’s federated learning system to incorporate the mini-batches to control model training taught by Verma. The motivation for doing so would have been to overcome shortcomings in controlling direction as stated in Verma, ¶35 “The above-described aspects of the invention address the shortcomings of the prior art by directing mini-batches to be constructed containing a representative sample of all classes data to each system node.”
Regarding claim 5:
Lee in view of Diao in further view of Verma teaches [t]he system according to claim 4, wherein, in each of the local devices, the hardware processor of the local device selects a mini-batch based on the selection request information transmitted thereto (Verma, ¶34 “The system causes each agent to create mini-batches of data to train an AI model by taking statistical characteristics into consideration.”).
It would have been obvious to combine the teachings of Lee and Verma for the reasons set forth in connection with claim 4 above.
Regarding claims 11-12:
Claims 11-12 are rejected under the same rationale as claims 4-5, respectively.
Claims 6, 7, 13 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Lee in view of Diao in further view of Lee et al. (“NeuralScale: Efficient Scaling of Neurons for Resource-Constrained Deep Neural Networks”, 2020) (hereinafter “Lee 2”).
Regarding claim 6:
Lee in view of Diao teaches [t]he system according to claim 1, wherein….
the hardware processor of the local device trains the local model stored in the storage thereof (Lee, ¶4 “The client may update the local model with the information on the global model received from the server.”), and
Lee in view of Diao does not teach “the local model is a scalable neural network capable of adjusting a computing cost,
the hardware processor of the local device calculates the local data information based on a computing cost required for training”
However Lee 2 teaches the local model is a scalable neural network capable of adjusting a computing cost (Lee 2, page 3, col 2, ¶2 “We conjecture that when all layers are pruned by at least a single parameter, most redundancy is removed and the residual parameters compose an efficient configuration. We stop pruning once the number of neurons (filters) is less than
ϵ
(we pick
ϵ
as 5% of the total neurons (filters) of the network in our implementation).” Here,
ϵ
can be considered the computing cost),
the hardware processor of the local device calculates the local data information based on a computing cost required for training (Lee 2, page 6, ¶1 “Second, the scaling of network should not be done linearly as was done in [23] and should follow a non-linear rule that we attempt to approximate using a power function. If we look closely, by applying architecture descent on datasets of higher complexity generates network with configuration that has more filters allocated toward the end.” Here the complexity can be considered the local data information based on a computing cost).
Lee in view of Diao and Lee 2 are analogous art because both references concern methods for training neural networks. Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Lee/Diao’s federated learning system to incorporate the pruning taught by Lee 2. The motivation for doing so would have been to improve accuracy as stated in Lee 2. page 3, ¶1 “A LT is a sparse architecture that is the result of unstructured pruning and has accuracy better than the original network (usually found at a parameter count of an order less than the parameter count of the original network).”.
Regarding claim 7:
Lee in view of Diao in further view of Lee 2 teaches [t]he system according to claim 6, wherein
the hardware processor of the server is further configured to transmit request information relating to the computing cost (Lee 2, page 3, col 2, ¶2 “We conjecture that when all layers are pruned by at least a single parameter, most redundancy is removed and the residual parameters compose an efficient configuration. We stop pruning once the number of neurons (filters) is less than
ϵ
(we pick
ϵ
as 5% of the total neurons (filters) of the network in our implementation).”) to a predetermined local device among the local devices, (Lee, ¶61 “In this example, the ratio between forgettable samples and unforgettable samples may be transmitted from the server 100 to the client 200. The server 100 may be configured to choose the ratio in consideration of the type of deep learning model, or may be configured to set the ratio when a user inputs the ratio to the server 100.”), and
the hardware processor of the predetermined local device trains the local model stored in the storage thereof based on the request information (Lee 2, page 5, col 2, ¶1 “Upon the completion of iterative pruning and parameter searching, we obtain a set of parameters that scales our network in a more efficient manner. We can then use this set of parameters to scale-up our network as shown in Algorithm 3 for further pruning. We then proceed with several iterations of architecture descent until the changes in the architecture configuration is minuscule, indicating convergence.” Here the pruning and architecture descent can be considered training based on request information).
Regarding claims 13-14:
Claims 13-14 are rejected under the same rationale as claims 6-7, respectively.
Response to Arguments
Applicant's arguments filed January 20th, 2026 (hereinafter “Remarks”) have been fully considered but they are not persuasive.
Regarding the objections to the title, Applicant’s amended Specification has overcome the objections, which are withdrawn.
Applicant’s arguments with respect to 35 U.S.C. § 102 and 35 U.S.C. § 103 rejections have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Rejections under 35 U.S.C. § 101:
Argument 1:
“[a]mended claim 1 is not “directed to” a mental process or mathematical relationship, but to a specific structure of a learning system/machine-learning training technique” (Remarks, page 10).
Examiners Response:
Examiner respectfully disagrees, the MPEP states “It is essential that the broadest reasonable interpretation (BRI) of the claim be established prior to examining a claim for eligibility. The BRI sets the boundaries of the coverage sought by the claim and will influence whether the claim seeks to cover subject matter that is beyond the four statutory categories or encompasses subject matter that falls within the exceptions.” See MPEP § 2106(II). The broadest reasonable interpretation of amended claim 1 can still be considered a mental process and mathematical relationships. The claim does not clarify the functions of the local and global aspects, and does not require that the local devices are not coupled to the server. Therefore, the implementation can be considered directed to a mental process that can be performed in the human mind, or by a human using a pen and paper. CyberSource Corp. v. Retail Decisions, Inc., 654 F.3d 1366, 1372, 99 USPQ2d 1690, 1695 (Fed. Cir. 2011). See MPEP § 2106.04(a)(2)(III).
Argument 2:
“[A]mended claim 1 realizes a distinct improvement in the training of a machine-learning model. In particular, with the technique recited in amended claim 1, it is possible to implement a model update while taking into consideration even a small number of important pieces of local data, to increase the influence of the parameters of the local model trained in a state where many pieces of local data that are hard to recognize are included, and to reduce a time necessary for training of the global model and the local models….even if amended claim 1 could be interpreted as reciting an abstract idea, the (alleged) abstract idea is clearly integrated into a practical application which results on a concrete technological improvement to computer-related technology.” (Remarks, pages 10-11).
Examiners Response:
Examiner respectfully disagrees, the MPEP states “An inventive concept “cannot be furnished by the unpatentable law of nature (or natural phenomenon or abstract idea) itself.” Genetic Techs. v. Merial LLC, 818 F.3d 1369, 1376, 118 USPQ2d 1541, 1546 (Fed. Cir. 2016).” See MPEP § 2106.05(I). Furthermore, “It should be noted that while this consideration is often referred to in an abbreviated manner as the “improvements consideration,” the word “improvements” in the context of this consideration is limited to improvements to the functioning of a computer or any other technology/technical field, whether in Step 2A Prong Two or in Step 2B.” See MPEP § 2106.04(d)(1). This argument is unpersuasive — the applicant merely uses a computer to perform processes which can be performed by a mental process. An improvement to taking into consideration even a small number of important pieces of local data, to increase the influence of the parameters may be an improvement in an abstract idea, but not an improvement in the functioning of a computer, or another technology.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Makhija et al. (“Architecture Agnostic Federated Learning for Neural Networks”, Makhija et al., 17 Feb 2022) discloses the Federated Heterogeneous Neural Networks (FedHeNN) framework that allows each client to build a personalized model without enforcing a common architecture across clients. This allows each client to optimize with respect to local data and compute constraints, while still benefiting from the learnings of other (potentially more powerful) clients.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JACOB Z SUSSMAN MOSS whose telephone number is (571) 272-1579. The examiner can normally be reached Monday - Friday, 9 a.m. - 5 p.m. ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kakali Chaki can be reached on (571) 272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.S.M./Examiner, Art Unit 2122
/KAKALI CHAKI/Supervisory Patent Examiner, Art Unit 2122
Prosecution Insights