DETAILED ACTION
Remarks
This office action is issued in response to communication filed on 10/15/2025. Claims 1-20 are pending in this Office Action.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 10/15/2025 with respect to claim 19 rejected under 35 USC 102 and 103 have been fully considered and are moot in view of new ground of rejection.
Applicant’s arguments with respect to the 35 USC 101 rejection have been considered and are not persuasive. The examiner respectfully traverses applicant’s arguments.
Applicant argues: “the Applicant's attorney respectfully requests reconsideration with respect to the claims "reflect[ing] an improvement to the technology or technical field" or "the search for a technological solution to a technological problem." The Memo, p. 4.(Applicant’s arguments at page 13)
The examiner respectfully disagrees. The combination of elements of the claim 11-19 when considered as a whole, do not reflect any improvement to the technology or technical field as applicant asserts. Instead, the claims 11-19 recite a series of steps that include judicial exception as well as insignificant extra solution activities and therefore, when considered a whole, do not provide an inventive concept. Accordingly, the examiner maintains the 101 rejection of claims 11-19.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
2. Claims 11-19 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. However, claims 1-10 and 20 are eligible because these claims do not recite Judicial exception.
Claims 11- 19
Step 1: Statutory Category ?: Yes. claim 11 -19 recite a method (i.e., a “process”) which is statutory category.
Step 2A-Prong 1: Judicial Exception Recited ?: Yes.
Claim 11 recites the limitation of “determining one or more remote system conditions are satisfied at the remote system, wherein the one or more remote system conditions include one or more of: a particular time of day, a particular day of week, whether a threshold quantity of updates have been utilized to update the updated global ML model, or whether performance of the updated global ML model satisfies a performance threshold” which is a mental process that can be performed in the human mind using observation, evaluation, judgment and opinion .
Step 2A-Prong 2: Integrated into a practical application? No.
Claim 11 recites additional elements of “receiving the updated global ML model” which is simply data gathering step and therefore are insignificant extra-solution activities. (See MPEP 2106.05(g)).
Claim 11 depends on claim 10 which recites additional limitation of “receiving, at the client device and from the remote system, an updated global ML model, the updated global ML model including at least the one or more updated first global ML layers and the one or more updated second global ML layers; and replacing, in the on-device memory of the client device, the on-device ML model with the updated global ML model” which is simply data gathering step and therefore are insignificant extra-solution activities. (See MPEP 2106.05(g)).
Claim 11 depends on claim 10 which further depends on claim 9. Claim 9 recites the additional limitation of “wherein causing the remote system to update the global ML model further comprises causing the one or more second global ML layers to be updated based on a second update while one or more of the first global ML layers are fixed, and wherein the second update is transmitted to the remote system from an additional client device that is in addition to the client device utilized to generate the first update” which is pre/post solution activity which is insignificant extra-solution activity.
Claim 11 depends on claim 10 which further depends on claim 9 and further depends on claim 1. Claim 1 recites additional limitations :
“one or more processors of a client device” which amount to no more than apply the exception using generic computer components and computer.
“receiving, from a user of the client device, client data, the client data being generated locally at the client device the client device” is simply data gathering step and therefore are insignificant extra-solution activities. (See MPEP 2106.05(g));
“processing, using an on-device machine learning (ML) model stored locally in on-device memory of the client device, the client data to generate predicted output, wherein the on- device machine learning model includes a plurality of on-device ML layers, and wherein the plurality of on-device ML layers include at least one or more first on-device ML layers and one or more second on-device ML layers” is pre/post solution activity which is insignificant extra-solution activity; (See MPEP 2106.05(g)
“generating, using unsupervised learning, a gradient based on the predicted output; generating, based on the gradient, a first update for the one or more first on-device ML layers of the on-device ML model stored locally in the on-device memory of the client device” are pre/post solution activities which is insignificant extra-solution activities; (See MPEP 2106.05(g)
“transmitting the first update to a remote system”,
“wherein transmitting the first update to the remote system causes the remote system to update a global ML model stored remotely in remote memory of the remote system, wherein the global ML model includes at least one or more first global ML layers and one or more second global ML layers, and wherein causing the remote system to update the global ML model includes causing the one or more first global ML layers to be updated based on the first update while one or more of the second global ML layers are fixed” which is simply data gathering step and therefore are insignificant extra-solution activities. (See MPEP 2106.05(g)).
Step 2B: Recites additional elements that amount to significantly more than the judicial exception? No.
Claim 11 does not include additional elements that are sufficient to amount to significantly more than judicial exception. As indicates above, the additional elements of “one or more processor” and “client device” are at best equivalent of merely adding the word “apply it” to the judicial exception. The other remaining elements are insignificant extra-solution activities and well-understood, routine conventional activities previously known to the industry and therefore do not amount to significantly more than the judicial exception. (See MPEP 2106.05(d)) and 2106.07(a)III). Even when considered in combination, these additional element do not integrate the judicial exception into a practical application. Claim 11 therefore is ineligible.
Claim 12 recites additional element of “wherein receiving the updated global ML model is further in response to determining one or more client device conditions are satisfied at the client device, wherein the one or more client device conditions include one or more of: a particular time of day, a particular day of week, whether the client device is charging, whether the client device has at least a threshold state of charge, whether a temperature of the client device is less than a temperature threshold, or whether the client device is being held by a user” . The determining step is a mental process that can be performed in the human mind using observation, evaluation, judgment and opinion . Claim 12 does not recite any additional limitation that integrates the abstract idea into a practical application in step 2A-Prong 2 and provides an inventive concept in step 2B . Claim 12 is not patent eligible.
Claim 13 recites additional element of “ the updated global ML model that, when received, causes the client device to replace, in the on-device memory of the client device, the on-device ML model with the updated global ML model” which is pre/post solution activity and thus is insignificant extra-solution activity. (See MPEP 2106.05(g)). The insignificant extra solution activity does not provide an inventive concept. Accordingly, claim 13 is ineligible.
Claim 14 recites additional element of “ identifying a target portion of the client data, the target portion of the client data being subsequent to a prepended portion of the client data that is received prior to the target portion, and the target portion of the client data being prior to an appended portion of the client data that is received subsequent to the target portion” ; “masking the target portion of the client data” which are mental processes that can be performed in the human mind using observation, evaluation, judgment and opinion including with the help of pen and paper.
The additional element of “wherein processing the client data using the on-device ML model to generate the predicted output comprises processing the prepended portion of the client data and the appended portion of the client data to generate one or more of a predicted target portion of the client data that is predicted to correspond to the target portion of the client data” which is pre/post solution activity and thus is insignificant extra-solution activity. (See MPEP 2106.05(g)). The insignificant extra solution activity does not provide an inventive concept. Accordingly, claim 14 is ineligible.
Claim 15 recites additional element of “wherein generating the gradient based on the predicted output using unsupervised learning comprises: comparing the predicted target portion of the client data to the target portion of the client data; and generating the gradient based on comparing the predicted target portion to the target portion” which is a mental process that can be performed in the human mind using observation, evaluation, judgment and opinion . Claim 15 does not recite any additional limitation that integrates the abstract idea into a practical application in step 2A-Prong 2 and provides an inventive concept in step 2B . Claim 15 is not patent eligible.
Claim 16 recites additional element of “compressing the first on-device ML layer and the second on-device ML layer into the one or more first on-device ML layers, wherein the first update for the one or more on-device ML layers is a first shared update for the first on-device ML layer and the second on-device ML layer” which is a mental process that can be performed in the human mind using observation, evaluation, judgment and opinion . Claim 16 does not recite any additional limitation that integrates the abstract idea into a practical application in step 2A-Prong 2 and provides an inventive concept in step 2B . Claim 16 is not patent eligible.
Claim 17 recites additional element of “utilizing the first update to update a first global ML layer corresponding to the first on- device ML layer; and utilizing the first update to update a second global ML layer corresponding to the second on-device ML layer” which is pre/post solution activity and thus is insignificant extra-solution activity. (See MPEP 2106.05(g)). The insignificant extra solution activity does not provide an inventive concept. Accordingly, claim 17 is ineligible.
Claim 18 recites additional element of “wherein the one or more second on-device ML layers include at least the second on-device ML layer and a third on-device ML layer, wherein the second on- device ML layer and the third on-device ML layer are compressed at the additional client device into the one or more second on-device ML layers at the additional client device, and wherein a second update generated locally at the additional client device is a shared second update for the second on-device ML layer and the third on-device ML layer” are pre/post solution activities and thus are insignificant extra-solution activities. (See MPEP 2106.05(g)). The insignificant extra solution activities do not provide an inventive concept. Accordingly, claim 18 is ineligible.
Claim 19:
Step 2A-Prong 1: Judicial Exception Recited ?: Yes.
Claim 19 recites the limitation of “ in response to determining one or more conditions are satisfied” is a mental process that can be performed in the human mind using observation, evaluation, judgment and opinion .
Step 2A-Prong 2: Integrated into a practical application? No.
Claim 19 recites additional elements of “one or more processors of a remote system” which amount to no more than apply the exception using generic computer components and computer.
The additional elements of “receiving, from a client device of a user and at the remote system, a first update for a global machine learning (ML) model stored remotely at the remote system, wherein the global ML model includes a plurality of global ML layers, and wherein the first update for the global ML model is only for one or more first global ML layers, of the plurality of global ML layers, of the global ML model” is simply data gathering step and therefore are insignificant extra-solution activities. (See MPEP 2106.05(g));
The additional element of “receiving, from an additional client device of an additional user and at the remote system, a second update for the global ML model stored remotely in remote memory of the remote system, wherein the second update for the global ML model is only for one or more second global ML layers, of the plurality of global ML layers, of the global ML model, and wherein the one or more second global ML layers of the global ML model are distinct from the one or more first global ML layers of the global ML model” is simply data gathering step and therefore are insignificant extra-solution activities. (See MPEP 2106.05(g));
The additional element of “causing, based on at least the first update received from the client device and the second update received from the additional client device, the global ML model to be updated to generate an updated global ML model” which is pre/post solution activity and thus is insignificant extra-solution activity. (See MPEP 2106.05(g));
The additional element of “transmitting the updated global ML model to one or more of: the client device, the additional client device, or one or more further additional client devices” is simply data gathering step and therefore are insignificant extra-solution activities. (See MPEP 2106.05(g)).
Step 2B: Recites additional elements that amount to significantly more than the judicial exception? No.
Claim 19 does not include additional elements that are sufficient to amount to significantly more than judicial exception. As indicates above, the additional elements of “one or more processors of a remote system” are at best equivalent of merely adding the word “apply it” to the judicial exception. The remaining additional elements are insignificant extra-solution activities and well-understood, routine conventional activities previously known to the industry and therefore do not amount to significantly more than the judicial exception. (See MPEP 2106.05(d)) and 2106.07(a)III). Even when considered in combination, these additional elements do not integrate the judicial exception into a practical application. Claim 19 therefore is ineligible.
Allowable Subject Matter
Claims 14-15 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-10,13 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Yang et al.(US Patent Application Publication 2023/0038310 A1, hereinafter “Yang”) and further in view of Geraci et al.(US Patent Application Publication 2020/0327433 A1, hereinafter “Geraci”)
As to claim 1, Yang teaches a method implemented by one or more processors of a client device, the method comprising: receiving, from a user of the client device, client data, the client data being generated locally at the client device; (Yang par [0072] teaches local dataset 211)
processing, using an on-device machine learning (ML) model stored locally in on-device memory of the client device, the client data to generate predicted output (Yang par [0072] teaches client device 210 obtains model 100 from the server. It may then train the received model using local dataset 211), wherein the on- device machine learning model includes a plurality of on-device ML layers , and wherein the plurality of on-device ML layers include at least one or more first on-device ML layers and one or more second on-device ML layers (Yang par [0071] teaches the model 100 comprises set of common layers 120 and the set of client specific layers 140) ;
generating, based on the gradient, a first update for the one or more first on-device ML layers of the on-device ML model stored locally in the on-device memory of the client device; and transmitting the first update to a remote system,(Yang par [0075] teaches after the training of the model 100, the client device 210 sends the updated set of common layers 120 to the server computing device 220. Alternatively, the client device may only send parameters of the updated set of common layers 120 that have changed to the server 220)
wherein transmitting the first update to the remote system causes the remote system to update a global ML model stored remotely in remote memory of the remote system, (Yang par [0096] teaches the server 220 may receive an updated set of common layers 120 from each of the client devices 210 and 210’)
wherein the global ML model includes at least one or more first global ML layers and one or more second global ML layers (Yang Fig.2 and par [0071] teaches the model 100 comprises set of common layers 120 and the set of client specific layers 140), and wherein causing the remote system to update the global ML model includes causing the one or more first global ML layers to be updated based on the first update while one or more of the second global ML layers are fixed. (Yang par [0096] teaches the server 220 may receive an updated set of common layers 120 from each of the client devices 210 and 210’)
Yang fails to expressly teach generating, using unsupervised learning, a gradient based on the predicted output.
However, Geraci teaches generating, using unsupervised learning, a gradient based on the predicted output;(Geraci par [0035] teaches learning algorithm may include supervised ,unsupervised, semi-supervised and reinforcement learning. Geraci par [0163] teaches a gradient may be generated based on local model 701 finally refined after a predetermined number of steps is repeatedly performed)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Yang and Geraci to achieve the claimed invention. One would have been motivated to make such combination to reduce time consumed to refine the local model.(Geraci par [0065])
As to claim 2, Yang and Geraci teach the method of claim 1, wherein the first update transmitted to the remote system comprises the gradient and an indication of the one or more first global ML layers to be updated based on the first update, and wherein the one or more first global ML layers of the global ML model stored remotely at the remote system correspond to the one or more first on- device ML layers of the on-device ML model stored locally at the client device. (Yang Fig.2 and Yang par [0071] teaches the model 100 comprises set of common layers 120 and the set of client specific layers 140. Yang Fig.2 shows server 220 includes layers 120 and 140)
As to claim 3, Yang and Geraci teach the method of claim 2, wherein causing the one or more first global ML layers to be updated based on the first update while one or more of the second global ML layers are fixed ( Yang par [0096] teaches the server 220 may receive an updated set of common layers 120 from each of the client devices 210 and 210’) comprises: causing, based on the gradient and based on the indication of the one or more first global ML layers to be updated based on the first update, the one or more first global ML layers to be updated based on the gradient to generate one or more updated first global ML layers without updating the one or more second global ML layers , the one or more updated first global ML layers including one or more updated first global weights for the one or more updated first global ML layers. (Yang par [0096] teaches the server 220 may receive an updated set of common layers 120 from each of the client devices 210 and 210’. Yang par [0075] teaches the client device may only send parameters of the updated set of common layers 120 that have changed to the server 220)
As to claim 4, Yang and Geraci teach the method of claim 1, wherein generating the first update for the one or more first on- device ML layers comprises: causing the one or more first on-device ML layers to be updated based on the gradient to generate one or more updated first on-device ML layers without updating the one or more second on-device ML layers, the one or more updated first on-device ML layers including one or more updated first on-device weights for the one or more updated first on-device ML layers.( Yang par [0075] teaches the client device may only send parameters of the updated set of common layers 120 that have changed to the server 220)
As to claim 5, Yang and Geraci teach the method of claim 4, wherein the first update transmitted to the remote system comprises the one or more updated first on-device ML layers and an indication of the one or more first global ML layers to be updated based on the first update, and wherein the one or more first global ML layers of the global ML model stored remotely at the remote system correspond to the one or more first on-device ML layers of the on-device ML model stored locally at the client device. (Yang par [0075] teaches the client device may only send parameters of the updated set of common layers 120 that have changed to the server 220.)
As to claim 6, Yang and Geraci teach the method of claim 5, wherein causing the one or more first global ML layers to be updated based on the first update while one or more of the second global ML layers are fixed comprises: causing, based on the one or more updated first on-device ML layers and based on the indication of the one or more first global ML layers to be updated based on the first update, the one or more first global ML layers to be replaced in the remote memory with the one or more updated first on-device ML layers without replacing the one or more second global ML layers.( Yang par [0075] teaches the client device may only send parameters (weights and / or biases) of the updated set of common layers 120 that have changed to the server 220.Yang par [0071] teaches the model 100 comprises set of common layers 120 and the set of client specific layers 140)
As to claim 7, Yang and Geraci teach the method of claim 4, wherein the first update transmitted to the remote system comprises the one or more updated first on-device weights for the one or more updated first on-device ML layers and an indication of the one or more first global ML layers to be updated based on the first update (Yang par [0075] teaches the client device may only send parameters (weights and / or biases) of the updated set of common layers 120 that have changed to the server 220), and wherein the one or more first global ML layers of the global ML model stored remotely at the remote system correspond to the one or more first on-device ML layers of the on-device ML model stored locally at the client device. (Yang par [0071] teaches the model 100 comprises set of common layers 120 and the set of client specific layers 140)
As to claim 8, Yang and Geraci teach the method of claim 7, wherein causing the one or more first global ML layers to be updated based on the first update while one or more of the second global ML layers are fixed comprises:
causing, based on the one or more updated first on-device weights for the one or more updated first on-device ML layers and based on the indication of the one or more first global ML layers to be updated based on the first update, one or more first global weights for the one or more first global ML layers to be replaced in the remote memory of the remote system with the one or more updated first on-device weights for the one or more updated first on-device ML layers without replacing one or more second global weights for the one or more second global ML layers. (Yang par [0075] teaches the client device may only send parameters (weights and / or biases) of the updated set of common layers 120 that have changed to the server 220. Yang par [0096] teaches the server 220 may receive an updated set of common layers 120 from each of the client devices 210 and 210’)
As to claim 9, Yang and Geraci teach the method of claim 1, wherein causing the remote system to update the global ML model further comprises causing the one or more second global ML layers to be updated based on a second update while one or more of the first global ML layers are fixed, and wherein the second update is transmitted to the remote system from an additional client device that is in addition to the client device utilized to generate the first update. (Yang par [0098] teaches the server may aggregate the received updated sets of common layers 120)
As to claim 10, Yang and Geraci teach the method of claim 9, further comprising: receiving, at the client device and from the remote system, an updated global ML model, the updated global ML model including at least the one or more updated first global ML layers and the one or more updated second global ML layers; and replacing, in the on-device memory of the client device, the on-device ML model with the updated global ML model. ( Yang par [0098] teaches the server may aggregate the received updated sets of common layers 120 to obtain one aggregated set of common layers 120 . Then the server 220 may send the aggregated set of common layers 120 to each of the plurality of client devices 210, 210’)
As to claim 13, Yang and Geraci teach the method of claim 10, wherein the updated global ML model received at the client device and from the remote system comprises one or more of: the updated global ML model that, when received, causes the client device to replace, in the on-device memory of the client device, the on-device ML model with the updated global ML model; (Yang par [0098] teaches the server may aggregate the received updated sets of common layers 120 to obtain one aggregated set of common layers 120 . Then the server 220 may send the aggregated set of common layers 120 to each of the plurality of client devices 210, 210)
the one or more updated first global ML layers that, when received, causes the client device to replace, in the on-device memory of the client device, the one or more first on-device ML layers with the one or more updated first global ML layers; the one or more updated second global ML layers that, when received, causes the client device to replace, in the on-device memory of the client device, the one or more second on- device ML layers with the one or more updated second global ML layers; one or more updated first global weights for the one or more updated first global ML layers that, when received, causes the client device to replace, in the on-device memory of the client device, one or more first local weights for the one or more first on-device ML layers with the one or more updated first global weights; or one or more updated second global weights for the one or more updated second global ML layers that, when received, causes the client device to replace, in the on-device memory of the client device, one or more second local weights for the one or more second on-device ML layers with the one or more updated second global weights.
Claim 20 merely recites a system to perform the method of claim 1. Accordingly, Yang and Geraci teach every limitation of claim 20 as indicates in the above rejection of claim 1.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Yang, Geraci and further in view of Gyllenhammar et al.(US Patent Application Publication 2022/0161816 A1, hereinafter “Gyllenhammar”)
As to claim 11, Yang and Geraci teach the method of claim 10 but fail to teach wherein receiving the updated global ML model is in response to determining one or more remote system conditions are satisfied at the remote system, wherein the one or more remote system conditions include one or more of: a particular time of day, a particular day of week, whether a threshold quantity of updates have been utilized to update the updated global ML model, or whether performance of the updated global ML model satisfies a performance threshold.
However, Gyllenhammar teaches wherein receiving the updated global ML model is in response to determining one or more remote system conditions are satisfied at the remote system, wherein the one or more remote system conditions include one or more of: a particular time of day, a particular day of week, whether a threshold quantity of updates have been utilized to update the updated global ML model, or whether performance of the updated global ML model satisfies a performance threshold. (Gyllenhammar par [0169] teaches update is stored until an event triggers a global update. The trigger event includes time elapsed since the global autoencoder model was last retrained, or the number of updates , or the volume of data received , any autoencoder model parameter updates received from each vehicle are incorporated in the central or global autoencoder model)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Yang. Geraci and Gyllenhammar to achieve the claimed invention. One would have been motivated to make such combination to save computing resources.
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Yang, Geraci , Gyllenhammar and further in view of Kasaragod et al.(US Patent Application Publication 2019/0036716 A1, hereinafter “Kasaragod”)
As to claim 12, Yang , Geraci and Gyllenhammar teach the method of claim 11 but fail to teach wherein receiving the updated global ML model is further in response to determining one or more client device conditions are satisfied at the client device, wherein the one or more client device conditions include one or more of: a particular time of day, a particular day of week, whether the client device is charging, whether the client device has at least a threshold state of charge, whether a temperature of the client device is less than a temperature threshold, or whether the client device is being held by a user.
However, Kasaragod teaches wherein receiving the updated global ML model is further in response to determining one or more client device conditions are satisfied at the client device, wherein the one or more client device conditions include one or more of: a particular time of day, a particular day of week, whether the client device is charging, whether the client device has at least a threshold state of charge, whether a temperature of the client device is less than a temperature threshold, or whether the client device is being held by a user. (Kasaragod par [0106] teaches the model training service may generate and deploy one or more additional updates for one or more of the respective local data processing models to one or more respective edge devices on a periodic basis and/or in response to a triggering event)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Yang, Geraci, Gyllenhammar and Kasaragod to achieve the claimed invention. One would have been motivated to make such combination to reduce the update frequency, thus saving network bandwidth.
Claims 16-18 are rejected under 35 U.S.C. 103 as being unpatentable over Yang, Geraci and further in view of Chen et al. "Communication Efficient Federated Deep learning With layerwise asynchronous model update and temporarily weighted aggregation”; IEEE transactions on Neural Networks and Learning Systems, vol 31, no.10’ dated oct 2020. Hereinafter “Chen”.
As to claim 16, Yang and Geraci teach the method of claim 1, wherein the one or more first on-device ML layers include at least a first on-device ML layer and a second on-device ML layer (Yang par [0071] teaches the model 100 comprises set of common layers 120 and the set of client specific layers 140),
Yang and Geraci fail to teach the method further comprising: prior to generating the gradient based on the predicted output: compressing the first on-device ML layer and the second on-device ML layer into the one or more first on-device ML layers, wherein the first update for the one or more on-device ML layers is a first shared update for the first on-device ML layer and the second on-device ML layer.
However , Chen teaches compressing the first on-device ML layer and the second on-device ML layer into the one or more first on-device ML layers, wherein the first update for the one or more on-device ML layers is a first shared update for the first on-device ML layer and the second on-device ML layer. (Chen section 3.3 teaches in order to further reduce the communication overhead of federated learning, the model is compressed by pruning)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Yang , Geraci and Chen to achieve the claimed invention. One would have been motivated to make such combination to reduce communication overhead.(Chen section 3.3)
As to claim 17, Yang , Geraci and Chen teach method of claim 16, wherein causing the one or more first global ML layers to be updated based on the first update while one or more of the second global ML layers are fixed comprises: utilizing the first update to update a first global ML layer corresponding to the first on- device ML layer; and utilizing the first update to update a second global ML layer corresponding to the second on-device ML layer. (Yang par [0075] teaches the client device may only send parameters (weights and / or biases) of the updated set of common layers 120 that have changed to the server 220)
As to claim 18, Yang , Geraci and Chen method of claim 16, wherein the one or more second on-device ML layers include at least the second on-device ML layer and a third on-device ML layer (Yang par [0071] teaches the model 100 comprises set of common layers 120 and the set of client specific layers 140) , wherein the second on- device ML layer and the third on-device ML layer are compressed at the additional client device into the one or more second on-device ML layers at the additional client device, and wherein a second update generated locally at the additional client device is a shared second update for the second on-device ML layer and the third on-device ML layer. ( Chen section 3.3 teaches in order to further reduce the communication overhead of federated learning, the model is compressed by pruning)
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Yang and further in view of Xu et al., (US Patent Application Publication 2021/0295168 A1, hereinafter “Xu”)
As to claim 19, Yang teaches a method implemented by one or more processors of a remote system, the method comprising: receiving, from a client device of a user and at the remote system, a first update for a global machine learning (ML) model stored remotely at the remote system, wherein the global ML model includes a plurality of global ML layers, and [wherein the first update for the global ML model is only for one or more first global ML layers, of the plurality of global ML layers, of the global ML model]; (Yang par [0071] teaches the model 100 comprises set of common layers 120 and the set of client specific layers 140. Yang par [0096] teaches after the training of the model 100 is finished on each of the client computing device 210, 210’, the server 220 may receive an updated set of common layers 120 from each of the client computing devices 210, 210’. An updates of client-specific layers 140 may not be received)
receiving, from an additional client device of an additional user and at the remote system, a second update for the global ML model stored remotely in remote memory of the remote system, [wherein the second update for the global ML model is only for one or more second global ML layers, of the plurality of global ML layers, of the global ML model, and wherein the one or more second global ML layers of the global ML model are distinct from the one or more first global ML layers of the global ML model]; (Yang par [0071] teaches the model 100 comprises set of common layers 120 and the set of client specific layers 140. Yang par [0096] teaches after the training of the model 100 is finished on each of the client computing device 210, 210’, the server 220 may receive an updated set of common layers 120 from each of the client computing devices 210, 210’. An updates of client-specific layers 140 may not be received)
causing, based on at least the first update received from the client device and the second update received from the additional client device, the global ML model to be updated to generate an updated global ML model; and in response to determining one or more conditions are satisfied (Yang par [0098] teaches the server computing 220 may be further configured to aggregate the received updated sets of common layers 120. Then the server may send the aggregated set of common layers 120 to each of the plurality of client computing devices 210, 210’. Yang par [0126] teaches steps S502-S603 may be repeated multiple times , until a mathematical condition or criterion is fulfilled to achieve a final model 100 for performing a specific task of machine learning:
transmitting the updated global ML model to one or more of: the client device, the additional client device, or one or more further additional client devices. (Yang par [0098] teaches the server computing 220 may be further configured to aggregate the received updated sets of common layers 120. Then the server may send the aggregated set of common layers 120 to each of the plurality of client computing devices 210, 210’. Yang par [0125] teaches updating by the client device, the model based on the aggregated set of common layers)
Yang fails to expressly teach wherein the first update for the global ML model is only for one or more first global ML layers, of the plurality of global ML layers , of the global ML model; wherein the second update for the global ML model is only for one or more second global ML layers, of the plurality of global ML layers, of the global ML model, and wherein the one or more second global ML layers of the global ML model are distinct from the one or more first global ML layers of the global ML model.
However, Chen teaches wherein the first update for the global ML model is only for one or more first global ML layers, of the plurality of global ML layers, of the global ML model; (Xu par [0057] teaches as each of the gradients for each layer are computed and become available, they are transmitted from the worker node 120-1 to the worker node 120-4. For example, the gradients for layer 3 are computed first and are then transmitted to the worker node 120-4. Xu par [0058] teaches worker node 120-4 synchronizes the gradients received from the worker node 120-1 with other received gradients as they are received. For example, the gradients for layer 3 are received first and are begun to be synchronized before layers 1 and 2)
wherein the second update for the global ML model is only for one or more second global ML layers, of the plurality of global ML layers, of the global ML model (Xu par [0053] teaches as the gradients are computed by each of the worker nodes 120-1,120-2 and 120-3, the gradients area distributed to worker node 120-4 as shown in Fig.4A. Xu par [0057] teaches as each of the gradients for each layer are computed and become available, they are transmitted from the worker node 120-1 to the worker node 120-4. For example, the gradients for layer 3 are computed first and are then transmitted to the worker node 120-4. Xu par [0058] teaches worker node 120-4 synchronizes the gradients received from the worker node 120-1 with other received gradients as they are received. For example, the gradients for layer 3 are received first and are begun to be synchronized before layers 1 and 2., and
wherein the one or more second global ML layers of the global ML model are distinct from the one or more first global ML layers of the global ML model.(Xu par [0058 teaches layer 1,2 and 3)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Yang and Xu to achieve the claimed invention. One would have been motivated to make such combination to reduce the gradient exchange throughput requirement between nodes.(Xu par [0027[)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HIEN DUONG whose telephone number is (571)270-7335. The examiner can normally be reached Monday-Friday 8:00AM-5:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Viker Lamardo can be reached at 571-270-5871. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HIEN L DUONG/Primary Examiner, Art Unit 2147