Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 1-20 rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
MPEP 2106 (III) sets out steps for evaluating whether a claim is drawn to patent-eligible subject matter. The analysis of claims 1-20, in accordance with these steps, follows.
Step 1 Analysis:
Claims 1-11 are directed to method (processes). Claim 12-20 are directed to an apparatus (machine). Therefore, claims 1-20 fall into one of four statutory categories (i.e., process, machine, article of manufacture).
As to claim 1,
Step 2A Prong 1: this claim recites the following abstract ideas:
determining, by using a prediction result of an object, update parameters associated with the object feature data, wherein the update parameters are used to update model parameters and comprise a plurality of sub-parameters for the plurality of computational layers; (the limitation describes calculating adjustment/update values from a result, which is a mental process implemented using a pen and paper.)
dividing the plurality of computational layers into first-type computational layers and second-type computational layers by using the plurality of sub-parameters, wherein sub-parameter values of the first-type computational layers fall within a specified range, and sub-parameter values of the second-type computational layers fall outside the specified range; (the limitation describes sorting/categorizing items based on whether their values fall within a range, which is an evaluation and judgment activity that can be performed as a mental process in the human mind.)
Step 2A Prong 2 and 2B: the claim recited the following additional elements:
A method for data privacy-preserving training of a service prediction model, wherein a plurality of member devices perform joint training, the service prediction model comprises a plurality of computational layers, and the method is performed by a member device; (This limitation is directed to mere instruction to apply the abstract idea on a generic computer to process, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i))
performing prediction by using the service prediction model and object feature data that are of a plurality of objects and that are held by the member device; (This limitation is directed to mere instruction to apply the abstract idea on a generic computer to process, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i))
performing privacy processing on the sub-parameters of the first-type computational layers; (This limitation is directed to mere instruction to apply the abstract idea on a generic computer to process, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i))
outputting processed sub-parameters of the first-type computational layers; (this limitation describes data transmission, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i))
obtaining aggregated sub-parameters of the first-type computational layers, wherein the aggregated sub-parameters are obtained through aggregation based on processed sub-parameters of at least two member devices and are associated with object feature data of the at least two member devices; and (this limitation describes data collection/receiving, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i))
updating the model parameters by using the aggregated sub-parameters of the first-type computational layers and sub-parameters of the second-type computational layers. (This limitation is directed to mere instruction to apply the abstract idea on a generic computer to process, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i))
The additional element does not integrate the judicial exception into practical application and does not amount to significantly more than the Judicial exception.
As to claim 2,
Step 2A Prong 1: this claim recites the following abstract ideas:
the update parameters are implemented based on a model parameter gradient or a model parameter difference, and the model parameter gradient is determined based on a prediction loss obtained in current training; (the limitation describes determining an adjustment value based on a loss value, which is a mental process implemented using a pen and paper.)
updating the initial model parameters by using the model parameter gradient to obtain simulated update parameters; and (the limitation describes calculating updated values from a gradient, which is a mental process implemented using a pen and paper.)
determining the model parameter difference based on a difference between the initial model parameters and the simulated update parameters. (the limitation describes calculating a difference between two values, which is a mental process implemented using a pen and paper.)
Step 2A Prong 2 and 2B: the claim recited the following additional elements:
obtaining initial model parameters in current training and a model parameter gradient obtained in current training; (this limitation describes data collection/receiving, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i))
The additional element does not integrate the judicial exception into practical application and does not amount to significantly more than the Judicial exception.
As to claim 3,
Step 2A Prong 1: this claim recites the following abstract ideas:
determining a plurality of sub-parameter representation values corresponding respectively to the plurality of sub-parameters by using vector elements comprised in the sub-parameters, wherein a sub-parameter representation value is used to represent a value of a corresponding sub-parameter; and (the limitation describes computing a representative value from data elements, which is a mental process implemented using a pen and paper.)
dividing the plurality of computational layers into the first-type computational layers and the second-type computational layers by using the plurality of sub-parameter representation values. (the limitation describes sorting/categorizing items based on their values, which is an evaluation and judgment activity that can be performed as a mental process in the human mind.)
Step 2A Prong 2 and 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B. Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) 1.), failing step 2A prong 2. The claim is ineligible.
As to claim 4,
Step 2A Prong 1: this claim recites the following abstract ideas:
the sub-parameter representation value is implemented by using one of a norm value, a mean value, a variance value, a standard deviation value, a maximum value, a minimum value, or a difference between a maximum value and a minimum value. (the limitation describes computing a statistical value such as a mean, variance, or maximum from data, which is a mental process implemented using a pen and paper.)
Step 2A Prong 2 and 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B. Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) 1.), failing step 2A prong 2. The claim is ineligible.
As to claim 5,
Step 2A Prong 1: this claim recites the following abstract ideas:
the sub-parameter representation values of the first-type computational layers are greater than the sub-parameter representation values of the second-type computational layers. (the limitation describes comparing two values to each other, which is a mental process implemented in the human mind.)
Step 2A Prong 2 and 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B. Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) 1.), failing step 2A prong 2. The claim is ineligible.
As to claim 6,
Step 2A Prong 1: this claim recites the following abstract ideas:
the sub-parameter values of the first-type computational layers fall within a specified range comprises that orders of magnitude of the plurality of sub-parameter values fall within a predetermined magnitude range. (the limitation describes comparing the magnitude of values against a predetermined range, which is a mental process implemented in the human mind.)
Step 2A Prong 2 and 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B. Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) 1.), failing step 2A prong 2. The claim is ineligible.
As to claim 7,
Step 2A Prong 1: This claim does not recite an additional abstract idea, but the claim depends on claim 1.
Step 2A Prong 2 and 2B: the claim recited the following additional elements:
determining noise data for the sub-parameters of the first-type computational layers based on an (ϵ, δ)-differential privacy algorithm; and (This limitation is directed to mere instruction to apply the abstract idea on a generic computer to process, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i))
separately combining the noise data with corresponding sub-parameters of the first-type computational layers to obtain corresponding processed sub-parameters. (This limitation is directed to mere instruction to apply the abstract idea on a generic computer to process, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i))
The additional element does not integrate the judicial exception into practical application and does not amount to significantly more than the Judicial exception.
As to claim 8,
Step 2A Prong 1: This claim does not recite an additional abstract idea, but the claim depends on claim 7.
Step 2A Prong 2 and 2B: the claim recited the following additional elements:
calculating a noise variance value of Gaussian noise by using differential privacy parameters ϵ and δ; and (This limitation is directed to mere instruction to apply the abstract idea on a generic computer to process, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i))
generating, based on the noise variance value, corresponding noise data for vector elements comprised in the sub-parameters of the first-type computational layers. (This limitation is directed to mere instruction to apply the abstract idea on a generic computer to process, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i))
The additional element does not integrate the judicial exception into practical application and does not amount to significantly more than the Judicial exception.
As to claim 9,
Step 2A Prong 1: this claim recites the following abstract ideas:
determining, by using several sub-parameters corresponding to the first-type computational layers, an overall representation value used to identify the sub-parameters of the first-type computational layers; and (the limitation describes computing an overall representative value from data, which is a mental process implemented using a pen and paper.)
performing numerical clipping on the sub-parameters of the first-type computational layers by using the overall representation value and a predetermined clipping parameter to obtain corresponding clipped sub-parameters of the first-type computational layers; and (the limitation describes bounding/limiting values according to a parameter, which is a mental process implemented using a pen and paper.)
Step 2A Prong 2 and 2B: the claim recited the following additional elements:
separately combining the noise data with the corresponding clipped sub-parameters of the first-type computational layers. (This limitation is directed to mere instruction to apply the abstract idea on a generic computer to process, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i))
The additional element does not integrate the judicial exception into practical application and does not amount to significantly more than the Judicial exception.
As to claim 10,
Step 2A Prong 1: this claim recites the following abstract ideas:
performing numerical clipping on the sub-parameters of the second-type computational layers by using the overall representation value and the predetermined clipping parameter to obtain corresponding clipped sub-parameters of the second-type computational layers; and (the limitation describes bounding/limiting values according to a parameter, which is a mental process implemented using a pen and paper.)
Step 2A Prong 2 and 2B: the claim recited the following additional elements:
updating the model parameters by using the aggregated sub-parameters and the corresponding clipped sub-parameters of the second-type computational layers. (This limitation is directed to mere instruction to apply the abstract idea on a generic computer to process, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i))
The additional element does not integrate the judicial exception into practical application and does not amount to significantly more than the Judicial exception.
As to claim 11,
Step 2A Prong 1: this claim recites the following abstract ideas:
determining, by the plurality of member devices using a prediction result of the object, update parameters associated with the object feature data, wherein the update parameters are used to update model parameters and comprise a plurality of sub-parameters for the plurality of computational layers; (the limitation describes calculating adjustment/update values from a result, which is a mental process implemented using a pen and paper.)
separately dividing, by the plurality of member devices, the plurality of computational layers into first-type computational layers and second-type computational layers by using the plurality of sub-parameters, wherein sub-parameter values of the first-type computational layers fall within a specified range, and sub-parameter values of the second-type computational layers fall outside the specified range; (the limitation describes sorting/categorizing items based on whether their values fall within a range, which is an evaluation and judgment activity that can be performed as a mental process in the human mind.)
performing, by the server, aggregation for the computational layers based on processed sub-parameters sent by at least two member devices to obtain aggregated sub-parameters corresponding respectively to the first-type computational layers; and (the limitation describes combining/aggregating sets of values, which is a mental process implemented using a pen and paper.)
Step 2A Prong 2 and 2B: the claim recited the following additional elements:
A method for data privacy-preserving training of a service prediction model, wherein a server and a plurality of member devices perform joint training, the service prediction model comprises a plurality of computational layers; (This limitation is directed to mere instruction to apply the abstract idea on a generic computer to process, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i))
separately performing, by the plurality of member devices, prediction by using the service prediction model and object feature data that are of a plurality of objects and that are respectively held by the plurality of member devices; (This limitation is directed to mere instruction to apply the abstract idea on a generic computer to process, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i))
separately performing, by the plurality of member devices, privacy processing on the sub-parameters of the first-type computational layers; (This limitation is directed to mere instruction to apply the abstract idea on a generic computer to process, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i))
separately sending, by the plurality of member devices, obtained processed sub-parameters to the server; (this limitation describes data transmission, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i))
sending the aggregated sub-parameters to a corresponding member device; and (this limitation describes data transmission, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i))
separately receiving, by the plurality of member devices, the aggregated sub-parameters sent by the server; and (this limitation describes data collection/receiving, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i))
updating, by the plurality of member devices, the model parameters by using the aggregated sub-parameters and sub-parameters of the second-type computational layers. (This limitation is directed to mere instruction to apply the abstract idea on a generic computer to process, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i))
The additional element does not integrate the judicial exception into practical application and does not amount to significantly more than the Judicial exception.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Wei et al. (Federated Learning With Differential Privacy: Algorithms and Performance Analysis, 2020) in view of Li et al. (US 12072954 B1) and Liu et al. (FedSel: Federated SGD under Local Differential Privacy with Top-k Dimension Selection, 24 March 2020).
As to claim 1, Wei teaches:
A method for data privacy-preserving training of a service prediction model, (see Wei section [Abstract] “Federated learning (FL), as a type of distributed machine learning, is capable of significantly preserving clients' private data from being exposed to adversaries.”)
wherein a plurality of member devices perform joint training, (see Wei section [II A] “consider a general FL system consisting of one server and N clients.”)
the service prediction model comprises a plurality of computational layers, (see Wei section [VI] “Our baseline model uses a MLP network with a single hidden layer containing 256 hidden units.”)
Examiner note: an MLP with a hidden layer plus the softmax output layer constitutes a plurality of computational layers.
and the method is performed by a member device and comprises: (see Wei section [II A] “An active client, participating in the local training, needs to find a vector w of an AI model to minimize a certain loss function.”)
performing prediction by using the service prediction model and object feature data that are of a plurality of objects and that are held by the member device; (see Wei section [II A] “Let D_i denote the local database held by the client C_i.”, and see Wei section [VI] “each client has 100 training samples locally.”)
Examiner note: the locally-held training samples in D_i are the plural “objects,” and local training runs the model on that object feature data.
determining, by using a prediction result of an object, update parameters associated with the object feature data, (see Wei section [II A] “All active clients locally compute training gradients or parameters.”)
Examiner note: the gradient is computed from the loss evaluated on the client's data, i.e., by using the model's prediction result for that data.
wherein the update parameters are used to update model parameters (see Wei section [III A] “Algorithm 1 … w(t)i = arg min{w_i} (F_i(w_i) + (μ/2)||w_i − w(t−1)||²).”)
Examiner note: the SGD update derived from the gradient produces the updated model parameters w(t)_i from w(t−1).
and comprise a plurality of sub-parameters for the plurality of computational layers; (see Wei section [II A] “w_i is the parameter vector trained at the i-th client.”)
Examiner note: the trained vector taken layer-by-layer supplies the per-layer sub-parameters.
performing privacy processing on the sub-parameters of the first-type computational layers; (see Wei section [III A] “Algorithm 1… Add noise and upload parameters w(t)_i = w(t)_i + n(t)_i.”)
outputting processed sub-parameters of the first-type computational layers; (see Wei section [III B] “upload the noised parameters w(t)_i to the server for aggregation.”)
obtaining aggregated sub-parameters of the first-type computational layers, (see Wei section [III B] “the server update the global parameters w(t) by aggregating the local parameters integrated with different weights.”)
wherein the aggregated sub-parameters are obtained through aggregation based on processed sub-parameters of at least two member devices (see Wei section [II A] eq. (1) “w = Σ_{i=1}^{N} p_i w_i.”)
Examiner note: the summation over N clients (N≥2) aggregates the processed parameters of at least two member devices.
and are associated with object feature data of the at least two member devices; and (see Wei section [II A] eq. (1) “w = Σ_{i=1}^{N} p_i w_i.”, and see Wei section [II A] “w_i is the parameter vector trained at the i-th client.”
Examiner note: because the aggregate sums each w_i trained on that client's local data D_i, it embodies the contributing devices' object feature data.
updating the model parameters by using the aggregated sub-parameters of the first-type computational layers (see Wei section [II A] “All clients update their respective models with the aggregated parameters and test the performance of the updated models.”)
Wei does not explicitly teaches "dividing the plurality of computational layers into first-type computational layers and second-type computational layers by using the plurality of sub-parameters", "wherein sub-parameter values of the first-type computational layers fall within a specified range", "and sub-parameter values of the second-type computational layers fall outside the specified range", and "and sub-parameters of the second-type computational layers"
However, Li, in view of Liu, teaches:
dividing the plurality of computational layers into first-type computational layers and second-type computational layers by using the plurality of sub-parameters, (see Li [Col 8 L 6] “a selective parameter sharing process might include replacing a component, w.sub.i, of a model ΔW.sub.k(t) if and only if abs(w.sub.i), an absolute value of w.sub.i, is greater than a threshold, τ.sub.k.sup.(t)).”, and see Liu section [1] “we privately select Top-k dimensions according to their contributions (i.e., the absolute values of gradients).”)
wherein sub-parameter values of the first-type computational layers fall within a specified range, (see Li [Col 8 L 8] “if and only if abs(w.sub.i), an absolute value of w.sub.i, is greater than a threshold, τ.sub.k.sup.(t)).”)
and sub-parameter values of the second-type computational layers fall outside the specified range; (see Li [Col 5 L 43] “less than all of a local model might be transmitted to a central server.”)
Examiner note: parameters that do not clear the threshold are the out-of-range, un-transmitted (second-type) set.
and sub-parameters of the second-type computational layers. (See Li [Col 5 L 45] “some elements of a local model are reset at a start of a subsequent training round.”, and see Li claim [9] “combining the one or more portions of weight values from the one or more first neural networks with one or more portions of weight values from the one or more second neural networks.”
Examiner note: the retained, un-shared local parameters supply the second-type sub-parameters used in the update, since Wei alone shares all parameters.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the federated-learning method of Wei to incorporate the magnitude-threshold division of sub-parameters into a shared (first-type) set and an un-shared (second-type) set taught by Li and Liu, in order to reduce communication overhead by transmitting only a sparse subset of the local parameters rather than Wei's full parameter vector (see Li [Col 9 L 2] "a sparse property of ΔW(t)k might be used in reducing communication overheads."; and see Li [Col 5 L 43] "less than all of a local model might be transmitted to a central server."), to further enhance privacy by limiting which parameters leave the device and thereby reducing the ability to reconstruct the local training data (see Li [Col 5 L 40] "privacy of local training data can be enhanced by limiting what aspects or portions of a local model that is transmitted to a central server."), and to improve model accuracy under a fixed privacy budget by concentrating the differential-privacy cost on the most significant, highest-magnitude parameters, since not all dimensions contribute equally (see Li [Col 9 L 63] "sharing fewer parameters can provide reduced overall DP costs and thus better model performance."; and see Liu section [1] "we privately select Top-k dimensions according to their contributions (i.e., the absolute values of gradients)."), and one of ordinary skill would have had a reasonable expectation of success because Wei, Li, and Liu are all directed to the same field of privacy-preserving federated SGD in which clients locally train a model and upload parameters to a server for aggregation (see Wei section [II A]; see Li [Col 4 L 21]; and see Liu section [2.1]).
As to claim 2, Wei as modified by Li and Liu teaches the method according to claim 1,
wherein the update parameters are implemented based on a model parameter gradient or a model parameter difference, (see Wei section [I] “distributed stochastic gradient descent (SGD) is adopted in FL for training ML models.”)
and the model parameter gradient is determined based on a prediction loss obtained in current training; and (see Wei section [IV] “F_i(w) is ρ-Lipschitz smooth, i.e., ||∇F_i(w) − ∇F_i(w')|| ≤ ρ||w − w'||.”)
Examiner note: ∇F_i is the gradient of the local loss function F_i computed in the current round.
Wei does not explicitly teaches "the model parameter difference is determined according to the following method: obtaining initial model parameters in current training and a model parameter gradient obtained in current training", "updating the initial model parameters by using the model parameter gradient to obtain simulated update parameters; and", and "determining the model parameter difference based on a difference between the initial model parameters and the simulated update parameters"
However, Li teaches
the model parameter difference is determined according to the following method: obtaining initial model parameters in current training and a model parameter gradient obtained in current training; (see Li [Col 4 L 45] “a shared local model comprises local model stochastic gradient descent (SGD) updates.”, and see Li [Col 6 L 41] “setting an initial local model for neural network 520 and initializing momentum parameters related to gradients of SGD updates.”, and see Li [Col 7 L 22] “a momentum-based SGD that takes previous SGD steps into account when computing a current SGD step.”)
Examiner note: the “initial local model” is the initial model parameters of the current round, and the "current SGD step" is the model parameter gradient obtained in current training.
updating the initial model parameters by using the model parameter gradient to obtain simulated update parameters; and (see Li [Col 3 L 46] “updates neural network (NN) parameters 206, such as neural network node weights.”, and see Li [Col 6 L 44] “at step 502, neural network 520 is trained on an image set 522 … thereby updating a local model 524.”)
Examiner note: training the initial local model with the SGD gradient yields the updated local model 524, which supplies the recited "simulated update parameters" (the locally-computed updated-parameter vector used to derive the difference, before the global model is loaded back).
determining the model parameter difference based on a difference between the initial model parameters and the simulated update parameters. (see Li [Col 5 L 38] “share with a central server local model trainable weights and/or local model updates.”, and see Li [Col 8 L 7] “replacing a component, w.sub.i, of a model ΔW.sub.k(t) if and only if abs(w.sub.i) … is greater than a threshold.”, and see Li [Col 9 L 1] “when partial model sharing is used, a sparse property of ΔW(t)k might be used in reducing communication overheads.”
Examiner note: the quantity Li computes and shares is the local model update ΔW_k^(t), a delta between the round's initial model and the trained (updated) local model, which is the recited model parameter difference.
As to claim 3, Wei as modified by Li and Liu teaches the method according to claim 1,
wherein the dividing the plurality of computational layers into first-type computational layers and second-type computational layers comprises: (see Li [Col 8 L 6] “a selective parameter sharing process might include replacing a component, w.sub.i, of a model ΔW.sub.k(t) if and only if abs(w.sub.i), an absolute value of w.sub.i, is greater than a threshold, τ.sub.k.sup.(t)).”, and see Liu section [1] “we privately select Top-k dimensions according to their contributions (i.e., the absolute values of gradients).”)
determining a plurality of sub-parameter representation values corresponding respectively to the plurality of sub-parameters by using vector elements comprised in the sub-parameters, (see Li [Col 8 L 7] “replacing a component, w.sub.i, of a model ΔW.sub.k(t) if and only if abs(w.sub.i), an absolute value of w.sub.i, is greater than a threshold, τ.sub.k.sup.(t)).”, and see Liu section [1] “we privately select Top-k dimensions according to their contributions (i.e., the absolute values of gradients).”)
Examiner note: the absolute value computed from each gradient/component is the scalar representation value characterizing that sub-parameter.
wherein a sub-parameter representation value is used to represent a value of a corresponding sub-parameter; and (see Liu section [4.1] "the dimension with the largest magnitude of its absolute value should be output with the highest probability.")
Examiner note: the absolute-value magnitude stands in for, and represents, the value of the corresponding sub-parameter.
dividing the plurality of computational layers into the first-type computational layers and the second-type computational layers by using the plurality of sub-parameter representation values. (see Li [Col 8 L 8] “if and only if abs(w.sub.i), an absolute value of w.sub.i, is greater than a threshold, τ.sub.k.sup.(t)).”, and see Liu Algorithm 3 [Page 18] “sort |r^t| and for j ∈ [d], set z_j ← 1 if |r^t_j| ∈ Top-k.”)
As to claim 4, Wei as modified by Li and Liu teaches the method according to claim 3,
wherein the sub-parameter representation value is implemented by using one of a norm value, a mean value, a variance value, a standard deviation value, a maximum value, a minimum value, or a difference between a maximum value and a minimum value. (see Liu section [4.1] “the dimension with the largest magnitude of its absolute value should be output with the highest probability.”, and see Li paragraph [Col 8 L 8] “abs(w.sub.i), an absolute value of w.sub.i.”)
As to claim 5, Wei as modified by Li and Liu teaches the method according to claim 3,
wherein the sub-parameter representation values of the first-type computational layers are greater than the sub-parameter representation values of the second-type computational layers. (see Liu [4.1] “the dimension with the largest magnitude of its absolute value should be output with the highest probability.”, and see Li [Col 8 L 8] “if and only if abs(w.sub.i) … is greater than a threshold.”)
Examiner note: the selected/shared (first-type) values are by construction the larger-magnitude ones, leaving the smaller as second-type.
As to claim 6, Wei as modified by Li and Liu teaches the method according to claim 1,
wherein the sub-parameter values of the first-type computational layers fall within a specified range comprises that orders of magnitude of the plurality of sub-parameter values fall within a predetermined magnitude range. (see Li [Col 8 L 8] “if and only if abs(w.sub.i), an absolute value of w.sub.i, is greater than a threshold, τ.sub.k.sup.(t)).”, and see Liu section [1] “we privately select Top-k dimensions according to their contributions (i.e., the absolute values of gradients).”)
As to claim 7, Wei as modified by Li and Liu teaches the method according to claim 1,
wherein the performing privacy processing on sub-parameters of the first-type computational layers comprises: determining noise data for the sub-parameters of the first-type computational layers based on an (ϵ, δ)-differential privacy algorithm; and (see Wei Def. 1 “A randomized mechanism M : X → R … satisfies (ϵ, δ)-DP, if … Pr[M(D_i) ∈ S] ≤ e^ϵ Pr[M(D'_i) ∈ S] + δ.”, and see Wei section [II C] “a Gaussian mechanism defined in [24] can be used to guarantee (ϵ, δ)-DP.”)
separately combining the noise data with corresponding sub-parameters of the first-type computational layers to obtain corresponding processed sub-parameters. (see Wei [III] Algorithm 1… Add noise and upload parameters w(t)_i = w(t)_i + n(t)_i.”)
As to claim 8, Wei as modified by Li and Liu teaches the method according to claim 7,
wherein the determining noise data for the sub-parameters of the first-type computational layers comprises: (see Wei Def. 1 “A randomized mechanism M : X → R … satisfies (ϵ, δ)-DP, if … Pr[M(D_i) ∈ S] ≤ e^ϵ Pr[M(D'_i) ∈ S] + δ.”, and see Wei section [II C] “a Gaussian mechanism defined in [24] can be used to guarantee (ϵ, δ)-DP.”)
calculating a noise variance value of Gaussian noise by using differential privacy parameters ϵ and δ; and (see Wei section [II C] “we choose noise scale σ ≥ cΔs/ϵ and the constant c ≥ √(2 ln(1.25/δ)) for ϵ ∈ (0, 1).”)
generating, based on the noise variance value, corresponding noise data for vector elements comprised in the sub-parameters of the first-type computational layers. (see Wei section [II C] “n is the value of an additive noise sample for a data in the dateset.”, and see Wei section [III] “Algorithm 1 …. w(t)_i = w(t)_i + n(t)_i.”
Examiner note: a Gaussian sample drawn at the computed scale is generated per parameter element and added.)
As to claim 9, Wei as modified by Li and Liu teaches the method according to claim 7,
before the separately combining the noise data with corresponding sub-parameters of the first-type computational layers, further comprising: (see Wei [III] Algorithm 1… Add noise and upload parameters w(t)_i = w(t)_i + n(t)_i.”)
determining, by using several sub-parameters corresponding to the first-type computational layers, an overall representation value used to identify the sub-parameters of the first-type computational layers; and (see Wei section [III A] “C is a clipping threshold for bounding ||w_i||.”)
Examiner note: the parameter-vector norm ||w_i|| is the overall representation value used, together with C, to characterize the set for clipping.
performing numerical clipping on the sub-parameters of the first-type computational layers by using the overall representation value and a predetermined clipping parameter to obtain corresponding clipped sub-parameters of the first-type computational layers; and (see Wei section [III] “Algorithm 1 … Clip the local parameters w(t)_i = w(t)_i / max(1, ||w(t)_i||/C).”)
Examiner note: C is the predetermined clipping parameter and ||w(t)_i|| the overall representation value in the clip ratio.
wherein the separately combining the noise data with corresponding sub-parameters of the first-type computational layers comprises: separately combining the noise data with the corresponding clipped sub-parameters of the first-type computational layers. (see Wei “Algorithm 1… Clip the local parameters w(t)_i = w(t)_i / max(1, ||w(t)_i||/C) … Add noise and upload parameters w(t)_i = w(t)_i + n(t)_i.”)
Examiner note: Wei's clip-then-add-noise order matches combining the noise with the clipped sub-parameters.
As to claim 10, Wei as modified by Li and Liu teaches the method according to claim 9,
wherein the updating the model parameters comprises: performing numerical clipping on the sub-parameters of the second-type computational layers by using the overall representation value and the predetermined clipping parameter to obtain corresponding clipped sub-parameters of the second-type computational layers; and (see Wei section [III] “Algorithm 1 … Clip the local parameters w(t)_i = w(t)_i / max(1, ||w(t)_i||/C).”, and see Wei section [III] “using a clipping technique, we can ensure that ∥ w_i ∥ ≤ C, where w_i denotes training parameters from the i-th client without perturbation and C is a clipping threshold for bounding w_i”)
updating the model parameters by using the aggregated sub-parameters and the corresponding clipped sub-parameters of the second-type computational layers. (See Wei section [II A] “All clients update their respective models with the aggregated parameters.”, and see Li claim [9] “combining the one or more portions of weight values from the one or more first neural networks with one or more portions of weight values from the one or more second neural networks.”)
As to claim 11, Wei teaches:
A method for data privacy-preserving training of a service prediction model, (see Wei section [Abstract] “Federated learning (FL), as a type of distributed machine learning, is capable of significantly preserving clients' private data from being exposed to adversaries.”)
wherein a server and a plurality of member devices perform joint training, (see Wei section [II A] “consider a general FL system consisting of one server and N clients.”)
the service prediction model comprises a plurality of computational layers, and the method comprises: (see Wei section [II A] “An active client, participating in the local training, needs to find a vector w of an AI model to minimize a certain loss function.”, and see Wei section [VI] “Our baseline model uses a MLP network with a single hidden layer containing 256 hidden units.”)
Examiner note: an MLP with a hidden layer plus the softmax output layer constitutes a plurality of computational layers.
separately performing, by the plurality of member devices, prediction by using the service prediction model and object feature data that are of a plurality of objects and that are respectively held by the plurality of member devices; (see Wei section [II A] “Let D_i denote the local database held by the client C_i.”, and see Wei section [VI] “each client has 100 training samples locally.”)
Examiner note: the locally-held training samples in D_i are the plural “objects,” and local training runs the model on that object feature data.
determining, by the plurality of member devices using a prediction result of the object, update parameters associated with the object feature data, (see Wei section [II A] “All active clients locally compute training gradients or parameters.”)
Examiner note: the gradient is computed from the loss evaluated on the client's data, i.e., by using the model's prediction result for that data.
wherein the update parameters are used to update model parameters (see Wei section [III A] “Algorithm 1 … w(t)i = arg min{w_i} (F_i(w_i) + (μ/2)||w_i − w(t−1)||²).”)
Examiner note: the SGD update derived from the gradient produces the updated model parameters w(t)_i from w(t−1).
and comprise a plurality of sub-parameters for the plurality of computational layers; (see Wei section [II A] “w_i is the parameter vector trained at the i-th client.”)
Examiner note: the trained vector taken layer-by-layer supplies the per-layer sub-parameters.
separately performing, by the plurality of member devices, privacy processing on the sub-parameters of the first-type computational layers; (see Wei section [III A] “Algorithm 1… Add noise and upload parameters w(t)_i = w(t)_i + n(t)_i.”)
separately sending, by the plurality of member devices, obtained processed sub-parameters to the server; (see Wei section [III B] “upload the noised parameters w(t)_i to the server for aggregation.”)
performing, by the server, aggregation for the computational layers based on processed sub-parameters sent by at least two member devices to obtain aggregated sub-parameters corresponding respectively to the first-type computational layers; and (see Wei [II A] “The server performs secure aggregation over the uploaded parameters from N clients without learning local information.”, and see Wei eq. [1] “w = Σ_{i=1}^{N} p_i w_i.”)
sending the aggregated sub-parameters to a corresponding member device; and (see Wei [II A] “The server broadcasts the aggregated parameters to the N clients.”)
separately receiving, by the plurality of member devices, the aggregated sub-parameters sent by the server; and (see Wei [III.B] “Based on the received global parameters w(t), each client will estimate the accuracy by using local testing databases.”)
Examiner note: the clients' use of the received global parameters w(t) shows they receive the server-sent aggregate.
updating, by the plurality of member devices, the model parameters by using the aggregated sub-parameters and sub-parameters of the second-type computational layers. (See Wei [II A] “All clients update their respective models with the aggregated parameters and test the performance of the updated models.”)
Wei does not explicitly teaches "separately dividing, by the plurality of member devices, the plurality of computational layers into first-type computational layers and second-type computational layers by using the plurality of sub-parameters", "wherein sub-parameter values of the first-type computational layers fall within a specified range", "and sub-parameter values of the second-type computational layers fall outside the specified range"
However, Li, in view of Liu, teaches:
separately dividing, by the plurality of member devices, the plurality of computational layers into first-type computational layers and second-type computational layers by using the plurality of sub-parameters, (see Li [Col 8 L 6] “a selective parameter sharing process might include replacing a component, w.sub.i, of a model ΔW.sub.k(t) if and only if abs(w.sub.i), an absolute value of w.sub.i, is greater than a threshold, τ.sub.k.sup.(t)).”, and see Liu section [1] “we privately select Top-k dimensions according to their contributions (i.e., the absolute values of gradients).”)
wherein sub-parameter values of the first-type computational layers fall within a specified range, (see Li [Col 8 L 8] “if and only if abs(w.sub.i), an absolute value of w.sub.i, is greater than a threshold, τ.sub.k.sup.(t)).”)
and sub-parameter values of the second-type computational layers fall outside the specified range; (see Li [Col 5 L 43] “less than all of a local model might be transmitted to a central server.”)
Examiner note: parameters that do not clear the threshold are the out-of-range, un-transmitted (second-type) set.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the federated-learning method of Wei to incorporate the magnitude-threshold division of sub-parameters into a shared (first-type) set and an un-shared (second-type) set taught by Li and Liu, in order to reduce communication overhead by transmitting only a sparse subset of the local parameters rather than Wei's full parameter vector (see Li [Col 9 L 2] "a sparse property of ΔW(t)k might be used in reducing communication overheads."; and see Li [Col 5 L 43] "less than all of a local model might be transmitted to a central server."), to further enhance privacy by limiting which parameters leave the device and thereby reducing the ability to reconstruct the local training data (see Li [Col 5 L 40] "privacy of local training data can be enhanced by limiting what aspects or portions of a local model that is transmitted to a central server."), and to improve model accuracy under a fixed privacy budget by concentrating the differential-privacy cost on the most significant, highest-magnitude parameters, since not all dimensions contribute equally (see Li [Col 9 L 63] "sharing fewer parameters can provide reduced overall DP costs and thus better model performance."; and see Liu section [1] "we privately select Top-k dimensions according to their contributions (i.e., the absolute values of gradients)."), and one of ordinary skill would have had a reasonable expectation of success because Wei, Li, and Liu are all directed to the same field of privacy-preserving federated SGD in which clients locally train a model and upload parameters to a server for aggregation (see Wei section [II A]; see Li [Col 4 L 21]; and see Liu section [2.1]).
As to claim 12, this is directed to an apparatus claim that corresponds to method claim 1. See the rejection for claim 1 above, which also applies to claim 12. Additionally the claim recites
one or more computers; and (see Li [Col 5 L 53] "this can be a hard process for a computer system to perform")
one or more computer memory devices interoperably coupled with the one or more computers and having tangible, non-transitory, machine-readable media storing one or more instructions that, when executed by the one or more computers, perform one or more operations comprising: (see Li [Col 101 L 1] " a computer-readable storage medium is a non-transitory computer-readable storage medium that excludes transitory signals (e.g., a propagating transient electric or electromagnetic transmission) but includes non-transitory data storage circuitry (e.g., buffers, cache, and queues) within transceivers of transitory signals. In at least one embodiment, code (e.g., executable code or source code) is stored on a set of one or more non-transitory computer-readable storage media having stored thereon executable instructions (or other memory to store executable instructions) that, when executed (i.e., as a result of being executed) by one or more processors of a computer system, cause computer system to perform operations described herein. In at least one embodiment, set of non-transitory computer-readable storage media comprises multiple non-transitory computer-readable storage media and one or more of individual non-transitory storage media of multiple non-transitory computer-readable storage media lack all of code while multiple non-transitory computer-readable storage media collectively store all of code.")
As to claim 13, this is directed to an apparatus claim that corresponds to method claim 2. See the rejection for claim 2 above, which also applies to claim 13.
As to claim 14, this is directed to an apparatus claim that corresponds to method claim 3. See the rejection for claim 3 above, which also applies to claim 14.
As to claim 15, this is directed to an apparatus claim that corresponds to method claim 4. See the rejection for claim 4 above, which also applies to claim 15.
As to claim 16, this is directed to an apparatus claim that corresponds to method claim 5. See the rejection for claim 5 above, which also applies to claim 16.
As to claim 17, this is directed to an apparatus claim that corresponds to method claim 6. See the rejection for claim 6 above, which also applies to claim 17.
As to claim 18, this is directed to an apparatus claim that corresponds to method claim 7. See the rejection for claim 7 above, which also applies to claim 18.
As to claim 19, this is directed to an apparatus claim that corresponds to method claim 8. See the rejection for claim 8 above, which also applies to claim 19.
As to claim 20, this is directed to an apparatus claim that corresponds to method claim 9. See the rejection for claim 9 above, which also applies to claim 20.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ABDULLAH K ABOUD whose telephone number is (571)272-0025. The examiner can normally be reached Mon-Fri 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Li B Zhen, can be reached at (571) 272-3768. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ABDULLAH KHALED ABOUD/Examiner, Art Unit 2121
/Li B. Zhen/Supervisory Patent Examiner, Art Unit 2121
Prosecution Insights