Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
2. This Office Action is sent in response to Applicant’s Communication received on 08/13/2026 for application number 18/565,015.
Response to Amendments
3. The Amendment filed 08/13/2026 has been entered. Claims 17, 19-22, 24-28, 31, and 33-35 have been amended. Claims 17, 19-31, and 33-36 remain pending in the application.
Response to Arguments
Applicant argues that Vepakomma does not disclose the features as recited in amended claim 17. However, the argument is moot since this is a newly presented limitation, thus changes the scope of the claim. However, a newly found reference, Li, is applied.
Claim Rejections – 35 USC § 103
4. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
5. Claims 17, 19, 21-25, 27-31, 33, and 35-36 are rejected under 35 U.S.C. 103 as being unpatentable over Vepakomma et al. (U.S. Patent Application Pub. No. US 20200349443 A1) in view of Li et al. (DeepObfuscator: Obfuscating Intermediate Representations with Privacy-Preserving Adversarial Learning on Smartphones; arXiv, published 6 May 2021, pages 1-12).
Claim 17: Vepakomma teaches a data protection method (i.e. this invention is well-suited for use in any context in which it is important to protect the privacy of a client's raw data (or of at least certain attributes of the raw data) during distributed deep learning; para. [0058]), comprises:
obtaining original feature information of a target batch of reference samples for a passive party of a joint training model (i.e. a deep neural network (DNN) may be trained on raw data from one or more clients, without sharing the raw data with the server computer that is helping to perform the training. For instance, in split learning, each of the clients may store, or have access to, a confidential subset of raw data. For instance, a no-peek DNN may be trained to minimize the following loss function, and n is the number of samples of input data; para. [0004, 0005, 0029]); and
processing the original feature information by means of a target feature processing network structure (i.e. the DNN may comprise two portions: (a) layers of the DNN that are performed on a client computer (“client layers”); and (b) layers that are performed on a server computer (“server layers”). The client may take, as input, confidential raw data. The client may: (a) perform forward propagation through the client layers up to and including a layer we sometimes call the “split layer”; para. [0005]), the client layers are a neural network structure that processes the original/confidential input data before any information is shared with the server, to obtain target feature information corresponding to the original feature information (i.e. The client may encrypt the outputs of the activation functions (activation outputs) of the split layer, such as by RSA encryption; and (c) send the encrypted activation outputs to the server … transformed activation outputs … a client calculates decorrelated representations of raw data before sharing information about this data with external computers; para. [0005, 0011, 0020]), these outputs correspond to the original feature information because they are generated by forward propagation from that data,
wherein a neural network structure is trained by at least aiming at minimizing a coupling degree between original training feature information and target training feature information of training samples for the passive party to obtain the target feature processing network structure (i.e. leakage of information is reduced because the loss function of the DNN is selected in such a way as to reduce distance correlation between: (a) raw data; and (b) activation outputs at any given layer of the DNN. Distance correlation is a measure of linear and non-linear statistical dependency … The no-peek DNN may be trained to minimize this loss function. Put differently, the no-peek DNN may be trained to jointly minimize distance correlation (to reduce leakage of information) and categorical cross entropy (to ensure that the DNN makes accurate predictions); para. [0014, 0016]), and the target training feature information is feature information corresponding to the original training feature information that is outputted from the neural network structure using the original training feature information as an input of the neural network structure (i.e. The client may take, as input, confidential raw data. The client may: (a) perform forward propagation through the client layers up to and including a layer we sometimes call the “split layer”; (b) encrypt the outputs of the activation functions (activation outputs) of the split layer, such as by RSA encryption; and (c) send the encrypted activation outputs to the server; para. [0005, 0029]), the activation outputs are produced by processing the raw input data through neural network layers, they are corresponding transformed feature information,
wherein the target feature processing network structure is obtained by training through (i.e. a no-peek DNN may be trained to minimize a loss function; para. [0028]):
obtaining original training feature information of a specified batch of training samples for the passive party and target training feature information outputted by the neural network structure through processing on the original training feature information (i.e. a deep neural network (DNN) may be trained on raw data from one or more clients, without sharing the raw data with the server computer that is helping to perform the training. For instance, in split learning, each of the clients may store, or have access to, a confidential subset of raw data. For instance, a no-peek DNN may be trained to minimize the following loss function, and n is the number of samples of input data; para. [0004, 0005, 0029]);
obtaining target gradient information corresponding to a parameter of the neural network structure (i.e. the server may: (a) decrypt the activation outputs and feed these as input into a first layer of the server layers; (b) perform forward propagation through the server layers; (c) perform backpropagation through the server layers; (c) encrypt gradients for the first layer of the server layers (e.g., by RSA encryption); para. [0005]), wherein the target gradient information is determined based on a predetermined loss function and the target training feature information (i.e. a no-peek DNN may be trained to minimize a loss function, where the loss function of the DNN is a sum of a weighted distance correlation and a weighted categorical cross-entropy. Put differently, a no-peek DNN may be trained to jointly minimize distance correlation and categorical cross entropy; para. [0028, 0029, 0031]), and the predetermined loss function comprises a loss function characterizing a coupling degree between the original training feature information and the target training feature information (i.e. leakage of information is reduced because the loss function of the DNN is selected in such a way as to reduce distance correlation between: (a) raw data; and (b) activation outputs at any given layer of the DNN. Distance correlation is a measure of linear and non-linear statistical dependency; para. [0014, 0015, 0032]);
updating the parameter of the neural network structure based on the target gradient information (i.e. perform backpropagation through the server layers; (c) encrypt gradients for the first layer of the server layers (e.g., by RSA encryption); and (d) send the encrypted gradients to the client. Then the client may: (a) decrypt the gradients and feed them into the split layer; and (b) perform backpropagation through the client layers; para. [0005, 0006, 0007]);
determining whether training of the neural network structure is completed (i.e. This process may be iterated until the desired goal (of both accurate predictions and reduced leakage) is achieved; para. [0035, 0053]); and
obtaining the target feature processing network structure in response to completion of the training of the neural network structure (i.e. a neural network (NN) onboard a client network is trained to maximize the objective function in Equation 2, and then activation outputs of a final layer of this client NN are sent to a server to initialize a NN onboard the serve; para. [0053, 0054, 0055]).
Vepakomma does not explicitly teach wherein the target gradient information comprises at least one of adversarial reconstruction gradient information, and noise regularization gradient information, and the predetermined loss function correspondingly comprises at least one of an adversarial reconstruction loss function, and a noise regularization loss function; and said obtaining the target gradient information corresponding to the parameter of the neural network structure correspondingly comprises at least one of: determining first gradient information corresponding to the parameter of the neural network structure according to the original training feature information, first prediction feature information, and the adversarial reconstruction loss function, and determining gradient information obtained from processing of the first gradient information by a gradient reversal layer as the adversarial reconstruction gradient information, wherein the first prediction feature information is obtained by reconstructing based on the target training feature information, and the adversarial reconstruction loss function is a function characterizing a distance between the first prediction feature information and the original training feature information; and determining the noise regularization gradient information according to second prediction feature information, noise information, and the noise regularization loss function, wherein the second prediction feature information is obtained through reconstruction based on the target training feature information, the first prediction feature information is the same as or different from the second prediction feature information, and the noise regularization loss function is a function characterizing an error between the second prediction feature information and the noise information.
However, Li teaches wherein the target gradient information comprises at least one of adversarial reconstruction gradient information, and noise regularization gradient information, and the predetermined loss function correspondingly comprises at least one of an adversarial reconstruction loss function, and a noise regularization loss function (i.e. we generate one additional Gaussian noise image 𝐼𝑛𝑜𝑖𝑠𝑒. The adversary reconstructor is trained to make each reconstructed image similar to 𝐼𝑛𝑜𝑖𝑠𝑒 but different from 𝐼𝑜, and the performance of the classifier should be maintained. When playing as a defender, the obfuscator can be trained as: L(𝐴𝑅2) = 1−MS-SSIM(𝐼𝑛𝑜𝑖𝑠𝑒,𝐼𝑟), 𝜃𝑜 = argmin 𝜃𝑜 L(𝐶) +𝜆2(L(𝐴𝑅2) − L(𝐴𝑅1)), where 𝜆2 is a tradeoff parameter … Defend against AR: L(𝐶)+L(𝐴𝑅2)−L(𝐴𝑅1)→update𝑂(𝜃𝑜) … adopt the Adam Optimizer[13] with an adaptive learning rate in all four stages in the adversarial training procedure ; Section 3.4, 3.5, 4.1); and said obtaining the target gradient information corresponding to the parameter of the neural network structure correspondingly comprises at least one of:
determining first gradient information corresponding to the parameter of the neural network structure according to the original training feature information, first prediction feature information, and the adversarial reconstruction loss function, and determining gradient information obtained from processing of the first gradient information by a gradient reversal layer as the adversarial reconstruction gradient information, wherein the first prediction feature information is obtained by reconstructing based on the target training feature information, and the adversarial reconstruction loss function is a function characterizing a distance between the first prediction feature information and the original training feature information; and
determining the noise regularization gradient information according to second prediction feature information, noise information, and the noise regularization loss function (i.e. The adversary reconstructor (𝐴𝑅), which is trained to recover the raw image from the eavesdropped features, When playing as an attacker, the adversary reconstructor is trained to optimize the quality of the reconstructed image 𝐼𝑟 as close as the original image 𝐼𝑜. To this end, we generate one additional Gaussian noise image 𝐼𝑛𝑜𝑖𝑠𝑒. L(𝐴𝑅1) = 1−MS-SSIM(𝐼𝑜,𝐼𝑟); Section 3.4), wherein the second prediction feature information is obtained through reconstruction based on the target training feature information (i.e. The adversary reconstructor (𝐴𝑅), which is trained to recover the raw image from the eavesdropped features, also plays an attacker role. The attacker can apply any neural network architecture in the adversary reconstructor design; Section 3.4), the first prediction feature information is the same as or different from the second prediction feature information (i.e. L(𝐴𝑅1) = 1−MS-SSIM(𝐼𝑜,𝐼𝑟). L(𝐴𝑅2) = 1−MS-SSIM(𝐼𝑛𝑜𝑖𝑠𝑒,𝐼𝑟); Section 3.4 and fig. 3), and the noise regularization loss function is a function characterizing an error between the second prediction feature information and the noise information (i.e. The adversary reconstructor is trained to make each reconstructed image similar to 𝐼𝑛𝑜𝑖𝑠𝑒 but different from 𝐼𝑜, and the performance of the classifier should be maintained. When playing as a defender, the obfuscator can be trained as: L(𝐴𝑅2) = 1−MS-SSIM(𝐼𝑛𝑜𝑖𝑠𝑒,𝐼𝑟); Section 3.4).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Vepakomma to include the feature of Li. One would have been motivated to make this modification because it prevents reconstruction of original data from intermediate feature representations while retaining useful task information.
Claim 19: Vepakomma and Li teach the method according to claim 17. Vepakomma further teaches wherein the target gradient information further comprises distance correlation gradient information (i.e. leakage of information is reduced because the loss function of the DNN is selected in such a way as to reduce distance correlation between: (a) raw data; and (b) activation outputs at any given layer of the DNN. Distance correlation is a measure of linear and non-linear statistical dependency; para. [0014, 0016]), and the predetermined loss function correspondingly further comprises a distance correlation loss function (i.e. a distance correlation term may be added to the loss function of a DNN that performs split learning. This distance correlation term may be a measure of distance correlation between input data and activation outputs of the split layer of the DNN. We sometimes call such a DNN (which has a distance correlation term added to its loss function and which performs split learning) a “no-peek” DNN; para. [0015, 0029]); and said obtaining the target gradient information corresponding to the parameter of the neural network structure correspondingly further comprises: determining the distance correlation gradient information according to the original training feature information, the target training feature information, and the distance correlation loss function, wherein the distance correlation loss function is a function characterizing a distance correlation between the original training feature information and the target training feature information (i.e. a no-peek DNN may be trained to minimize the following loss function: α1DCOR(X, Z)+α2CCE(Y true , Y) where DCOR is distance correlation, CCE is categorical cross entropy, X is input data, Z is estimated activation outputs of an intermediate layer of the DNN, Ytrue is true labels for the input data, Y is predicted labels for the input data, α1 and α2 are scalar weights, and n is the number of samples of input data; para. [0029, 0031, 0032]).
Claim 21: Vepakomma and Li teach the method according to claim 17. Vepakomma further teaches wherein the target gradient information further comprises cross entropy gradient information (i.e. the loss function of a no-peek DNN is a sum of a weighted distance correlation and a weighted categorical cross-entropy. The no-peek DNN may be trained to minimize this loss function. Put differently, the no-peek DNN may be trained to jointly minimize distance correlation (to reduce leakage of information) and categorical cross entropy (to ensure that the DNN makes accurate predictions); para. [0016, 0029]), and the predetermined loss function further comprises a cross entropy loss function (i.e. To reduce this distance correlation (and thus leakage of information), a no-peek DNN may be trained to minimize a loss function, where the loss function of the DNN is a sum of a weighted distance correlation and a weighted categorical cross-entropy. Put differently, a no-peek DNN may be trained to jointly minimize distance correlation and categorical cross entropy; para. [0028, 0034]); and
said obtaining the target gradient information corresponding to the parameter of the neural network structure further comprises: transmitting the target training feature information to an active party of the joint training model (i.e. the client may: (a) perform forward propagation through the client layers up to and including a layer we sometimes call the “split layer”; (b) encrypt the outputs of the activation functions (activation outputs) of the split layer, such as by RSA encryption; and (c) send the encrypted activation outputs to the server. Then, the server may: (a) decrypt the activation outputs and feed these as input into a first layer of the server layers; (b) perform forward propagation through the server layers; (c) perform backpropagation through the server layers; (c) encrypt gradients for the first layer of the server layers (e.g., by RSA encryption); and (d) send the encrypted gradients to the client; para. [0005]), enabling the active party to perform label data prediction based on the target training feature information (i.e. Y is predicted labels for the input data, α1 and α2 are scalar weights, and n is the number of samples of input data; para. [0029, 0034]), and determining the cross entropy gradient information based on a label data prediction result and the cross entropy loss function (i.e. the categorical cross entropy term tends to reduce error in prediction. In Equation 1, CCE(Ytrue, Y) is categorical cross entropy between (a) true labels for the input data and (b) predicted labels for the input data. The categorical cross entropy may be calculated by computing the cross entropy of the output of a softmax function; para. [0034]), wherein the cross entropy loss function is a function characterizing cross entropy between the label data prediction result and real label data (i.e. the categorical cross entropy term tends to reduce error in prediction. In Equation 1, CCE(Ytrue, Y) is categorical cross entropy between (a) true labels for the input data and (b) predicted labels for the input data. The categorical cross entropy may be calculated by computing the cross entropy of the output of a softmax function; para. [0034]); and
receiving the cross entropy gradient information transmitted by the active party (i.e. the server may: (a) decrypt the activation outputs and feed these as input into a first layer of the server layers; (b) perform forward propagation through the server layers; (c) perform backpropagation through the server layers; (c) encrypt gradients for the first layer of the server layers (e.g., by RSA encryption); and (d) send the encrypted gradients to the client; para. [0005]).
Claim 22: Vepakomma and Li teach the method according to claim 21. Vepakomma further teaches wherein said determining whether the training of the neural network structure is completed comprises (i.e. In split learning, raw data from multiple clients may be employed to train the DNN. The raw data for each client may be different. The forward propagation (through client layers and then server layers) and backpropagation (through server layers and then client layers) may be repeated multiple times for each client during training of the DNN; para. [0006, 0028, 0035]): determining that the training of the neural network structure is completed in response to that a sum of a function value of the cross entropy loss function and a product of a target function value and a corresponding weight is minimum (i.e. the loss function of a no-peek DNN is a sum of a weighted distance correlation and a weighted categorical cross-entropy. The no-peek DNN may be trained to minimize this loss function. Put differently, the no-peek DNN may be trained to jointly minimize distance correlation (to reduce leakage of information) and categorical cross entropy (to ensure that the DNN makes accurate predictions) … a no-peek DNN may be trained to minimize the following loss function: α1DCOR(X, Z)+α2CCE(Y true , Y) (Equation 1) where DCOR is distance correlation, CCE is categorical cross entropy, X is input data, Z is estimated activation outputs of an intermediate layer of the DNN, Ytrue is true labels for the input data, Y is predicted labels for the input data, α1 and α2 are scalar weights, and n is the number of samples of input data; para. [0016, 0028, 0029]), wherein the target function value comprises at least one of a function value of a distance correlation loss function, a function value of the adversarial reconstruction loss function, and a function value of the noise regularization loss function (i.e. In order to reduce distance correlation, a distance correlation term may be added to the loss function of a DNN that performs split learning. This distance correlation term may be a measure of distance correlation between input data and activation outputs of the split layer of the DNN. We sometimes call such a DNN (which has a distance correlation term added to its loss function and which performs split learning) a “no-peek” DNN; para. [0015, 0032]).
Claim 23: Vepakomma and Li teach the method according to claim 17. Vepakomma further teaches comprising: transmitting the target feature information to an active party of the joint training model, so as for the active party to determine gradient transmission information of a parameter of the joint training model based on the target feature information (i.e. the client may: (a) perform forward propagation through the client layers up to and including a layer we sometimes call the “split layer”; (b) encrypt the outputs of the activation functions (activation outputs) of the split layer, such as by RSA encryption; and (c) send the encrypted activation outputs to the server. Then, the server may: (a) decrypt the activation outputs and feed these as input into a first layer of the server layers; (b) perform forward propagation through the server layers; (c) perform backpropagation through the server layers; (c) encrypt gradients for the first layer of the server layers (e.g., by RSA encryption); and (d) send the encrypted gradients to the client; para. [0005]); and
receiving gradient transmission information transmitted by the active party, and updating the parameter of the joint training model according to the gradient transmission information (i.e. the server may: (a) decrypt the activation outputs and feed these as input into a first layer of the server layers; (b) perform forward propagation through the server layers; (c) perform backpropagation through the server layers; (c) encrypt gradients for the first layer of the server layers (e.g., by RSA encryption); and (d) send the encrypted gradients to the client. Then the client may: (a) decrypt the gradients and feed them into the split layer; and (b) perform backpropagation through the client layers; para. [0005]).
Claim 24: Vepakomma teaches a training method for a feature processing network structure (i.e. To reduce this distance correlation (and thus leakage of information), a no-peek DNN may be trained to minimize a loss function; para. [0028, 0054]), comprising:
obtaining original training feature information of a specified batch of training samples for a passive party of a joint training model (i.e. a deep neural network (DNN) may be trained on raw data from one or more clients, without sharing the raw data with the server computer that is helping to perform the training. For instance, in split learning, each of the clients may store, or have access to, a confidential subset of raw data. For instance, a no-peek DNN may be trained to minimize the following loss function, and n is the number of samples of input data; para. [0004, 0005, 0029]), the client in split learning has confidential raw data used to train the DNN, and target training feature information outputted by a neural network structure through processing the original training feature information (i.e. the DNN may comprise two portions: (a) layers of the DNN that are performed on a client computer (“client layers”); and (b) layers that are performed on a server computer (“server layers”). The client may take, as input, confidential raw data. The client may: (a) perform forward propagation through the client layers up to and including a layer we sometimes call the “split layer”; (b) encrypt the outputs of the activation functions (activation outputs) of the split layer, such as by RSA encryption; and (c) send the encrypted activation outputs to the server; para. [0005, 0029, 0030]), the client processes raw/input data through client layers and outputs activation outputs;
obtaining target gradient information corresponding to a parameter of the neural network structure (i.e. the server may: (a) decrypt the activation outputs and feed these as input into a first layer of the server layers; (b) perform forward propagation through the server layers; (c) perform backpropagation through the server layers; (c) encrypt gradients for the first layer of the server layers (e.g., by RSA encryption); and (d) send the encrypted gradients to the client. Then the client may: (a) decrypt the gradients and feed them into the split layer; and (b) perform backpropagation through the client layers; para. [0005]), gradient information through its forward/backpropagation training process, wherein the target gradient information is determined according to a predetermined loss function and the target training feature information (i.e. a no-peek DNN may be trained to minimize a loss function, where the loss function of the DNN is a sum of a weighted distance correlation and a weighted categorical cross-entropy. Put differently, a no-peek DNN may be trained to jointly minimize distance correlation and categorical cross entropy, loss function: α1DCOR(X, Z)+α2CCE(Y true , Y) (Equation 1) where DCOR is distance correlation, CCE is categorical cross entropy, X is input data, Z is estimated activation outputs of an intermediate layer of the DNN, a computer employs the Pytorch autograd software package in order to minimize the loss function set forth in Equation 1; para. [0028, 0029, 0031]), the gradients are generated during training to minimize a predetermined loss function that includes the activation output Z, and the predetermined loss function comprises a loss function characterizing a coupling degree between the original training feature information and the target training feature information (i.e. leakage of information is reduced because the loss function of the DNN is selected in such a way as to reduce distance correlation between: (a) raw data; and (b) activation outputs at any given layer of the DNN. Distance correlation is a measure of linear and non-linear statistical dependency; para. [0014, 0015, 0032]);
updating the parameter of the neural network structure based on the target gradient information (i.e. perform backpropagation through the server layers; (c) encrypt gradients for the first layer of the server layers (e.g., by RSA encryption); and (d) send the encrypted gradients to the client. Then the client may: (a) decrypt the gradients and feed them into the split layer; and (b) perform backpropagation through the client layers; para. [0005, 0006, 0007]), updating neural network parameters by backpropagation, wherein the neural network structure is trained by at least aiming at minimizing the coupling degree between the original training feature information and the target training feature information (i.e. The no-peek DNN may be trained to minimize this loss function. Put differently, the no-peek DNN may be trained to jointly minimize distance correlation (to reduce leakage of information) and categorical cross entropy (to ensure that the DNN makes accurate predictions); para. [0016, 0028, 0037]);
determining whether training of the neural network structure is completed (i.e. This process may be iterated until the desired goal (of both accurate predictions and reduced leakage) is achieved; para. [0006, 0035]); and
obtaining a target feature processing network structure in response to completion of the training of the neural network structure (i.e. a neural network (NN) onboard a client network is trained to maximize the objective function in Equation 2, and then activation outputs of a final layer of this client NN are sent to a server to initialize a NN onboard the serve; para. [0053, 0054, 0055]), obtaining the trained feature-processing structure after training because the learned activations, weights, and biases are used after training to initialize or operate another neural network structure.
Vepakomma does not explicitly teach wherein the target gradient information comprises at least one of adversarial reconstruction gradient information, and noise regularization gradient information, and the predetermined loss function correspondingly comprises at least one of an adversarial reconstruction loss function, and a noise regularization loss function; and said obtaining the target gradient information corresponding to the parameter of the neural network structure correspondingly comprises at least one of: determining first gradient information corresponding to the parameter of the neural network structure according to the original training feature information, first prediction feature information, and the adversarial reconstruction loss function, and determining gradient information obtained from processing of the first gradient information by a gradient reversal layer as the adversarial reconstruction gradient information, wherein the first prediction feature information is obtained by reconstructing based on the target training feature information, and the adversarial reconstruction loss function is a function characterizing a distance between the first prediction feature information and the original training feature information; and determining the noise regularization gradient information according to second prediction feature information, noise information, and the noise regularization loss function, wherein the second prediction feature information is obtained through reconstruction based on the target training feature information, the first prediction feature information is the same as or different from the second prediction feature information, and the noise regularization loss function is a function characterizing an error between the second prediction feature information and the noise information.
However, Li teaches wherein the target gradient information comprises at least one of adversarial reconstruction gradient information, and noise regularization gradient information, and the predetermined loss function correspondingly comprises at least one of an adversarial reconstruction loss function, and a noise regularization loss function (i.e. we generate one additional Gaussian noise image 𝐼𝑛𝑜𝑖𝑠𝑒. The adversary reconstructor is trained to make each reconstructed image similar to 𝐼𝑛𝑜𝑖𝑠𝑒 but different from 𝐼𝑜, and the performance of the classifier should be maintained. When playing as a defender, the obfuscator can be trained as: L(𝐴𝑅2) = 1−MS-SSIM(𝐼𝑛𝑜𝑖𝑠𝑒,𝐼𝑟), 𝜃𝑜 = argmin 𝜃𝑜 L(𝐶) +𝜆2(L(𝐴𝑅2) − L(𝐴𝑅1)), where 𝜆2 is a tradeoff parameter … Defend against AR: L(𝐶)+L(𝐴𝑅2)−L(𝐴𝑅1)→update𝑂(𝜃𝑜) … adopt the Adam Optimizer[13] with an adaptive learning rate in all four stages in the adversarial training procedure ; Section 3.4, 3.5, 4.1); and said obtaining the target gradient information corresponding to the parameter of the neural network structure correspondingly comprises at least one of:
determining first gradient information corresponding to the parameter of the neural network structure according to the original training feature information, first prediction feature information, and the adversarial reconstruction loss function, and determining gradient information obtained from processing of the first gradient information by a gradient reversal layer as the adversarial reconstruction gradient information, wherein the first prediction feature information is obtained by reconstructing based on the target training feature information, and the adversarial reconstruction loss function is a function characterizing a distance between the first prediction feature information and the original training feature information; and
determining the noise regularization gradient information according to second prediction feature information, noise information, and the noise regularization loss function (i.e. The adversary reconstructor (𝐴𝑅), which is trained to recover the raw image from the eavesdropped features, When playing as an attacker, the adversary reconstructor is trained to optimize the quality of the reconstructed image 𝐼𝑟 as close as the original image 𝐼𝑜. To this end, we generate one additional Gaussian noise image 𝐼𝑛𝑜𝑖𝑠𝑒. L(𝐴𝑅1) = 1−MS-SSIM(𝐼𝑜,𝐼𝑟); Section 3.4), wherein the second prediction feature information is obtained through reconstruction based on the target training feature information (i.e. The adversary reconstructor (𝐴𝑅), which is trained to recover the raw image from the eavesdropped features, also plays an attacker role. The attacker can apply any neural network architecture in the adversary reconstructor design; Section 3.4), the first prediction feature information is the same as or different from the second prediction feature information (i.e. L(𝐴𝑅1) = 1−MS-SSIM(𝐼𝑜,𝐼𝑟). L(𝐴𝑅2) = 1−MS-SSIM(𝐼𝑛𝑜𝑖𝑠𝑒,𝐼𝑟); Section 3.4 and fig. 3), and the noise regularization loss function is a function characterizing an error between the second prediction feature information and the noise information (i.e. The adversary reconstructor is trained to make each reconstructed image similar to 𝐼𝑛𝑜𝑖𝑠𝑒 but different from 𝐼𝑜, and the performance of the classifier should be maintained. When playing as a defender, the obfuscator can be trained as: L(𝐴𝑅2) = 1−MS-SSIM(𝐼𝑛𝑜𝑖𝑠𝑒,𝐼𝑟); Section 3.4).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Vepakomma to include the feature of Li. One would have been motivated to make this modification because it prevents reconstruction of original data from intermediate feature representations while retaining useful task information.
Claims 25, 27, and 28 are similar in scope to Claims 19, 21, 22 and are rejected under a similar rationale.
Claim 29: Vepakomma teaches a non-transitory computer-readable medium, having a computer program stored thereon, wherein the program, when executed by a processor, implements steps (i.e. a machine-accessible medium has instructions encoded thereon that specify steps in a software program; and (b) the computer accesses the instructions encoded on the machine-accessible medium, in order to determine steps to execute in the program; para. [0073, 0074]) of the method according to claim 17 (see rejection of claim 17 above).
Claim 30: Vepakomma teaches a non-transitory computer-readable medium, having a computer program stored thereon, wherein the program, when executed by a processor, implements steps (i.e. a machine-accessible medium has instructions encoded thereon that specify steps in a software program; and (b) the computer accesses the instructions encoded on the machine-accessible medium, in order to determine steps to execute in the program; para. [0073, 0074]) of the method according to claim 24 (see rejection of claim 24 above).
Claims 31, 33 and 35 are similar in scope to Claims 17, 19, 21 and are rejected under a similar rationale.
Claim 36: Vepakomma teaches an electronic device, comprising: a memory having a computer program stored thereon; and a processor configured to execute the computer program in the memory to implement the training method for a feature processing network structure (i.e. one or more computers are programmed to perform any and all calculations, computations, programs, algorithms, computer functions and computer tasks described or implied herein. For example, in some cases: (a) a machine-accessible medium has instructions encoded thereon that specify steps in a software program; and (b) the computer accesses the instructions encoded on the machine-accessible medium, in order to determine steps to execute in the program. In exemplary implementations, the machine-accessible medium may comprise a tangible non-transitory medium. In some cases, the machine-accessible medium comprises (a) a memory unit or (b) an auxiliary memory storage device. For example, in some cases, a control unit in a computer fetches the instructions from memory; para. [0074]) according to claim 24 (see rejection of claim 24 above).
Claim Rejections – 35 USC § 103
6. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
7. Claims 20, 26, and 34 are rejected under 35 U.S.C. 103 as being unpatentable over Vepakomma in view of Li, and further in view of Sohn et al. (U.S. Patent Application Pub. No. US 20200143079 A1).
Claim 20: Vepakomma and Li teach the method according to claim 17. Vepakomm does not explicitly teach wherein the first prediction feature information is feature information outputted from a feature reconstruction network structure obtained by inputting the target training feature information into the feature reconstruction network structure; and in a case where the target gradient information comprises the adversarial reconstruction gradient information and the predetermined loss function comprises the adversarial reconstruction loss function, a training process of the target feature processing network structure further comprises: determining second gradient information corresponding to a parameter of the feature reconstruction network structure based on the original training feature information, the first prediction feature information, and the adversarial reconstruction loss function; and returning the second gradient information to the feature reconstruction network structure, so as for the feature reconstruction network structure to update the parameter of the feature reconstruction network structure based on the second gradient information.
However, Sohn teaches wherein the first prediction feature information is feature information outputted from a feature reconstruction network structure (i.e. Given a paired dataset {(Xϵχ2, Z)}, the adversary inverts the feature via a decoder Deca:
PNG
media_image1.png
38
33
media_image1.png
Greyscale
→X, which is trained to reconstruct the input X from the feature Z by minimizing the reconstruction loss; para. [0023, 0024]) obtained by inputting the target training feature information into the feature reconstruction network structure (i.e. This is a generous setting to the adversary since the adversary can create a large-scale paired database of input Xϵχ2 and the feature Z=Enc(X); para. [0021, 0023, 0045]); and
in a case where the target gradient information comprises the adversarial reconstruction gradient information and the predetermined loss function comprises the adversarial reconstruction loss function, a training process of the target feature processing network structure further comprises (i.e. the exemplary embodiments consider a model inversion (MI) attack based on the neural decoder that is trained to reconstruct the data using data-feature pairs. Perceptual and generative adversarial network (GAN) losses are employed to improve the generation quality. Finally, the exemplary embodiments identify adversarial data reconstruction learning that involves an alternate training of an encoder and a decoder. While the decoder, simulating the adversary's attack, is trained to reconstruct input from the feature, the encoder is trained to maximize the reconstruction error to prevent the decoder from inverting features while minimizing the task loss; para. [0017, 0034]):
determining second gradient information corresponding to a parameter of the feature reconstruction network structure based on the original training feature information, the first prediction feature information, and the adversarial reconstruction loss function (i.e. the exemplary embodiments employ adversarial learning. Specifically, the decoder (Dec) is trained to compete against an Enc in a way that it learns to decode Z of the current Enc into X by minimizing the reconstruction loss:
PNG
media_image2.png
38
29
media_image2.png
Greyscale
p=
PNG
media_image3.png
38
25
media_image3.png
Greyscale
{(Xϵχ 1 ,Z)}[∥Dec(Z)−X∥ 2]; para. [0034]); and
returning the second gradient information to the feature reconstruction network structure, so as for the feature reconstruction network structure to update the parameter of the feature reconstruction network structure based on the second gradient information (i.e. FIGS. 4 and 5, described in further detail below, depict the main modules of the exemplary embodiments. The exemplary embodiments adopt an alternative update strategy for the proposed learning framework. The Deca and D are updated first on the public data χ2 according to (5) whiling fixing the Enc and f, and Enc and f are updated on the private data by (9) and so forth until convergence; para. [0038, 0047]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to modify the combination of Vepakomma and Li to include the feature of Sohn. One would have been motivated to make this modification because it provides privacy protection by directly training the feature representation against reconstruction attacks.
Claims 26 and 34 are similar in scope to Claim 20 and are rejected under a similar rationale.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure.
Sun et al. (Pub. No. US 20220383054 A1), a related method generally adopts a joint training model to complete joint training of a machine learning model on the premise of not exposing original data. For a supervised machine learning model, a party with a sample label is generally referred to as an active party, and a party without a sample label is referred to as a passive party. A sample label owned by the active party is one of important data needed to be protected in the joint training model.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
It is noted that any citation to specific pages, columns, lines, or figures in the prior art references and any interpretation of the references should not be considered to be limiting in any way. A reference is relevant for all it contains and may be relied upon for all that it would have reasonably suggested to one having ordinary skill in the art. In re Heck, 699 F.2d 1331, 1332-33, 216 U.S.P.Q. 1038, 1039 (Fed. Cir. 1983) (quoting In re Lemelson, 397 F.2d 1006, 1009, 158 U.S.P.Q. 275, 277 (C.C.P.A. 1968)).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TAN TRAN whose telephone number is (303)297-4266. The examiner can normally be reached on Monday - Thursday - 8:00 am - 5:00 pm MT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matt Ell can be reached on 571-270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TAN H TRAN/Primary Examiner, Art Unit 2141
Prosecution Insights