DETAILED ACTION
Claims 1-30 are presented for examination
This office action is in response to submission of application on 16-SEPTEMBER-2022.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The amendment filed on 30-MARCH-2026 in response to the non-final office action mailed 05-FEBURARY-2026 has been entered. Claims 1-30 remain pending in the application.
With regards to the 101 rejection, the rejection to claim 1 has not been overcome by the applicant’s amendments and arguments. The applicant’s arguments and remarks do not overcome the 101 rejection set forth and as such the rejection has been maintained.
With regards to the 103 rejections, the applicant’s amendments to the claims have not overcome the rejections to claims 1-30 as the former prior art sufficiently teaches the newly added limitations of the amended claims.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-8 and 15-30 are rejected under 35 U.S.C. 101 because the claimed invention is directed to
an abstract idea (Abstract Idea) without significantly more.
Regarding claim 1, in Step 1 of the 101 analysis set forth in MPEP 2106, the claim recites a method for executing a processor method. A method is one of the four statutory categories of invention.
In Step 2a Prong 1 of the 101 analysis set forth in the MPEP 2106, the examiner has determined
that the following limitations recite a process that, under the broadest reasonable interpretation, covers
a mental process but for recitation of generic computer components:
refining one or more weights associated with a second set of layers of the neural network using the set of intermediate activations; (one can mentally refine data as a process of simply evaluating the activations and making a judgement on the evaluations)
If claim limitations, under their broadest reasonable interpretation, covers performance of the limitations as a mental process but for the recitation of generic computer components, then it falls within the mental process grouping of abstract ideas. According, the claim “recites” an abstract idea.
In Step 2a Prong 2 of the 101 analysis set forth in MPEP 2106, the examiner has determined that the following additional elements do not integrate this judicial exception into a practical application:
A processor-implemented method of federated learning surrogation, comprising: (Generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05(h))
receiving a set of intermediate activations at a trusted server from a node device, wherein the node device generated the set of intermediate activations using a first set of layers of a neural network; (Adding insignificant extra-solution activity (mere data gathering) to the judicial exception (MPEP 2106.05(g))
and transmitting one or more weight updates corresponding to the refined one or more weights from the trusted server to a federated learning server. (Adding insignificant extra-solution activity (mere data gathering) to the judicial exception (MPEP 2106.05(g))
Since the claim does not contain any other additional elements that are indicative of integration into a practical application, the claim is “directed” to an abstract idea.
In Step 2b of the 101 analysis set forth in the 2019 PEG, the examiner has determined that the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above, the additional element (ii) recites generally linking the use of the judicial exception to a particular technological environment or field of use, and (iii) and (iv) recites adding insignificant extra-solution activity, which is not indicative of significantly more. Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible
Regarding claim 2, it is dependent upon claim 1, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 2 recites The processor-implemented method of Claim 1, further comprising receiving, from the federated learning system, an updated version of the neural network, wherein the updated version of the neural network was generated based at least in part on the one or more weight updates. (Adding insignificant extra-solution activity (mere data gathering) to the judicial exception (MPEP 2106.05(g)), which is not indicative of integration into a practical application. In step 2B, merely applying a computer tool is not indicative of significantly more.)Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible.
Regarding claim 2, it is dependent upon claim 1, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 2 recites The processor-implemented method of Claim 1, further comprising receiving, from the federated learning system, an updated version of the neural network, wherein the updated version of the neural network was generated based at least in part on the one or more weight updates. (Adding insignificant extra-solution activity (mere data gathering) to the judicial exception (MPEP 2106.05(g)), which is not indicative of integration into a practical application. In step 2B, merely applying a computer tool is not indicative of significantly more.)Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible.
Regarding claim 3, it is dependent upon claim 1, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 3 recites The processor-implemented method of Claim 1, wherein the received set of intermediate activations is encrypted, (Generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05(h)) the method further comprising, prior to refining the one or more weights, decrypting the set of intermediate activations. (In step 2A prong 2, decrypting data is a mere application of a computer tool (M.L. Model), which is not indicative of integration into a practical application. In step 2B, merely applying a computer tool is not indicative of significantly more.), which is not indicative of integration into a practical application. In step 2B, merely applying a computer tool is not indicative of significantly more.)Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible.
Regarding claim 4, it is dependent upon claim 1, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 4 recites The processor-implemented method of Claim 1, wherein: the neural network comprises L layers, the first set of layers corresponds to an initial set of layers from layer 1 to layer P, and the second set of layers corresponds to a final set of layers from layer P + 1to layer L. (Generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05(h)), which is not indicative of integration into a practical application. In step 2B, merely applying a computer tool is not indicative of significantly more.), which is not indicative of integration into a practical application. In step 2B, merely applying a computer tool is not indicative of significantly more.)Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible.
Regarding claim 5, it is dependent upon claim 4, and thereby incorporates the limitations of, and corresponding analysis applied to claim 4. Further, claim 5 recites The processor-implemented method of Claim 4, wherein P was selected using a cost function based on one or more of: a computational cost of processing input data at each layer in the neural network, a transmission cost of transmitting intermediate activations associated with each layer in the neural network, or a level of privacy associated with the intermediate activations associated with each layer in the neural network. (Generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05(h)), which is not indicative of integration into a practical application. In step 2B, merely applying a computer tool is not indicative of significantly more.), which is not indicative of integration into a practical application. In step 2B, merely applying a computer tool is not indicative of significantly more.)Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible.
Regarding claim 6, it is dependent upon claim 1, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 6 recites The processor-implemented method of Claim 1, wherein the first set of layers is frozen while the second set of layers is refined. (Generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05(h)), which is not indicative of integration into a practical application. In step 2B, merely applying a computer tool is not indicative of significantly more.), which is not indicative of integration into a practical application. In step 2B, merely applying a computer tool is not indicative of significantly more.)Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible.
Regarding claim 7, it is dependent upon claim 1, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 7 recites The processor-implemented method of Claim 1, wherein refining the one or more weights associated with the second set of layers comprises performing a plurality of training epochs using the set of intermediate activations. (Generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05(h)), which is not indicative of integration into a practical application. In step 2B, merely applying a computer tool is not indicative of significantly more.), which is not indicative of integration into a practical application. In step 2B, merely applying a computer tool is not indicative of significantly more.)Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible.
Regarding claim 8, it is dependent upon claim 1, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 8 recites The processor-implemented method of Claim 1, further comprising: receiving a second set of intermediate activations from a second node device; (Adding insignificant extra-solution activity (mere data gathering) to the judicial exception (MPEP 2106.05(g)) determining that the second set of intermediate activations are outliers; (In step 2A, prong 1, this recites a mathematical concept but for recitation of generic computer components which is not indicative of integration into a practical application.) and discarding the second set of intermediate activations. (Adding insignificant extra-solution activity (mere data gathering) to the judicial exception (MPEP 2106.05(g)), which is not indicative of integration into a practical application. In step 2B, merely applying a computer tool is not indicative of significantly more.), which is not indicative of integration into a practical application. In step 2B, merely applying a computer tool is not indicative of significantly more.)Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible.
Regarding claims 15-30, they comprise of limitations similar to those of claims 1-8 and are therefore rejected for similar rationale.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-4, 15-18, and 23-26 are rejected under 35 U.S.C. 103 as being unpatentable over VEPAKOMMA (U.S. Pub. No. US 20200349443 A1) in view of AKDENIZ (U.S. Pub. No. US 20230068386 A1).
Regarding claim 1, VEPAKOMMA substantially teaches the claim including:
A processor-implemented method of federated learning surrogation, comprising: receiving a set of intermediate activations at a trusted server from a node device, wherein the node device generated the set of intermediate activations using a first set of layers of a neural network; ([0005] The client may take, as input, confidential raw data. The client may: (a) perform forward propagation through the client layers up to and including a layer we sometimes call the “split layer” (i.e. first layers) ; (b) encrypt the outputs of the activation functions (activation outputs) of the split layer, such as by RSA encryption; and (c) send the encrypted activation outputs to the server. ) refining, at the trusted server, one or more weights associated with a second set of layers of the neural network using the set of intermediate activations; ([0017] In some cases: (a) it is important to protect the confidentiality of all of the raw data; and (b) thus the loss function minimizes distance correlation between the activation outputs and the entire raw dataset. [0036] FIG. 1 shows hardware employed for a no-peek DNN. In FIG. 1, a set of client server (e.g., 101, 102, 103, 104) and a host computer (105) perform split learning. However, in FIG. 1: (a) the loss function of the DNN includes a distance correlation term; and (b) minimizing the loss function tends to minimize distance correlation between raw data and split layer activation outputs, and thus to reduce leakage of information. The clients may send encrypted activation outputs 111 to the host during forward propagation. Likewise, the host may send encrypted gradients 112 to the clients. The training of the DNN may be performed one client at a time. For instance, the training may be performed in peer-to-peer mode or in centralized mode, as described in the Background section above. [0055] In a third version of universal decorrelation, weights and biases learned while maximizing the objective function in Equation 2 are used for transfer learning. (the minimization relates to the second layers and outputs weights, as seen in paragraph 55 Further, it should be remembered that the specification of the applicant’s invention states that a server designed to protect user data is a trusted server.))
While VEPAKOMMA does teach refining weights from a node device, it does not explicitly teach:
and transmitting one or more weights from the trusted server corresponding to the refined one or more weights to a federated learning server.
However, in analogous art that similarly employs federated learning, AKDENIZ teaches:
and transmitting one or more weights from the trusted server corresponding to the refined one or more weights to a federated learning server. ([0121] Recently, federated learning has been proposed for distributed GD computation, where learning takes place by a federation of client computing nodes (which may also be referred to herein as “client devices”) that are coordinated by a central server (which may be referred to herein as a MEC server or controller node). [0446] The trusted server 2255 then sends the distorted aggregated data to the central server )
It would have been obvious to a person skilled in the art before the effective filing date of the invention to have combined with AKDENIZ‘s transmission of data and, with VEPAKOMMA‘s federated models and weight refinement, with a reasonable expectation of success, a federated learning system where the weights are refined, as in VEPAKOMMA, and then transmitted from a trusted server to a federated learning server, as found in AKDENIZ. A person of ordinary skill would have been motivated to increase accuracy (AKDENIZ [0005]).
Regarding claim 2, VEPAKOMMA further teaches:
The processor-implemented method of Claim 1, further comprising receiving, from the federated learning server, an updated version of the neural network, wherein the updated version of the neural network was generated based at least in part on the one or more weight updates. ( [0005] and (c) send the encrypted activation outputs to the server. Then, the server may: (a) decrypt the activation outputs and feed these as input into a first layer of the server layers; (b) perform forward propagation through the server layers; (c) perform backpropagation through the server layers; (c) encrypt gradients for the first layer of the server layers (e.g., by RSA encryption); and (d) send the encrypted gradients to the client. Then the client may: (a) decrypt the gradients and feed them into the split layer; and (b) perform backpropagation through the client layers. [0054] In a second version of universal decorrelation, a neural network (NN) onboard a client is trained to maximize the objective function in Equation 2, and then weights and biases learned during this training are used to initialize client layers (onboard the same client) of a no-peek or split learning DNN. In this second version of universal decorrelation, after this initialization: (a) the universal decorrelator is no longer involved; and (b) training of the DNN proceeds by forward propagation and backpropagation through client layers and server layers of the DNN, as usual. (it should be noted that in federated learning, receiving an updated system is done through gradients as shown in this mapping.))
Regarding claim 3, VEPAKOMMA further claims:
The processor-implemented method of Claim 1, wherein the received set of intermediate activations is encrypted, the method further comprising, prior to refining the one or more weights, decrypting the set of intermediate activations. ( [0005] perform forward propagation through the client layers up to and including a layer we sometimes call the “split layer”; (b) encrypt the outputs of the activation functions (activation outputs) of the split layer, such as by RSA encryption; and (c) send the encrypted activation outputs to the server. Then, the server may: (a) decrypt the activation outputs and feed these as input into a first layer of the server layers; [0014] In some implementations, leakage of information is reduced because the loss function of the DNN is selected in such a way as to reduce distance correlation between: (a) raw data; and (b) activation outputs at any given layer of the DNN. (the weight refining is done during the loss function after receiving the encrypted activations.))
Regarding claim 4, VEPAKOMMA further claims:
The processor-implemented method of Claim 1, wherein: the neural network comprises L layers, the first set of layers corresponds to an initial set of layers from layer 1 to layer P, and the second set of layers corresponds to a final set of layers from layer P + 1to layer L. ( [0005] In split learning, a DNN may be trained without sharing raw data, as follows: Loosely speaking, the DNN may be split. That is, the DNN may comprise two portions: (a) layers of the DNN that are performed on a client computer (“client layers”); and (b) layers that are performed on a server computer (“server layers”).)
Regarding claims 15-18 and 23-26, they comprise of limitations similar to those of claims 1-4 and are rejected for similar rationale.
Claims 5, 19, 27 are rejected under 35 U.S.C. 103 as being unpatentable over VEPAKOMMA (U.S. Pub. No. US 20200349443 A1), AKDENIZ (U.S. Pub. No. US 20230068386 A1) in further view of CHEN (U.S. Pub. No. US 20210042620 A1)
While VEPAKOMMA, as modified by AKDENIZ, teaches claim 4, which claim 5 is dependent upon, it does not explicitly teach:
The processor-implemented method of Claim 4, wherein P was selected using a cost function based on one or more of: a computational cost of processing input data at each layer in the neural network, a transmission cost of transmitting intermediate activations associated with each layer in the neural network, or a level of privacy associated with the intermediate activations associated with each layer in the neural network.
However, in analogous art that similarly has two sets of layers, CHEN teaches:
The processor-implemented method of Claim 4, wherein P was selected using a cost function based on one or more of: a computational cost of processing input data at each layer in the neural network, ([0118] The system can also reduce idle computing time by partitioning the neural network into composite layers that each have similar computational requirements. The system can also assign computationally more intensive composite layers to devices with greater computing resources. In some implementations, the system can partition the neural network by performing an initial partition to obtain a set of composite layers, and then obtain a predicted computational cost for performing either a forward pass or a backward pass through each composite layer. Next the system can compute a variance between the computational costs for all of the composite layers in the set, and determine if the variance falls within a predetermined threshold. If the variance does not fall within the threshold, e.g., because the relative computational costs between composite layers are too dissimilar, then the system can repeatedly partition the neural network to obtain different sets of composite layers until the system obtains a set of composite layers whose variance meets the threshold. The net effect of tuning composite layers and assigning devices based on computational capability is reducing the time a device has to wait to receive required data from a neighboring device. (the partition or split or branch point of the model is selected using a computation cost.)) a transmission cost of transmitting intermediate activations associated with each layer in the neural network, or a level of privacy associated with the intermediate activations associated with each layer in the neural network.
It would have been obvious to a person skilled in the art before the effective filing date of the invention to have combined with CHEN‘s layer selection using cost functions and, with VEPAKOMMA‘s, as modified by AKDENIZ, federated models, with a reasonable expectation of success, a cost function that is used to select layers, as in CHEN, used to select a split/branch point in a federated model, as found in VEPAKOMMA as modified by AKDENIZ. A person of ordinary skill would have been motivated to reduce overhead (CHEN [0016]).
Regarding claims 18 and 27, they comprise of limitations similar to those of claim 5 and are therefore rejected for similar rationale.
Claims 6, 20, 28 are rejected under 35 U.S.C. 103 as being unpatentable over VEPAKOMMA (U.S. Pub. No. US 20200349443 A1), AKDENIZ (U.S. Pub. No. US 20230068386 A1) in further view of ALADAHALLI (U.S. Pub. No. US 20220309315 A1)
While VEPAKOMMA teaches claim 1, which claim 6 is dependent upon, it does not explicitly teach:
The processor-implemented method of Claim 1, wherein the first set of layers is frozen while the second set of layers is refined
However, in analogous art that similarly branches the neural network, ALADAHALLI teaches:
The processor-implemented method of Claim 1, wherein the first set of layers is frozen while the second set of layers is refined. ([0032] In various instances, this can be facilitated by inserting into the existing neural network an additional set of layers that branch off from an existing set of layers in the existing neural network. In such case, the additional set of layers can receive as input latent activation maps (e.g., hidden activation values) that are generated by some part of the existing set of layers, and the additional set of layers can be arranged in parallel with a remainder of the existing set of layers. In various aspects, the weights and/or biases of the additional set of layers can be trained as desired, and the weights and/or biases of the existing set of layers can be frozen and/or unchanged during such training. Accordingly, the additional set of layers can be trained to perform the another computing task without affecting and/or deteriorating the performance of the existing set of layers with respect to the given computing task.)
It would have been obvious to a person skilled in the art before the effective filing date of the invention to have combined with ALADAHALLI‘s layer freezing and dynamic layers and, with VEPAKOMMA‘s, as modified by AKDENIZ, federated models, with a reasonable expectation of success, a neural network that has frozen and active layers, as in ALADAHALLI, that is in a federated system, as found in VEPAKOMMA as modified by AKDENIZ. A person of ordinary skill would have been motivated to improve the automation of the models (ALADAHALLI [0002]).
Regarding claims 20 and 28, they comprise of limitations similar to those of claim 6 and are therefore rejected for similar rationale.
Claims 7, 21, 29 are rejected under 35 U.S.C. 103 as being unpatentable over VEPAKOMMA (U.S. Pub. No. US 20200349443 A1), AKDENIZ (U.S. Pub. No. US 20230068386 A1) in view of GHARIBI (U.S. Pub. No. US 20220029971 A1)
While VEPAKOMMA, as modified by AKDENIZ, teaches claim 1, which claim 7 is dependent upon, it does not explicitly teach:
The processor-implemented method of Claim 1, wherein the first set of layers is frozen while the second set of layers is refined
However, in analogous art that similarly uses a federated system, GHARIBI teaches:
The processor-implemented method of Claim 1, wherein refining the one or more weights associated with the second set of layers comprises performing a plurality of training epochs using the set of intermediate activations. ([0024] The approach disclosed below involves calculating an average loss value. The new approach differs from the prior systems which simply compute a loss gradient at a final layer of the server system and back propagates the loss function to refresh weights. [0042] After an “epoch” iteration in which all the batch data of all the clients are processed, and each client-side model is updated via the received gradients from the server-side model 142, a new process is introduced to receive each client-side model, process it to generate a weighted average, and return to each client the weighted averaged model for the next epoch to proceed. (the batch data can be substituted with the activations from VEPAKOMMA))
It would have been obvious to a person skilled in the art before the effective filing date of the invention to have combined with GHARIBI‘s weight refining using epochs and, with VEPAKOMMA‘s, as modified by AKDENIZ, federated models with activations, with a reasonable expectation of success, a neural network trains in epochs using weights, as in GHARIBI, and activations, as found in VEPAKOMMA as modified by AKDENIZ. A person of ordinary skill would have been motivated to improve privacy metrics (GHARIBI [0009] ).
Regarding claims 21 and 29, they comprise of limitations similar to those of claim 7 and are therefore rejected for similar rationale.
Claims 8, 22, 30 are rejected under 35 U.S.C. 103 as being unpatentable over VEPAKOMMA (U.S. Pub. No. US 20200349443 A1), AKDENIZ (U.S. Pub. No. US 20230068386 A1) in view of LIN (U.S. Pub. No. US 20170228659 A1)
Regarding claim 8, VEPAKOMMA further teaches:
The processor-implemented method of Claim 1, further comprising: receiving a second set of intermediate activations from a second node device; ([0036] FIG. 1 shows hardware employed for a no-peek DNN. In FIG. 1, a set of client server (e.g., 101, 102, 103, 104) and a host computer (105) perform split learning. However, in FIG. 1: (a) the loss function of the DNN includes a distance correlation term; and (b) minimizing the loss function tends to minimize distance correlation between raw data and split layer activation outputs, and thus to reduce leakage of information. The clients may send encrypted activation outputs 111 to the host during forward propagation. (it should be noted that each client, meaning a second, third, fourth, etc., sends an activation.) )
While VEPAKOMMA, as modified by AKDENIZ, does teach receiving a second set of activations, it does not explicitly teach:
determining that the second set of intermediate activations are outliers; and discarding the second set of intermediate activations.
However, in analogous art that similarly teaches gathering data, LIN teaches:
determining that the second set of intermediate activations are outliers; and discarding the second set of intermediate activations. ([0043] In some embodiments, the data module 202 includes instructions for causing the machine-learning application 102 to filter or remove data (e.g., outlier data) from the visual web data obtained by the machine-learning application 102.)
It would have been obvious to a person skilled in the art before the effective filing date of the invention to have combined with LIN‘s data filtering and, with VEPAKOMMA‘s, as modified by AKDENIZ, activation gathering, with a reasonable expectation of success, a neural network filters data, as in LIN, where the data is activation data, as found in VEPAKOMMA as modified by AKDENIZ. A person of ordinary skill would have been motivated to improve data recognition (LIN [0002]).
Regarding claims 22 and 30, they comprise of limitations similar to those of claim 8 and are therefore rejected for similar rationale.
Claims 9, 11-12, 14 are rejected under 35 U.S.C. 103 as being unpatentable over GHARIBI (U.S. Pub. No. US 20220029971 A1) in view of VEPAKOMMA (U.S. Pub. No. US 20200349443 A1)
Regarding claim 9, GHARIBI teaches the invention substantially including:
A processor-implemented method of federated learning surrogation, comprising: receiving, at a node device, at least a first set of layers of a neural network from a federated learning server; ([0040] FIG. 1B illustrates another variation on the structure shown in FIG. 1A with the addition of the algorithm provider 104 in step 1 as splitting the model or algorithm into a server-side model 142 and a client-side model 144A. In this case, the algorithm provider or server 104 will distribute the client-side model 144A to one or more clients 102 and the distributed client-side model 144B has its respective split layers 110, 116, 117. US)
However, while GHARIBI does teach receiving the first set of layers, it does not explicitly teach:
processing local data using the first set of layers to generate a set of intermediate activations; and transmitting the set of intermediate activations from the node device to a trusted server.
However, in analogous art that similarly uses a federated system, VEPAKOMMA teaches:
processing local data using the first set of layers to generate a set of intermediate activations; and transmitting the set of intermediate activations from the node device to a trusted server, wherein transmitting to the trusted server enhances user privacy. ( [0005] The client may take, as input, confidential raw data. The client may: (a) perform forward propagation through the client layers up to and including a layer we sometimes call the “split layer”; (b) encrypt the outputs of the activation functions (activation outputs) of the split layer, such as by RSA encryption; and (c) send the encrypted activation outputs to the server. [0036] FIG. 1 shows hardware employed for a no-peek DNN. In FIG. 1, a set of client server (e.g., 101, 102, 103, 104) and a host computer (105) perform split learning. However, in FIG. 1: (a) the loss function of the DNN includes a distance correlation term; and (b) minimizing the loss function tends to minimize distance correlation between raw data and split layer activation outputs, and thus to reduce leakage of information. The clients may send encrypted activation outputs 111 to the host during forward propagation. Likewise, the host may send encrypted gradients 112 to the clients. The training of the DNN may be performed one client at a time. For instance, the training may be performed in peer-to-peer mode or in centralized mode, as described in the Background section above. (the client servers, where the DNN is trained and used, are trusted as the DNN within them uses a loss function to enhance user privacy by minimizing the distance correlation.))
It would have been obvious to a person skilled in the art before the effective filing date of the invention to have combined with VEPAKOMMA‘s activation generation and, with GHARIBI‘s layer reception, with a reasonable expectation of success, federated system that generates activations, as in VEPAKOMMA, where the layers are from a system, as found in GHARIBI. A person of ordinary skill would have been motivated to better preserve confidential data (VEPAKOMMA [0004]).
Regarding claim 11, VEPAKOMMA further teaches:
The processor-implemented method of Claim 9, further comprising receiving, from the federated learning server, an updated version of the neural network, wherein the updated version of the neural network was generated based at least in part on the set of intermediate activations. ( [0005] and (c) send the encrypted activation outputs to the server. Then, the server may: (a) decrypt the activation outputs and feed these as input into a first layer of the server layers; (b) perform forward propagation through the server layers; (c) perform backpropagation through the server layers; (c) encrypt gradients for the first layer of the server layers (e.g., by RSA encryption); and (d) send the encrypted gradients to the client. Then the client may: (a) decrypt the gradients and feed them into the split layer; and (b) perform backpropagation through the client layers. (the act of receiving, decrypting and backpropagating the gradients is how the model is updated as that is how federated systems update the models.))
Regarding claim 12, VEPAKOMMA further teaches:
The processor-implemented method of Claim 11, further comprising: processing local data using the first set of layers of the updated version of the neural network to generate a new set of intermediate activations; and transmitting the new set of intermediate activations to the trusted server. ((16) FIG. 1 shows hardware employed for a no-peek DNN. In FIG. 1, a set of client server (e.g., 101, 102, 103, 104) and a host computer (105) perform split learning. However, in FIG. 1: (a) the loss function of the DNN includes a distance correlation term; and (b) minimizing the loss function tends to minimize distance correlation between raw data and split layer activation outputs, and thus to reduce leakage of information. The clients may send encrypted activation outputs 111 to the host during forward propagation. Likewise, the host may send encrypted gradients 112 to the clients. The training of the DNN may be performed one client at a time. For instance, the training may be performed in peer-to-peer mode or in centralized mode, as described in the Background section above.)
Regarding claim 14, VEPAKOMMA teaches:
The processor-implemented method of Claim 9, further comprising encrypting the set of intermediate activations prior to transmitting the set of intermediate activations to the trusted server. ([0005] The client may take, as input, confidential raw data. The client may: (a) perform forward propagation through the client layers up to and including a layer we sometimes call the “split layer”; (b) encrypt the outputs of the activation functions (activation outputs) of the split layer, such as by RSA encryption; and (c) send the encrypted activation outputs to the server.)
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over GHARIBI (U.S. Pub. No. US 20220029971 A1) in view of VEPAKOMMA (U.S. Pub. No. US 20200349443 A1) in further view of SATHEESH (U.S. Pub. No. US 20220351039 A1)
Regarding claim 10, VEPAKOMMA further teaches:
and the node device uses only the first set of layers to generate the set of intermediate activations. ([0005] The client may take, as input, confidential raw data. The client may: (a) perform forward propagation through the client layers up to and including a layer we sometimes call the “split layer”; (b) encrypt the outputs of the activation functions (activation outputs) of the split layer, such as by RSA encryption; and (c) send the encrypted activation outputs to the server.)
While GHARIBI, as modified by VEPAKOMMA, does teach generating activations, it does not explicitly teach:
The processor-implemented method of Claim 9, wherein: the node device receives the neural network including the first set of layers and a second set of layers,
However, in analogous art that similarly uses a federated system, SATHEESH teaches:
The processor-implemented method of Claim 9, wherein: the node device receives the neural network including the first set of layers and a second set of layers, ([0017] According to a third aspect, a central node or server is provided. The central node or server includes a memory; and a processor coupled to the memory. The processor is configured to: receive a first model from a first user device and a second model from a second user device, wherein the first model is of a neural network model type and has a first set of layers and the second model is of the neural network model type and has a second set of layers different from the first set of layers)
It would have been obvious to a person skilled in the art before the effective filing date of the invention to have combined with SATHEESH‘s reception of two types of layers at a node device and, with GHARIBI‘s, as modified by VEPAKOMMA, activation generation, with a reasonable expectation of success, a node device that received two layers, as in SATHEESH, where the first layers generate activations, as found in GHARIBI, as modified by VEPAKOMMA. A person of ordinary skill would have been motivated to better preserve confidential data (SATHEESH [0003]).
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over GHARIBI (U.S. Pub. No. US 20220029971 A1) in view of VEPAKOMMA (U.S. Pub. No. US 20200349443 A1) in further view of SATHYA (U.S. Pub. No. US 20210012225 A1)
Regarding claim 13, while GHARIBI does teach claim 11, which claim 13 is dependent upon, it does not explicitly teach:
The processor-implemented method of Claim 11, further comprising processing local data using the updated version of the neural network to generate an inference.
However, in analogous art that similarly trains a model, SATHYA teaches:
The processor-implemented method of Claim 11, further comprising processing local data using the updated version of the neural network to generate an inference. ([0006] According to an embodiment, the present invention discloses a method for ranking. The method includes the steps of receiving at least one model from a central server to form at least one model node; training the at least one model node with at least one data node to generate a trained model; generating a weight from each of the at least one data node for the trained model; transferring, the weight from the at least one data node to the central server; inferencing an inference output using the trained model and the at least one data node;)
Response to Arguments
Applicant’s arguments filed 30-MARCH-2026 have been fully considered, but they are found to be non-persuasive.
With regards to the applicant’s remarks regarding the 101 rejection in the non-final action, the applicant argues that the newly amended claim 1 overcomes the 101 rejection set forth. The examiner acknowledges this argument but has found it to be non-persuasive.
In regards to the applicant’s remarks, the applicant argues:
More specifically, the 2024 Al SME Update explicitly states that "[t]he mental processes grouping is not without limits, and as such, claim limitations that only encompass [technology] in a way that cannot practically be performed in the human mind do not fall within this grouping." Id. (emphasis added). Here, the Office asserts that "refining one or more weights associated with a set of layers of the neural network using the set of intermediate activations" is "a mental process but for the recitation of generic computer components." FOA at p. 3. However, Applicant respectfully submits that even if a human can "refine[]" a weight of an intermediate activation, the claims as amended further include the feature "receiving a set of intermediate activations at a trusted server from a node device" and "transmitting one or more weight updates corresponding to the refined one or more weights from the trusted server to a federated learning server." As explained in the 2024 Al SME Update, these elements clearly "only encompass[es]0 in a waythat cannot practically be performed in the human mind." As a human mind cannot make use of or apply "a trusted server," "a node device," or "a federated learning server," these elements can only be performed by the claimed technology itself. As the2024 Al SME Update explains, "ms do not recite a mental process when they contain limitations that cannot practically be performed in the human mind." 2024 Al SME Update at 58136. Thus, the pending claims do not recite an abstract idea and are eligible under Step 2A, Prong 1. Step 2A, Prong 2 - The Claims Integrate any Alleged Abstract Idea into a Practical Application
With regards to this argument, while it is true that the human mind cannot directly make use of a federated server or trusted server, simply claiming that the refining takes place in a trusted server does not mean that the human mind cannot perform the refining. The human mind can indeed refine data, simply stating that the data is refined at the trusted server is not enough to indicate that a human mind cannot refine the data, nor does it indicate that the data cannot be refined without the use of the trusted server.
Applicant respectfully directs the Examiner's attention to Example 48, Claim 3. See July 2024 Subject Matter Eligibility Examples, p. 18. Example 48 describes an artificial-intelligence- based method of analyzing speech signals and separating desired speech from extraneous or background speech. The USPTO explained that while Claim 3 of Example 48 recites abstract ideas, the Claim is directed to an improvement to existing speech-to-text technology. Specifically, the USPTO noted how the Claim recited that the deep neural network, which was trained on speech source separation, could be used to make "individual transcription of each separated speech signal possible." Id., p. 28 (citing M.P.E.P. § 2106.05(a)). Accordingly, the USPTO said that this Claim was patent eligible under Step 2A, Prong 2. Likewise, here, the features of the present claims reflect an improvement to a technology or technical field. For example, the practical application improves the technical field of machine learning (and more particularly, to federated learning). More specifically, training a machine learning model may be performed via "node devices [which] only process local data using [a] model (or a subset of the model), rather than performing actual on-node training." Specification at para. [0025]. Certain techniques described therein may reduce "the computational burden on the nodes (e.g., requiring reduced computational time, reduced power consumption, reduced memory demands, and the like)," As well as "enhance[] or protect[] user privacy." Id. at paras. [0021] and [0025]. Thus, particular solutions discussed therein improve upon conventional techniques, such as by enabling more efficient use of computational resources and greater user privacy.
With regards to this argument, it must be remembered that the improvement cannot be the abstract idea itself. The claims as presented include limitations that indicate a transmission of data and a field of use. There is no practical application of the abstract idea of refining the weights to bring the abstract idea into the practical application found within the claim limitations. As outlined by MPEP 2106(a)(II), the claims themselves have to show the improvement as well: “To show that the involvement of a computer assists in improving the technology, the claims must recite the details regarding how a computer aids the method, the extent to which the computer aids the method, or the significance of a computer to the performance of the method. Merely adding generic computer components to perform the method is not sufficient. Thus, the claim must include more than mere instructions to perform the method on a generic component or machinery to qualify as an improvement to an existing technology. “ The improvement cannot simply lay in the abstract idea itself.
With regards to the applicant’s remarks regarding the 103 rejection in the non-final action, the applicant argues that the prior art does not teach the newly amended claims 1, 9, 15, and 23. The examiner acknowledges this argument and has adjusted VEPAKOMMA. Namely, VEPAKOMMA paragraph 36 has been added to show that VEPAKOMMA does indeed teach that the client servers it teaches are trusted servers, as per the definition provided by applicant’s summary paragraph 21. Further, the examiner has adjusted all dependent claims accordingly.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SKIELER A KOWALIK whose telephone number is (571)272-1850. The examiner can normally be reached 8-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Mariela D Reyes can be reached at (571)270-1006. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SKIELER ALEXANDER KOWALIK/Examiner, Art Unit 2142
/Mariela Reyes/Supervisory Patent Examiner, Art Unit 2142
Prosecution Insights