DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-25 are presented for examination.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 10 and 17 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
With respect to claim 10, it is unclear what the limitation “a portion of the computation of the averages” [line 2] refers to. Claim 10 is depended on claim 9. However, claim 9 clearly recites “an average of corresponding parameters” [line 1]. For the purposes of examination, examiner would interpret this limitation as “a portion of the computation of the averages of corresponding parameters”.
With respect to claim 17, it is unclear what the limitation “modifying the level of compression” [line 3] refers to. Claim 17 is depended on claim 11, and this claim is not depended on claim 16 where it is first recited “a level of compression” [line 2]. For the purposes of examination, examiner would interpret this limitation as “modifying the level of compression”.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-25 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Independent claims
Step 1
Claim 1 is drawn to a method for performing distributed machine learning (ML) across a plurality of computers, and claim 18 is drawn to a non-transitory machine-readable medium storing a program including instructions to execute the method of claim 1. Each of these claim groups falls under one of four categories of statutory subject matter (process/method, machines/product/apparatus, manufactures, and composition of matter).
Step 2A – Prong 1
Claims 1 and 18 are directed to a judicially recognized exception of an abstract idea without significantly more.
Claims 1 and 18 recite a method of compressing the set of ML parameters based on a current state of a connection to a central computer that receives sets of ML parameters from a plurality of the computers that under its broadest reasonable interpretation enumerates a mathematical concept. The “compressing” step is a functional process which involves the underlying mathematical algorithms used for compression. Therefore, the step of compressing the set of ML parameters is nothing more than a mathematical concept (MPEP 2106.04(a)(2)(I)).
Claims 1 and 18 recite further a method of process the compressed set of ML parameters along with corresponding sets of ML parameters that under its broadest reasonable interpretation enumerates a mathematical concept. Processing parameters in a central computer is a computational task that, by itself, is related to a mathematical concept. Therefore, the step of processing compressed set of ML parameters is nothing more than a mathematical concept (MPEP 2106.04(a)(2)(I)).
Step 2A – Prong 2
Claims 1 and 18 recite further receiving a set of ML parameters from the first computer related to training an ML model that fail to integrate the abstract idea into a practical application. The step of receiving ML parameters is a form of insignificant input and output solution activities, where receiving a set of ML parameters is necessary for all uses of judicial exception. This additional element does not integrate the abstract idea into a practical application because this limitation is considered as one of examples of mere data gathering activities (MPEP 2106.05(g)).
Claims 1 and 18 recite further sending the compressed set of ML parameters to the central computer for the central computer to … received from the other computers of the plurality of computers that fail to integrate the abstract idea into a practical application. The step of sending compressed ML parameters is a form of insignificant input and output solution activities, where sending compressed set of ML parameters is necessary for all uses of judicial exception. This additional element does not integrate the abstract idea into a practical application because this limitation is considered as a routine and conventional computing activity (MPEP 2106.05(d)).
Step 2B
The additional elements in step 2A-Prong 2 those are forms of insignificant extra-solution activities, do not amount to significantly more than an abstract idea because the court decision have determined that these additional elements of receiving a set of ML parameters; and sending compressed set of ML parameters to be well-understood, routine, and conventional when claimed in a merely generic manner (MPEP 2106.05(d)(II)).
As such, claims 1 and 18 are not patent eligible.
Dependent claims
Claims 2-17 and 19-25 merely narrow the previously recited abstract idea limitations. For the reasons described above with respect to claims 1 and 18, this judicial exception is not meaningfully integrated into a practical application, or significantly more than the abstract idea. The claims disclose similar limitations described for the independent claims above and do not provide anything more than the mathematical process. Therefore, claims 2-17 and 19-25 also recite abstract ideas that do not integrate into a practical application or amount to significantly more than the judicial exception, and are rejected under U.S.C. 101.
Step 1
Claims 2-17 are drawn to a method for performing distributed machine learning (ML) across a plurality of computers, and claims 19-25 drawn to a non-transitory machine-readable medium storing a program including instructions to execute the method of claims 2-17. Each of these claim groups falls under one of four categories of statutory subject matter (process/method, machines/product/apparatus, manufactures, and composition of matter).
Step 2A – Prong 1
Dependent claims 2 and 19 recite further the mathematical concept by the set of ML parameters is a set of local gradients computed for the ML model at the first computer; and the central computer receives corresponding sets of local gradients computed for the ML model from each of the computers of the plurality of computers those are based on one or more features of the ML project (MPEP 2106.04(a)(2)(I)).
Dependent claims 3 and 19 recite further the mathematical concept by each local gradient in the set of local gradients specifies an update to a different trainable parameter of the ML model based on local training of the ML model at the first computer; and each of the respective other computers of the plurality of computers computes respective sets of local gradients specifying updates to the trainable parameters based on respective local training of the ML model at the respective other computer those are based on one or more features of the ML project (MPEP 2106.04(a)(2)(I)).
Dependent claims 5 and 20 recite further the mathematical concept by compressing the second set of local gradients based on a second state of the connection to the central computer; and process the compressed second set of local gradients along with corresponding sets of local gradients those are based on one or more features of the ML project (MPEP 2106.04(a)(2)(I)).
Dependent claims 6 and 21 recite further the mathematical concept by compressing the second set of local gradients based on a second state of the connection to the central computer; and process the compressed second set of local gradients along with corresponding sets of local gradients those are based on one or more features of the ML project (MPEP 2106.04(a)(2)(I)).
Dependent claims 7 and 22 recite further the mathematical concept by a second smart NIC of the central computer processes the compressed set of ML parameters along with the corresponding sets of ML parameters from the other computers those are based on one or more features of the ML project (MPEP 2106.04(a)(2)(I)).
Dependent claims 8 and 22 recite further the mathematical concept by the second smart NIC computes a set of global parameters based on the received corresponding sets of compressed parameters and provides the global parameter set to the plurality of computers those are based on one or more features of the ML project (MPEP 2106.04(a)(2)(I)).
Dependent claims 9 and 23 recite further the mathematical concept by each global parameter in the global parameter set is an average of corresponding parameters received from the plurality of computers those are based on one or more features of the ML project (MPEP 2106.04(a)(2)(I)).
Dependent claims 10 and 23 recite further the mathematical concept by the second smart NIC performs at least a portion of the computation of the averages without decompressing the sets of ML parameters received from the plurality of computers those are based on one or more features of the ML project (MPEP 2106.04(a)(2)(I)).
Dependent claim 12 recites further the mathematical concept by the second smart NIC decompresses the compressed ML parameters received from the plurality of computers and compresses the global parameters prior to providing the global parameters to the plurality of computers those are based on one or more features of the ML project (MPEP 2106.04(a)(2)(I)).
Dependent claim 13 recites further the mathematical concept by decompressing the compressed set of global parameters that is based on one or more features of the ML project (MPEP 2106.04(a)(2)(I)).
Dependent claims 14 and 24 recite further the mathematical concept by compressing the set of ML parameters comprises quantizing each parameter to a specific number of bits that is based on one or more features of the ML project (MPEP 2106.04(a)(2)(I)).
Dependent claims 16 and 25 recite further the mental process by evaluating the current state of the connection to the central computer to determine a level of compression that is based on one or more features of the ML project (MPEP 2106.04(a)(2)(III)).
Dependent claim 17 recites further the mental process by receiving a congestion control message at the first smart NIC; and modifying the level of compression for subsequent sets of ML parameter based on the received congestion control message that is based on one or more features of the ML project (MPEP 2106.04(a)(2)(III)).
Step 2A – Prong 2
Dependent claims 4, 20 and 21 recite further the insignificant extra solution activities by the ML model is a neural network; and the set of local gradients comprises gradients for updating the trainable parameters of a particular layer of the neural network. This additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (MPEP 2106.05(g)).
Dependent claim 5 and 20 recite further the insignificant extra solution activities by receiving a second set of local gradients from the first computer for updating trainable parameters of a second layer of the neural network; and sending the compressed second set of local gradients to the central computer for the central computer to … received from the other computers of the plurality of computers. These additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea (MPEP 2106.05(g)).
Dependent claims 6 and 21 recite further the insignificant extra solution activities by receiving a second set of local gradients from the first computer for updating trainable parameters of a second layer of a second neural network; and sending the compressed second set of local gradients to the central computer for the central computer to … received from the other computers of the plurality of computers. These additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea (MPEP 2106.05(g)).
Dependent claim 11 recites further the insignificant extra solution activities by the global parameters are used by each respective computer of the plurality of computers to update a respective copy of the ML model at the respective computer. This additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (MPEP 2106.05(g)).
Dependent claim 13 recites further the insignificant extra solution activities by receiving a compressed set of global parameters from the central computer based on the central computer processing the sets of ML parameters received from the plurality of computers; and providing the decompressed set of global parameters to the first computer for training the ML model. These additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea (MPEP 2106.05(g)).
Dependent claims 15 and 24 recite further the insignificant extra solution activities by the specific number of bits varies based on the current state of the connection to the central computer. This additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (MPEP 2106.05(g)).
As such, dependent claims 2-17 and 19-25 are not patent eligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-25 are rejected under 35 U.S.C. 103 as being unpatentable over Savic et al (US 20190325302 A1) hereafter Savic, and further in view of Ramachandran et al (US 20220245522 A1) hereafter Ramachandran.
With respect to claim 1, Savic teaches a method for performing distributed machine learning (ML) across a plurality of computers (a distributed computing environment comprises a large scale of shared computing resources over a cluster of computing nodes, wherein the data includes training data to build and optimize deep learning (DL) models and model parameters [par. 0003]),
receiving a set of ML parameters from the first computer related to training an ML model (a parameter server system provides a communication synchronization protocol in which multiple worker nodes involved in a distributed DL training that have shared access to a set of model parameters of a given DL model [par. 0004, 0005]);
compressing the set of ML parameters (data compression methods can be used to compress local processing results, for example, embedded data compression/decompression engines of the smart NIC devices that execute parameter server nodes. The local parameter server node receives the local processing results of all other worker nodes operating on remote host nodes, wherein the local processing results are transmitted from remote parameter server nodes to the remote host nodes. A host node may include multiple virtual worker nodes those are connected to an NIC [par. 0057-0061 and FIG. 5]); and
sending the compressed set of ML parameters to the central computer for the central computer to process the compressed set of ML parameters along with corresponding sets of ML parameters received from the other computers of the plurality of computers (each worker node communicates with one of the parameter server nodes to send computed parameters to parameter server node. Computed/compressed parameters will then be sent to the remote host nodes [par. 0030, 0056-0059]).
However, Savic does not disclose the method comprising: a smart network interface controller (NIC); compressing ML parameters based on a current state of a connection.
In the same field of endeavor, Ramachandran teaches the method comprising: a smart network interface controller (NIC) (a compute node cluster includes a network interface controller (NIC) that is coupled to a switch via a link [par. 0047 and FIG. 4]);
compressing ML parameters based on a current state of a connection (the model is trained during the training phase using a set of inputs that involves adjusting the weights of the network to improve the accuracy of the output. The model employs backpropagation to train the network that is a method to adjust the connection weights. Backpropagation calculates the gradient of the cost function associated with a given state with respect to the weight. Backpropagation is the mechanism for training a model that will be later be compressed [par. 0030, 0031]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have incorporated the concept of employing selective compression for addressing congestion control for AI workloads as suggested by Ramachandran into the concept of implementing a parameter server within a networking infrastructure of a computing system to reduce communication bandwidth as suggested by Savic because both of these systems addressing the process of compressing input parameters and sending the compressed parameters to the host/main nodes of the system. Doing so would be desirable because the system of Savic would be more efficient by including a network interface controller (NIC) that is also a part of a distributed network environment which uses the same ML model on compute nodes, and compressing the parameters based on the state of the connection weights using stochastic gradient descent of other ML methods (Ramachandran, [par. 0031, 0039, 0047]).
With respect to claim 2, the combination of Savic and Ramachandran teaches wherein:
the set of ML parameters is a set of local gradients computed for the ML model at the first computer (Savic, the parameter server node performs an all-reduce operation on aggregated parameters by computing an average of all the local gradients provided by the worker nodes [par. 0030, 0031]); and
the central computer receives corresponding sets of local gradients computed for the ML model from each of the computers of the plurality of computers (Savic, the globally shared parameters on each of the parameter server nodes are updated with the computed gradient average, wherein each parameter server node receives a set of locally computed parameters. A host compute node is also called a server node, which includes multiple virtual worker nodes [par. 0030, 0031 and FIG. 4]).
With respect to claim 3, the combination of Savic and Ramachandran teaches wherein:
each local gradient in the set of local gradients specifies an update to a different trainable parameter of the ML model based on local training of the ML model at the first computer (Savic, an error backpropagation process is utilized to compute the gradient of loss with respect to the DL parameters. The gradient parameters computed by all worker nodes for a given iteration are then aggregated/synchronized/averaged and the averaged gradient parameters are pushed to each work node, so the worker node can then perform a parameter update process using the averaged gradient parameters to update the model parameters of the DL network model [par. 0016, 0017, 0027]); and
each of the respective other computers of the plurality of computers computes respective sets of local gradients specifying updates to the trainable parameters based on respective local training of the ML model at the respective other computer (Savic, the model parameters are aggregated/synchronized and the averaged gradient parameters are pushed to each worker node to perform the update based on the respective worker nodes [par. 0016, 0017, 0027, 0030]).
With respect to claim 4, the combination of Savic and Ramachandran teaches wherein:
the ML model is a neural network (Savic, a deep neural network model is used to implement a parameter server (PS) system using a cluster of worker nodes [par. 0004]); and
the set of local gradients comprises gradients for updating the trainable parameters of a particular layer of the neural network (Savic, a DL application can utilize a DNN that comprises a feedforward artificial neural network with multiple hidden layers. The filters of a convolutional layer each comprise a set of learnable parameters/weights [par. 0014]).
With respect to claim 5, the combination of Savic and Ramachandran teaches wherein the set of local gradients is a first set of local gradients, the particular layer is a first layer of the neural network (Savic, the feedforward process in the DL training process is performed layer by layer to process the subset of data using a given DL model [par. 0014, 0029]), and the current state of the connection is a first state, the method further comprising, at the smart NIC:
receiving a second set of local gradients from the first computer for updating trainable parameters of a second layer of the neural network (Savic, a parameter server system provides a communication synchronization protocol in which multiple worker nodes involved in a distributed DL training that have shared access to a set of model parameters of a given DL model [par. 0004, 0005]);
compressing the second set of local gradients based on a second state of the connection to the central computer (Ramachandran, the model is trained during the training phase using a set of inputs that involves adjusting the weights of the network to improve the accuracy of the output. The model employs backpropagation to train the network that is a method to adjust the connection weights. Backpropagation calculates the gradient of the cost function associated with a given state with respect to the weight. Backpropagation is the mechanism for training a model that will be later be compressed [par. 0030, 0031]); and
sending the compressed second set of local gradients to the central computer for the central computer to process the compressed second set of local gradients along with corresponding sets of local gradients received from the other computers of the plurality of computers (Savic, each worker node communicates with one of the parameter server nodes to send computed parameters to parameter server node. Computed/compressed parameters will then be sent to the remote host nodes [par. 0030, 0056-0059]).
With respect to claim 6, the combination of Savic and Ramachandran teaches wherein the set of local gradients is a first set of local gradients, the particular layer is a first layer of a first neural network (Savic, the feedforward process in the DL training process is performed layer by layer to process the subset of data using a given DL model [par. 0014, 0029]), and the current state of the connection is a first state, the method further comprising, at the smart NIC:
receiving a second set of local gradients from the first computer for updating trainable parameters of a second layer of a second neural network (Savic, a parameter server system provides a communication synchronization protocol in which multiple worker nodes involved in a distributed DL training that have shared access to a set of model parameters of a given DL model [par. 0004, 0005]);
compressing the second set of local gradients based on a second state of the connection to the central computer (Ramachandran, the model is trained during the training phase using a set of inputs that involves adjusting the weights of the network to improve the accuracy of the output. The model employs backpropagation to train the network that is a method to adjust the connection weights. Backpropagation calculates the gradient of the cost function associated with a given state with respect to the weight. Backpropagation is the mechanism for training a model that will be later be compressed [par. 0030, 0031]); and
sending the compressed second set of local gradients to the central computer for the central computer to process the compressed second set of local gradients along with corresponding sets of local gradients received from the other computers of the plurality of computers (Savic, each worker node communicates with one of the parameter server nodes to send computed parameters to parameter server node. Computed/compressed parameters will then be sent to the remote host nodes [par. 0030, 0056-0059]).
With respect to claim 7, the combination of Savic and Ramachandran teaches wherein the smart NIC is a first smart NIC, wherein a second smart NIC of the central computer processes the compressed set of ML parameters along with the corresponding sets of ML parameters from the other computers (Savic, each compute node comprises a plurality of GPU devices and respective network interface cards (NICs), wherein NICs 230-1 and 230-2 execute respective PS nodes. Each NIC configured to execute a parameter server node of a distributed server system within a networking infrastructure of a computing system [par. 0040, 0046 and FIG. 2]).
With respect to claim 8, the combination of Savic and Ramachandran teaches wherein the second smart NIC computes a set of global parameters based on the received corresponding sets of compressed parameters and provides the global parameter set to the plurality of computers (Savic, the NIC comprises a multi-core SoC that executes core logic to implement a parameter server node which performs various functions for processing and synchronizing local and global model parameters for worker nodes. Once each global parameter is aggregated, each is then sent to other non-master PS nodes [par. 0043-0046]).
With respect to claim 9, the combination of Savic and Ramachandran teaches wherein each global parameter in the global parameter set is an average of corresponding parameters received from the plurality of computers (Savic, the local parameter server node performs a parameter aggregation process using an average method to compute average of all the local processing results received from the local worker nodes and remote worker nodes executing on remote host nodes. The parameter server nodes communicate with each other to aggregate the local parameters, such as compute global average gradients, and update the DL model parameters. Each parameter server node pushes the global updated parameters to the worker nodes, and the worker nodes then proceed to use global updated parameters to perform weight update [par. 0030, 0031, 0057-0059]).
With respect to claim 10, the combination of Savic and Ramachandran teaches wherein the second smart NIC performs at least a portion of the computation of the averages without decompressing the sets of ML parameters received from the plurality of computers (Savic, the parameter server nodes send the local computed parameters to one of the parameter server nodes which is designed to performed an all reduce operation. The designated parameter server node computes an average of all the local gradients provided by the worker nodes. The compressed data can be decompressed prior to performing the all-reduce computations [par. 0016, 0031, 0038, 0054, 0059-0061]).
With respect to claim 11, the combination of Savic and Ramachandran teaches wherein the global parameters are used by each respective computer of the plurality of computers to update a respective copy of the ML model at the respective computer (Savic, in each iteration of a backpropagation process in data parallel training, each worker node has access to a complete copy of a current state of the DL model, which is maintained in the data stores of globally shared model parameters and maintained by respective parameter server nodes [par. 0016, 0028, 0031, 0044]).
With respect to claim 12, the combination of Savic and Ramachandran teaches wherein the second smart NIC decompresses the compressed ML parameters received from the plurality of computers and compresses the global parameters prior to providing the global parameters to the plurality of computers (Savic, the global aggregated processing results are computed by the master PS, wherein these results can then be transferred to the worker nodes executing on remote host nodes. Compressing local processing results reduces the communication bandwidth for transmitting the local processing results to the master PS. The compressed data that is received can be decompressed prior to performing the other computations [par. 0060, 0061]).
With respect to claim 13, the combination of Savic and Ramachandran teaches further comprising, after sending the compressed parameter to the central computer:
receiving a compressed set of global parameters from the central computer based on the central computer processing the sets of ML parameters received from the plurality of computers (Savic, each parameter server node will receive a set of locally computed parameters from one or more associated worker nodes, the parameter worker nodes then use the global updated parameters to perform updating the weight [par. 0030]);
decompressing the compressed set of global parameters (Savic, the global aggregated results are computed by the master node, and the data compression methods can be utilized to compress the local results, such that compressed data is received can be decompressed [par. 0058-0061]); and
providing the decompressed set of global parameters to the first computer for training the ML model (Savic, after decompressing, the parameters are pushed to the host system to perform data parallel training process [par. 0061, 0062]).
With respect to claim 14, the combination of Savic and Ramachandran teaches wherein compressing the set of ML parameters comprises quantizing each parameter to a specific number of bits (Ramachandran, most ML/AI models employ 32-bit floating point data, but the gradient descent data are compressed by converting 32-bit to 16-bit floating point data [par. 0056, 0068]).
With respect to claim 15, the combination of Savic and Ramachandran teaches wherein the specific number of bits varies based on the current state of the connection to the central computer (Ramachandran, the system comprises switches that maintain metadata relating to ingress and egress buffer/queue fill levels. The buffer/queue has finite sizes (bits) and may approach or reach an overfill state, which may result in packets being dropped [par. 0054, 0068]).
With respect to claim 16, the combination of Savic and Ramachandran teaches further comprising evaluating the current state of the connection to the central computer to determine a level of compression (Ramachandran, variable compression of exchanged model data like a local gradient data is used to avoid network congestion based on real-time and/or projected network congestion levels [par. 0065, 0094]).
With respect to claim 17, the combination of Savic and Ramachandran teaches wherein evaluating the current state of the connection comprises:
receiving a congestion control message at the first smart NIC; and modifying the level of compression for subsequent sets of ML parameter based on the received congestion control message (Ramachandran, a network congestion may occur during a sync/communication phase following a sync state. A message may be used to announce each compute node works on may be pre-batched or batching may be performed on the compute node following the download [par. 0040, 0095]).
With respect to claim 18, it is a non-transitory computer-readable medium that is corresponding to the method of claim 1. Therefore, it is rejected for the same reason as claimed in claim 1 above.
With respect to claim 19, the combination of Savic and Ramachandran teaches wherein:
the set of ML parameters is a set of local gradients computed for the ML model at the first computer, each local gradient in the set of local gradients specifying an update to a different trainable parameter of the ML model based on local training of the ML model at the first computer (Savic, an error backpropagation process is utilized to compute the gradient of loss with respect to the DL parameters. The gradient parameters computed by all worker nodes for a given iteration are then aggregated/synchronized/averaged and the averaged gradient parameters are pushed to each work node, so the worker node can then perform a parameter update process using the averaged gradient parameters to update the model parameters of the DL network model [par. 0016, 0017, 0027]);
the central computer receives corresponding sets of local gradients computed for the ML model from each of the computers of the plurality of computers, each of the respective other computers of the plurality of computers computing respective sets of local gradients specifying updates to the trainable parameters based on respective local training of the ML model at the respective other computer (Savic, the model parameters are aggregated/synchronized and the averaged gradient parameters are pushed to each worker node to perform the update based on the respective worker nodes [par. 0016, 0017, 0027, 0030]).
With respect to claim 20, the combination of Savic and Ramachandran teaches wherein:
the ML model is a neural network (Savic, a deep neural network model is used to implement a parameter server (PS) system using a cluster of worker nodes [par. 0004]);
the set of local gradients is a first set of local gradients for updating the trainable parameters of a first layer of the neural network (Savic, a DL application can utilize a DNN that comprises a feedforward artificial neural network with multiple hidden layers. The filters of a convolutional layer each comprise a set of learnable parameters/weights [par. 0014]);
the current state of the connection is a first state; and the program further comprises sets of instructions for:
receiving a second set of local gradients from the first computer for updating trainable parameters of a second layer of the neural network (Savic, a parameter server system provides a communication synchronization protocol in which multiple worker nodes involved in a distributed DL training that have shared access to a set of model parameters of a given DL model [par. 0004, 0005]);
compressing the second set of local gradients based on a second state of the connection to the central computer (Ramachandran, the model is trained during the training phase using a set of inputs that involves adjusting the weights of the network to improve the accuracy of the output. The model employs backpropagation to train the network that is a method to adjust the connection weights. Backpropagation calculates the gradient of the cost function associated with a given state with respect to the weight. Backpropagation is the mechanism for training a model that will be later be compressed [par. 0030, 0031]); and
sending the compressed second set of local gradients to the central computer for the central computer to process the compressed second set of local gradients along with corresponding sets of local gradients received from the other computers of the plurality of computers (Savic, each worker node communicates with one of the parameter server nodes to send computed parameters to parameter server node. Computed/compressed parameters will then be sent to the remote host nodes [par. 0030, 0056-0059]).
With respect to claim 21, the combination of Savic and Ramachandran teaches wherein:
the ML model is a first neural network (Savic, a deep neural network model is used to implement a parameter server (PS) system using a cluster of worker nodes [par. 0004]);
the set of local gradients is a first set of local gradients for updating the trainable parameters of a first layer of the first neural network (Savic, a DL application can utilize a DNN that comprises a feedforward artificial neural network with multiple hidden layers. The filters of a convolutional layer each comprise a set of learnable parameters/weights [par. 0014]);
the current state of the connection is a first state; and the program further comprises sets of instructions for:
receiving a second set of local gradients from the first computer for updating trainable parameters of a second layer of a second neural network (Savic, a parameter server system provides a communication synchronization protocol in which multiple worker nodes involved in a distributed DL training that have shared access to a set of model parameters of a given DL model [par. 0004, 0005]);
compressing the second set of local gradients based on a second state of the connection to the central computer (Ramachandran, the model is trained during the training phase using a set of inputs that involves adjusting the weights of the network to improve the accuracy of the output. The model employs backpropagation to train the network that is a method to adjust the connection weights. Backpropagation calculates the gradient of the cost function associated with a given state with respect to the weight. Backpropagation is the mechanism for training a model that will be later be compressed [par. 0030, 0031]); and
sending the compressed second set of local gradients to the central computer for the central computer to process the compressed second set of local gradients along with corresponding sets of local gradients received from the other computers of the plurality of computers (Savic, each worker node communicates with one of the parameter server nodes to send computed parameters to parameter server node. Computed/compressed parameters will then be sent to the remote host nodes [par. 0030, 0056-0059]).
With respect to claim 22, the combination of Savic and Ramachandran teaches wherein:
the smart NIC is a first smart NIC (Savic, each compute node comprises a plurality of GPU devices and respective network interface cards (NICs), wherein NICs 230-1 and 230-2 execute respective PS nodes. Each NIC configured to execute a parameter server node of a distributed server system within a networking infrastructure of a computing system [par. 0040, 0046 and FIG. 2]);
a second smart NIC of the central computer processes the compressed set of ML parameters along with the corresponding sets of ML parameters from the other computers, said processing comprising computing a set of global parameters based on the received corresponding sets of compressed parameters and providing the global parameter set to the plurality of computers (Savic, the NIC comprises a multi-core SoC that executes core logic to implement a parameter server node which performs various functions for processing and synchronizing local and global model parameters for worker nodes. Once each global parameter is aggregated, each is then sent to other non-master PS nodes [par. 0043-0046]).
With respect to claim 23, the combination of Savic and Ramachandran teaches wherein:
each global parameter in the global parameter set is an average of corresponding parameters received from the plurality of computers (Savic, the local parameter server node performs a parameter aggregation process using an average method to compute average of all the local processing results received from the local worker nodes and remote worker nodes executing on remote host nodes. The parameter server nodes communicate with each other to aggregate the local parameters, such as compute global average gradients, and update the DL model parameters. Each parameter server node pushes the global updated parameters to the worker nodes, and the worker nodes then proceed to use global updated parameters to perform weight update [par. 0030, 0031, 0057-0059]);
the second smart NIC performs at least a portion of the computation of the averages without decompressing the sets of ML parameters received from the plurality of computers (Savic, the parameter server nodes send the local computed parameters to one of the parameter server nodes which is designed to performed an all reduce operation. The designated parameter server node computes an average of all the local gradients provided by the worker nodes. The compressed data can be decompressed prior to performing the all-reduce computations [par. 0016, 0031, 0038, 0054, 0059-0061]); and
the global parameters are used by each respective computer of the plurality of computers to update a respective copy of the ML model at the respective computer (Savic, in each iteration of a backpropagation process in data parallel training, each worker node has access to a complete copy of a current state of the DL model, which is maintained in the data stores of globally shared model parameters and maintained by respective parameter server nodes [par. 0016, 0028, 0031, 0044]).
With respect to claim 24, the combination of Savic and Ramachandran teaches wherein:
the set of instructions for compressing the set of ML parameters comprises a set of instructions for quantizing each parameter to a specific number of bits (Ramachandran, most ML/AI models employ 32-bit floating point data, but the gradient descent data are compressed by converting 32-bit to 16-bit floating point data [par. 0056, 0068]); and
the specific number of bits varies based on the current state of the connection to the central computer (Ramachandran, the system comprises switches that maintain metadata relating to ingress and egress buffer/queue fill levels. The buffer/queue has finite sizes (bits) and may approach or reach an overfill state, which may result in packets being dropped [par. 0054, 0068]).
With respect to claim 25, it is a non-transitory computer-readable medium that is corresponding to the method of claim 16. Therefore, it is rejected for the same reason as claimed in claim 16 above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Karumbunathan et al (US 20190286373 A1) disclosed servicing I/O operations in a cloud-based storage system, including: receiving, by the cloud-based storage system, a request to write data to the cloud-based storage system; storing, in solid-state storage of the cloud-based storage system, the data; storing, in object storage of the cloud-based storage system, the data; detecting that at least some portion of the solid-state storage of the cloud-based storage system has become unavailable; identifying data that was stored in the portion of the solid-state storage of the cloud-based storage system that has become unavailable; retrieving, from object storage of the cloud-based storage system, the data that was stored in the portion of the solid-state storage of the cloud-based storage system that has become unavailable; and storing, in solid-state storage of the cloud-based storage system, the retrieved data.
Kim et al (US 20210374503 A1) disclosed a distributed network includes a first group of computing devices. Each computing device is to be coupled to two neighbor computing devices of the first group of computing device and is to: (i) aggregate gradient values received from a first neighbor computing device with local gradient values to generate a partial aggregate of gradient values that are to train a neural network model; (ii) transfer the partial aggregate of gradient values to a second neighbor computing device; and repeat (i) and (ii) until a first aggregate of gradient values from the first group of computing devices is buffered at a first computing device of the first group of computing devices. The first computing device is to transfer the first aggregate of gradient values to a second group of computing devices of the distributed network for further aggregation.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Quoc Phung whose telephone number is (703) 756 1330. The examiner can normally be reached on Monday through Friday from 9am to 5pm PT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) athttp://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Welch can be reached on 571-272-7212. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Q.L.P./Examiner, Art Unit 2143
/JENNIFER N WELCH/Supervisory Patent Examiner, Art Unit 2143