Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment/Arguments
Applicant’s arguments to the rejections under 35 U.S.C. 102 and 35 U.S.C. 103 filed on February 10, 2026 have been fully considered but are not persuasive. Applicant asserts that none of the cited references discloses or suggests training the neural network model to generate a target model/rule configured to implement NLP training, and relies on paragraph [0083] and [0095] of the Specification in support of this position. However, Applicant’s arguments are not persuasive.
The cited specification passages describe example embodiments, but the claims are given their broadest reasonable interpretation. Under BRI, the recited “target model/rule” encompasses a trained neural network model and does not require the specific architecture or data processing described in the specification.
As set forth in the rejection, Sridharan teaches training a neural network model in a distributed system in which different computational nodes receive different portions of input data and performing training computations (paragraphs [0189]-[0191]). Sridharan further teaches that training includes updating model weights using backpropagation and stochastic gradient descent (paragraph [0173]) and synchronizing weights across nodes using collective communication operations such as allreduce and allgather (paragraph [0202], which corresponds to the claimed complete weight coefficient. While Sridharan discusses communication efficiency in distributed systems, the cited portions explicitly describe the underlying neural-network training operations performed by the accelerators.
Further, the claim recites “model/rule” in the disjunctive, which encompasses a trained model, Sridharan teaches training a neural network, which results in a trained neural network model. Sridharan additionally teaches that trained neural network models are used in applications including natural language process (paragraph [0211]. Accordingly, Sridharan teaches the disputed limitation.
Applicant acknowledges that Sridharan discloses distributed training strategies, including model parallelisms and data parallelism, but asserts that Sridharan is silent in regarding generating a target model/rule configured to implement NLP training. However, Applicant’s argument is not persuasive.
Sridharan’s disclosure of model parallelism and data parallelism are expressly directed to performing neural-network training across multiple computational nodes, where nodes collectively train the network using portions of the model and/or input data (paragraph [0189]-[0190]). Sridharan further teaches that training includes updating model weights using backpropagation and stochastic gradient descent (paragraph [0173]). These teachings correspond to training a neural network to produce a trained model.
Further, Sridharan explicitly identifies natural language processing as an application of the trained neural network system. Thus, the trained model produced by the disclosed training process corresponds to the claimed target model configured to implement NLP training. Applicant’s argument that Sridharan is “silent” is not persuasive.
Applicant asserts that Lopez does not remedy the deficiencies of Sridharan with respect to the features of independent claim 1. However, Applicant’s argument is not persuasive.
As set forth above, the rejection of claim 1 relies on Sridharan alone for the disputed limitations, including training a neural network model to generate a target model configured to implement NLP training. Accordingly, no deficiency exists that requires remedy by Lopez.
Therefore, Applicant’s argument regarding Lopez does not overcome the rejection.
Applicant asserts that the cited references fail to disclose or suggest each feature of independent claim 1 and therefore the claim is allowable. However, Applicant’s argument is not persuasive. As set forth above, Sridharan teaches each and every limitation of claim 1, and accordingly, the rejection of claim 1 under 35 U.S.C. 102 is maintained.
With respect to independent claims 10 and 19, these claims recite limitations similar to those of claim 1 and are rejected for the same reasons as claim 1, as set forth in the Office Action. The remaining dependent claims are not separately patentable for the reasons set forth above and in the Office Action.
For the reasons set forth above and in the Office Action, the rejections of claims 1-20 under 35 U.S.C. 102 and 103 are maintained.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-8, 10-17, 19, and 20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Sridharan et al. (Pub. No.: US 20190205745 A1 (Filed: 2019)).
Regarding claim 1, Sridharan teaches the following limitations:
A training apparatus for a neural network model, wherein the training apparatus is for implementing natural language processing (NLP) training, the training apparatus comprising: computer hardware configured to implement a plurality of accelerators, wherein each accelerator of the plurality of accelerators is configured to (Sridharan [Abstract] “a system to configure distributed training of a neural network…worker nodes configured to perform distributed training of the neural network” paragraph [0143] “The integrated circuit package assembly 1170 illustrates an implementation of one or more processor or accelerator devices as described herein. The package assembly 1170 includes multiple units of hardware logic… implemented at least partly in configurable logic or fixed-functionality logic hardware, and can include one or more portions of any of the processor core(s), graphics processor(s), or other accelerator devices described herein” [0164] “A machine learning application 1502 can be configured to train a neural network using a training dataset…“ [0196] “ Parallel processor accelerated machine learning can also be used to accelerate natural language processing… Exemplary natural language processor applications include automatic machine translation between human languages.” – Sridharan teaches a training apparatus for a neural network model implemented as computer hardware that includes a plurality of accelerator devices (e.g., GPUs). The disclosed worker nodes and accelerator devices correspond to the claimed plurality of accelerators. Sridharan further teaches that each accelerator is configured to perform neural network training operations, which correspond to the training apparatus. Sridharan additionally teaches that parallel processor accelerated machine learning is used for natural language processing.)
store some weight coefficients, wherein the some weight coefficients stored in each accelerator of the plurality of accelerators form a complete weight coefficient (Sridharan, paragraph [0203] “ As shown in FIG. 20B, model parallelism can be implemented in which the model or set of weights is split across multiple nodes.“ Paragraph [0204] “or a layer of a neural network, the input data 2002, weight data 2004, and/or activation data 2006 is partitioned and distributed across multiple compute nodes (e.g., Node 0-Node 3). Paragraph [0202] “An allreduce operation 2005 is used to update the weights of each layer…an allgather operation is used after stochastic gradient descent to synchronize weights across nodes.” [0143] “The integrated circuit package assembly 1170 illustrates an implementation of one or more processor or accelerator devices as described herein…can include one or more portions of any of the processor core(s), graphics processor(s), or other accelerator devices” [0191] “in a distributed system in which each computational node includes multiple GPUs. Each node can have a complete instance of the model with separate GPUs within each node” –Sridharan teaches partitioning the model’s weights such that each accelerator stores only a portion (“some weight coefficients”), with the complete weight coefficient obtained by combining the distributed portions via allgather/reduce operations.)
add the some weight coefficients stored in each accelerator of the plurality of accelerators to obtain the complete weight coefficient (Sridharan, paragraph [0203] “ As shown in FIG. 20B, model parallelism can be implemented in which the model or set of weights is split across multiple nodes.“ Sridharan, paragraph [0202] “An allreduce operation 2005 is used to update the weights of each layer for the next forward pass. In one embodiment, distributed weight update can be enabled in which a reduce_scatter is used calculate an average for gradients before stochastic gradient descent is performed and an allgather operation is used after stochastic gradient descent to synchronize weights across nodes.” [0143] “implementation of one or more processor or accelerator devices as described herein…can include one or more portions of any of the processor core(s), graphics processor(s), or other accelerator devices described herein.” [0191] “in a distributed system in which each computational node includes multiple GPUs. Each node can have a complete instance of the model with separate GPUs within each node” – teaches distributed weight storage across multiple accelerators and then combining those distributed portions using collective communication operations such as allreduce, reduce_scatter, and allgather. These operations aggregate the partial weight data from each accelerator and produce a synchronized, complete set of weights. This directly corresponds to “adding the some weight coefficients stored in each accelerator…to obtain the complete weight coefficient,” because the system retrieves and aggregates partial weight values across accelerators to reconstruct the full weight coefficient.); and
train the neural network model based on input data received by the accelerator and the complete weight coefficient to generate a target model/rule, wherein the target model/rule is configured to implement NLP training, and wherein different input data is received by each different accelerator of the plurality of accelerators (Sridharan, paragraph [0143] “implementation of one or more processor or accelerator devices as described herein…can include one or more portions of any of the processor core(s), graphics processor(s), or other accelerator devices described herein. [0189] “In model parallelism 1902, different computational nodes in a distributed system can perform training computations for different parts of a single network. For example, each layer of a neural network can be trained by a different processing node of the distributed system.” [0190] “In data parallelism 1904, the different nodes of the distributed network have a complete instance of the model and each node receives a different portion of the data.” [0191] “in a distributed system in which each computational node includes multiple GPUs. Each node can have a complete instance of the model with separate GPUs within each node are used to train different portions of the model.” [0173] “Backpropagation of errors is a common method used to train neural networks…The network can then learn from those errors using an algorithm, such as the stochastic gradient descent algorithm, to update the weights of the of the neural network.” [0202] “ An allreduce operation 2005 is used to update the weights of each layer for the next forward pass. In one embodiment, distributed weight update can be enabled in which a reduce_scatter is used calculate an average for gradients before stochastic gradient descent is performed and an allgather operation is used after stochastic gradient descent to synchronize weights across nodes.” [0211] “The applications 2104 include multi-modal fusion and decision-making applications that can process the input to enable machine learning tasks such as image understanding, video summarization, speech and natural language processing” – teaches training a neural network model in a distributed system in which different computational nodes (accelerators) receive different portions of input data and perform training computations, which corresponds to training a neural network model based on input data received by each accelerator, wherein different input data is received by each different accelerator. Sridharan further teaches that training includes updating model weights using backpropagation and stochastic gradient descent. Sridharan additionally teaches that aggregation operations, including allreduce and allgather, are used to synchronize weights across nodes, which corresponds to the complete weight coefficient used in the training. Sridharan further teaches that the trained neural network models are used in applications including natural language processing.)
Regarding claim 2, Sridharan teaches all the elements of claim 1, therefore is rejected for the same reasons as those presented for claim 1. Sridharan further teaches:
calculating gradient information based on the input data received by the given accelerator and the complete weight coefficient (Sridharan, paragraph [0143] “implementation of one or more processor or accelerator devices as described herein…can include one or more portions of any of the processor core(s), graphics processor(s), or other accelerator devices described herein. [0191] “in a distributed system in which each computational node includes multiple GPUs. Each node can have a complete instance of the model with separate GPUs within each node are used to train different portions of the model.” Paragraph [0202] “ An allreduce operation 2005 is used to update the weights of each layer…an allgather operation is used…to synchronize weights across nodes.”);
calculating a target gradient based on gradient information calculated by each accelerator the plurality of accelerators (Sridharan, paragraph [0143] “implementation of one or more processor or accelerator devices as described herein…can include one or more portions of any of the processor core(s), graphics processor(s), or other accelerator devices described herein. [0191] “in a distributed system in which each computational node includes multiple GPUs. Each node can have a complete instance of the model with separate GPUs within each node are used to train different portions of the model.” Paragraph [0202] “a reduce_scatter is used calculate an average for gradients before stochastic gradient descent is performed” – teaches that during training, each accelerator independently computes gradient information for its assigned portion of the mini-batch. Sridharan teaches that these per-accelerator gradients are then combined using a reduce_scatter operation to generate a single aggregated gradient used for the weight update.);
updating the some weight coefficients of the given accelerator based on the target gradient to generate updated some weight coefficients, and training the neural network model based on the updated some weight coefficients of the given accelerator (Sridharan, paragraph [0143] “implementation of one or more processor or accelerator devices as described herein…can include one or more portions of any of the processor core(s), graphics processor(s), or other accelerator devices described herein. [0191] “in a distributed system in which each computational node includes multiple GPUs. Each node can have a complete instance of the model with separate GPUs within each node are used to train different portions of the model.” Paragraph [0173] “Backpropagation of errors is a common method used to train neural networks. An input vector is presented to the network for processing. The output of the network is compared to the desired output using a loss function and an error value is calculated for each of the neurons in the output layer. The error values are then propagated backwards until each neuron has an associated error value which roughly represents its contribution to the original output. The network can then learn from those errors using an algorithm, such as the stochastic gradient descent algorithm, to update the weights of the of the neural network.” – Sridharan teaches that each node, which is an accelerator such as a GPU, computes gradient information for its assigned portion of the mini-batch. Sridharan explains that backpropagation generates gradient values used during training to update the weights of the neural network using a learning algorithm such as stochastic gradient descent.).
Regarding claim 3, Sridharan teaches all the elements of claim 2, therefore is rejected for the same reasons as those presented for claim 2. Sridharan further teaches:
store some initial variables of an optimizer, wherein the some initial variables stored in each accelerator of the plurality of accelerators form a complete initial variable of the optimizer, and the optimizer is configured to update the weight coefficient of the neural network model (Sridharan, paragraph [0143] “implementation of one or more processor or accelerator devices as described herein…can include one or more portions of any of the processor core(s), graphics processor(s), or other accelerator devices described herein. [0191] “in a distributed system in which each computational node includes multiple GPUs. Each node can have a complete instance of the model with separate GPUs within each node are used to train different portions of the model.” Paragraph [0173] “ The learning model describes how to adjust the weights within the model to reduce the output error of the network…The network can then learn from those errors using an algorithm, such as the stochastic gradient descent algorithm, to update the weights of the of the neural network.” [0202] “The mini-batch is split across several compute nodes [accelerator], with each node responsible for computing gradients with respect to all model parameters …An allreduce operation 2005 is used to update the weights of each layer… distributed weight update can be enabled in which a reduce_scatter is used calculate an average for gradients before stochastic gradient descent is performed and an allgather operation is used after stochastic gradient descent to synchronize weights across nodes.” – teaches the system uses an optimization algorithm (e.g., stochastic gradient descent) to update the model’s weight coefficients. The optimizer operates in a distributed manner where each accelerator stores and processes a portion of the optimizer’s required variables. Specifically, the reduce_scatter operation distributes the gradient values across accelerators before the optimizer step, and the allgather operation synchronizes the updated weights afterwards. Under BRI, because each accelerator performs its portion of the optimization locally, each accelerator must store the corresponding portion of the optimizer’s variables (e.g., gradient values used by SGD). These locally held portions constitute the “some initial variables,” which collectively form the complete initial variable set used by the optimizer to update the weight coefficients.)
wherein updating the some weight coefficients of the given accelerator based on the target gradient comprises: processing the target gradient and the some weight coefficients based on the some initial variables to obtain a processed target gradient; and updating the some weight coefficients to generate the updated some weight coefficients based on the processed target gradient (Sridharan, paragraph [0143] “implementation of one or more processor or accelerator devices as described herein…can include one or more portions of any of the processor core(s), graphics processor(s), or other accelerator devices described herein. [0191] “in a distributed system in which each computational node includes multiple GPUs. Each node can have a complete instance of the model with separate GPUs within each node are used to train different portions of the model.” Paragraph [0173] “The learning model describes how to adjust the weights within the model to reduce the output error of the network…The error values are then propagated backwards until each neuron has an associated error value which roughly represents its contribution to the original output. The network can then learn from those errors using an algorithm, such as the stochastic gradient descent algorithm, to update the weights of the of the neural network.” Paragraph [0184] “Errors are then propagated back through the system. The training framework 1804 can adjust to adjust the weights that control the untrained neural network 1806.” [0202] “a reduce_scatter is used calculate an average for gradients before stochastic gradient descent is performed” – the reference explains that once gradients from all accelerators are aggregated using the reduce_scatter operation, the system performs stochastic gradient descent (SGD) as the optimization step that applies algorithm-defined state variables – such as learning rate, momentum, or similar optimizer variables – to the target gradient in order to compute the final update value. This corresponds to processing the target gradient and the some weight coefficients based on the some initial variables to obtain a processed target gradient. Sridharan further states that the processed update value is then applied to adjust each accelerator’s local weights before synchronization, which corresponds to updating the some weight coefficients to generate the updated some weight coefficient based on the processed target gradient.)
Regarding claim 4, Sridharan teaches all the elements of claim 3, therefore is rejected for the same reasons as those presented for claim 3. Sridharan further teaches:
calculating a scalar representation of the target gradient (Sridharan, paragraph [0202] “ a reduce_scatter is used calculate an average for gradients before stochastic gradient descent is performed” [0200] Table 1 defines “REDUCE_SCATTER Element-wise reduction on vector of element, with the resulting vector split into disjoint segments…” – Because REDUCE_SCATTER performs an element-wise reduction, each accelerator necessarily contributes its local gradient values as individual scalar elements into the reduction. The use of these per-element scalar values to perform element-wise reduction constitutes calculating a scalar representation of the target gradient.”);
adding the scalar representation of the target gradient in each accelerator of the plurality of accelerators to obtain a summation result of the target gradient (Sridharan, paragraph [0143] “implementation of one or more processor or accelerator devices as described herein…can include one or more portions of any of the processor core(s), graphics processor(s), or other accelerator devices described herein. [0191] “in a distributed system in which each computational node includes multiple GPUs. Each node can have a complete instance of the model with separate GPUs within each node are used to train different portions of the model.” Paragraph [0202] “An allreduce operation 2005 is used to update the weights…a reduce_scatter is used calculate an average for gradients before stochastic gradient descent is performed and an allgather operation is used after stochastic gradient descent to synchronize weights across nodes.” [0200] Table 1 defines “REDUCE_SCATTER Element-wise reduction on vector of element, with the resulting vector split into disjoint segments, with different segments sent to each process in a group ALLREDUCE A reduce combined with a broadcast, where the outcome of the reduce operation is broadcast to all processes within a group.” – The ALLREDUCE operation of Sridharan performs a reduction that adds the scalar gradient contributions from each accelerator and collects the results. The reduce phase requires adding the gradient values contributed by the accelerator, which produces the summation result of the target gradient as required by the claim.)
calculating a vector representation of the target gradient based on the summation result (Sridharan, paragraph [0200] table 1 defines “REDUCE_SCATTER Element-wise reduction on vector of element, with the resulting vector split into disjoint segments, with different segments sent to each process in a group ALLREDUCE A reduce combined with a broadcast, where the outcome of the reduce operation is broadcast to all processes within a group.” [0202] “a reduce_scatter is used calculate an average for gradients before stochastic gradient descent is performed and an allgather operation is used after stochastic gradient descent to synchronize weights across nodes.” – Sridharan defines the reduce and reduce_scatter operations as acting on a vector of elements, where each element is reduced (summed or averaged) during the reduction phase. The ALLREDUCE operation then broadcasts the “outcome of the reduce operation” as a vector back to the processes. Because each element of this resulting vector is the reduced (summation-based) outcome for the gradient component, the broadcast vector constitutes a vector representation derived from the summation result for each component of the target gradient.); and
processing the vector representation of the target gradient and the some weight coefficients based on the some initial variables, to obtain the processed target gradient (Sridharan, paragraph [0200] “REDUCE_SCATTER Element-wise reduction on vector of element, with the resulting vector split into disjoint segments…” [0202] “In one embodiment, distributed weight update can be enabled in which a reduce_scatter is used calculate an average for gradients before stochastic gradient descent is performed and an allgather operation is used after stochastic gradient descent to synchronize weights across nodes.” - after the vector of reduced gradient values produced in Sridharan, the reference teaches that “stochastic gradient descent is performed.” Stochastic gradient descent uses an aggregated gradient vector together with optimizer state (such as learning rate or similar update variables) to compute the final update value applied to the weights. Thus, Sridharan’s description of performing stochastic gradient descent corresponds to processing the vector representation of the target gradient together with the weight values and the optimizer’s initial variables to obtain the processed target gradient.).
Regarding claim 5, Sridharan teaches all the elements of claim 4, therefore is rejected for the same reasons as those presented for claim 4. Sridharan further teaches:
adding the scalar representation of the target gradient in each accelerator of the plurality of accelerators by using an allreduce operation in collective communication, to obtain the summation result of the target gradient (Sridharan, paragraph [0143] “implementation of one or more processor or accelerator devices as described herein…can include one or more portions of any of the processor core(s), graphics processor(s), or other accelerator devices described herein. [0191] “in a distributed system in which each computational node includes multiple GPUs. Each node can have a complete instance of the model with separate GPUs within each node are used to train different portions of the model.” Paragraph [0200] table 1 defines “ALLREDUCE A reduce combined with a broadcast, where the outcome of the reduce operation is broadcast to all processes within a group.” [0202] “An allreduce operation 2005 is used to update the weights of each layer for the next forward pass.” [0206] “An initial Allreduce operation 2012 is performed for Layer N and a set of Allreduce operations 2011A, 2011B, 2011N are performed to update the weights of each layer for the next forward pass. The Allreduce operations are reduce operations for which the results are broadcast or transferred to the receive buffers of all processes in the communication group.” – Sridharan defines ALLREDUCE as a reduce operation whose result is broadcast, meaning the reduce phase adds the contributed values to produce a single reduced result. Sridharan further teaches that ALLREDUCE operations are used during backpropagation to update weights. Thus, the ALLREDUCE performs the required addition of the scalar gradient values to obtain the summation result of the target gradient.).
Regarding claim 6, Sridharan teaches all the elements of claim 1, therefore is rejected for the same reasons as those presented for claim 1. Sridharan further teaches:
wherein the some weight coefficients comprise weight coefficients assigned to the plurality of accelerators one by one after the complete weight coefficient is evenly divided gradient (Sridharan, paragraph [0143] “implementation of one or more processor or accelerator devices as described herein…can include one or more portions of any of the processor core(s), graphics processor(s), or other accelerator devices described herein. [0191] “in a distributed system in which each computational node includes multiple GPUs. Each node can have a complete instance of the model with separate GPUs within each node are used to train different portions of the model.” Sridharan paragraph [0203] “As shown in FIG. 20B, model parallelism can be implemented in which the model or set of weights is split across multiple nodes.” [0204] “Node 0 receives a first block of input data 2002A and weight data 2004A. Compute operations are performed at Node 0 to generate a first partial activation 2006A. Likewise, Node 1 receives a second block of input data 2002B and weight data 2004B. Compute operations are performed at Node 1 to generate a second partial activation 2006B. Node 2 can perform compute operations on third input data 2002C and weight data 2004C to generate a third partial activation 2006C. Node 3 can perform compute operations on fourth input data 2002D and weight data 2004D to generate a fourth partial activation 2006D.” [0199] “Model parallelism uses the same data for each compute node, with the model split among compute nodes.” - Sridharan teaches model parallelism in which “the model or set of weights is split across multiple nodes,” each node receives its own block of weight data (e.g., 2004A, 2004B, 2004C, 2004D). Dividing the complete weight set into blocks and assign each block to a different node corresponds to evenly dividing the weight coefficients and assigning the resulting portions to the accelerators one by one, as required by the claim.).
Regarding claim 7, Sridharan teaches all the elements of claim 1, therefore is rejected for the same reasons as those presented for claim 1. Sridharan further teaches:
adding, by using an allgather operation in collective communication, the some weight coefficients stored in each accelerator of the plurality of accelerators, to obtain the complete weight coefficient (Sridharan, paragraph [0143] “implementation of one or more processor or accelerator devices as described herein…can include one or more portions of any of the processor core(s), graphics processor(s), or other accelerator devices described herein. [0191] “in a distributed system in which each computational node includes multiple GPUs. Each node can have a complete instance of the model with separate GPUs within each node are used to train different portions of the model.” Paragraph [0202] “an allgather operation is used after stochastic gradient descent to synchronize weights across nodes.” [0203] “In the backward pass an allgather operation is performed to combine strips of gradients computed on each node.” [0206] “ The back propagation 2028 can also include Allgather 2013… communication operations. For the Allgather operation 2013 data is gathered from all tasks and the combined data is distributed to all tasks.“ – Sridharan teaches the use of an allgather operation in collective communication to combine distributed data stored on multiple accelerators. The allgather operation gathers the portions held by each accelerator and distributes the combined result to all nodes. When applied to weight partitions, the allgather operation adds (collects) the weight coefficients stored on each accelerator to reconstruct the complete weight coefficient.).
Regarding claim 8, Sridharan teaches all the elements of claim 2, therefore is rejected for the same reasons as those presented for claim 2. Sridharan further teaches:
calculating, by using a reduce-scatter operation in collective communication, the target gradient based on the gradient information calculated by each accelerator the plurality of accelerators (Sridharan, paragraph [0143] “implementation of one or more processor or accelerator devices as described herein…can include one or more portions of any of the processor core(s), graphics processor(s), or other accelerator devices described herein. [0191] “in a distributed system in which each computational node includes multiple GPUs. Each node can have a complete instance of the model with separate GPUs within each node are used to train different portions of the model.” Paragraph [0200] “REDUCE_SCATTER Element-wise reduction on vector of element, with the resulting vector split into disjoint segments, with different segments sent to each process in a group” [0202] “distributed weight update can be enabled in which a reduce_scatter is used calculate an average for gradients before stochastic gradient descent is performed…” [0206] “A reduce_scatter 2007 operation can be performed to transfer the data in a single communication operation.” – Sridharan defines reduce_scatter as performing an element-wise reduction on values contributed by each accelerator and scattering the reduced results. Sridharan further states that reduce_scatter is used to calculate an average for gradients before the optimizer step. Because the reduce phase combines (reduces) the gradient information computed by each accelerator, this means “calculating the target gradient based on the gradient information calculated by each accelerator…by using a reduce_scatter operation.”).
Regarding claim 10, Sridharan teaches the following limitations:
A training method for a neural network model, wherein the training method is for implementing natural language processing (NLP) training, wherein the training method is applied to a plurality of accelerators included in a training apparatus, the method comprising (Sridharan, [0164] “A machine learning application 1502 can be configured to train a neural network using a training dataset or to use a trained deep neural network to implement machine intelligence. The machine learning application 1502 can include training and inference functionality for a neural network and/or specialized software that can be used to train a neural network before deployment.” Paragraph [0143] “implementation of one or more processor or accelerator devices as described herein…can include one or more portions of any of the processor core(s), graphics processor(s), or other accelerator devices described herein. [0191] “in a distributed system in which each computational node includes multiple GPUs. Each node can have a complete instance of the model with separate GPUs within each node are used to train different portions of the model.” [0211] “ The applications 2104 include multi-modal fusion and decision-making applications that can process the input to enable machine learning tasks such as image understanding, video summarization, speech and natural language processing,” – teaches a training method for a neural network model, where a machine learning application performs training of a neural network using a training dataset. Sridharan further teaches that the training is performed in a distributed system including a plurality of accelerators (e.g., GPUs), which corresponds to the training method being applied to a plurality of accelerators included in a training apparatus. Sridharan additionally teaches that the neural network models are used in applications including natural language processing.)
storing, by each accelerator of the plurality of accelerators, some weight coefficients, wherein the some weight coefficients stored in each accelerator of the plurality of accelerators form a complete weight coefficient (Sridharan, paragraph [0143] “implementation of one or more processor or accelerator devices as described herein…can include one or more portions of any of the processor core(s), graphics processor(s), or other accelerator devices described herein. [0191] “in a distributed system in which each computational node includes multiple GPUs. Each node can have a complete instance of the model with separate GPUs within each node are used to train different portions of the model.” Paragraph [0203] “ As shown in FIG. 20B, model parallelism can be implemented in which the model or set of weights is split across multiple nodes.“ Paragraph [0204] “or a layer of a neural network, the input data 2002, weight data 2004, and/or activation data 2006 is partitioned and distributed across multiple compute nodes (e.g., Node 0-Node 3). Paragraph [0202] “An allreduce operation 2005 is used to update the weights of each layer… an allgather operation is used after stochastic gradient descent to synchronize weights across nodes.” – teaches that the distributed training system is implemented using accelerators such as GPUs. Sridharan teaches partitioning the model’s weights such that each accelerator stores only a portion (“some weight coefficients”), with the complete weight coefficient obtained by combining the distributed portions via allgather/reduce operations.)
aggregating, by each accelerator of the plurality of accelerators, the some weight coefficients stored in each accelerator of the plurality of accelerators to obtain the complete weight coefficient (Sridharan, paragraph [0143] “implementation of one or more processor or accelerator devices as described herein…can include one or more portions of any of the processor core(s), graphics processor(s), or other accelerator devices described herein. [0191] “in a distributed system in which each computational node includes multiple GPUs. Each node can have a complete instance of the model with separate GPUs within each node are used to train different portions of the model.” Paragraph [0203] “ As shown in FIG. 20B, model parallelism can be implemented in which the model or set of weights is split across multiple nodes.“ Sridharan, paragraph [0202] “An allreduce operation 2005 is used to update the weights of each layer for the next forward pass. In one embodiment, distributed weight update can be enabled in which a reduce_scatter is used calculate an average for gradients before stochastic gradient descent is performed and an allgather operation is used after stochastic gradient descent to synchronize weights across nodes.” – teaches distributed weight storage across multiple accelerators and then combining those distributed portions using collective communication operations such as allreduce, reduce_scatter, and allgather. These operations aggregate the partial weight data from each accelerator and produce a synchronized, complete set of weights.); and
training, by each accelerator of the plurality of accelerators, a neural network model based on input data received by the accelerator and the complete weight coefficient by generating a target model/rule, wherein the target model/rule is configured to implement NLP training, and wherein different input data is received by each different accelerator of the plurality of accelerators (Sridharan, paragraph [0143] “implementation of one or more processor or accelerator devices as described herein…can include one or more portions of any of the processor core(s), graphics processor(s), or other accelerator devices described herein. [0189] “In model parallelism 1902, different computational nodes in a distributed system can perform training computations for different parts of a single network. For example, each layer of a neural network can be trained by a different processing node of the distributed system.” [0190] “In data parallelism 1904, the different nodes of the distributed network have a complete instance of the model and each node receives a different portion of the data.” [0191] “in a distributed system in which each computational node includes multiple GPUs. Each node can have a complete instance of the model with separate GPUs within each node are used to train different portions of the model.” [0173] “Backpropagation of errors is a common method used to train neural networks…The network can then learn from those errors using an algorithm, such as the stochastic gradient descent algorithm, to update the weights of the of the neural network.” [0202] “ An allreduce operation 2005 is used to update the weights of each layer for the next forward pass. In one embodiment, distributed weight update can be enabled in which a reduce_scatter is used calculate an average for gradients before stochastic gradient descent is performed and an allgather operation is used after stochastic gradient descent to synchronize weights across nodes.” [0211] “The applications 2104 include multi-modal fusion and decision-making applications that can process the input to enable machine learning tasks such as image understanding, video summarization, speech and natural language processing” – teaches training, by each computational node (accelerator), a neural network model in a distributed system in which different computational nodes receive different portions of input data and perform training computations, which corresponds to training a neural network model based on input data received by each accelerator, wherein different input data is received by each different accelerator. Sridharan further teaches that training includes updating model weights using backpropagation and stochastic gradient descent. Sridharan additionally teaches that aggregation operations, including allreduce and allgather, are used to synchronize weights across nodes, which corresponds to the complete weight coefficient used in the training. Sridharan further teaches that the trained neural network models are used in applications including natural language processing.)
Regarding claim 11, Sridharan teaches all the elements of claim 10, therefore is rejected for the same reasons as those presented for claim 10. The claim recites similar limitations corresponding to claim 2. Therefore, the claim is rejected for similar reasons as claim 2 using similar teachings and rationale.
Regarding claim 12, Sridharan teaches all the elements of claim 11, therefore is rejected for the same reasons as those presented for claim 11. The claim recites similar limitations corresponding to claim 3. Therefore, the claim is rejected for similar reasons as claim 3 using similar teachings and rationale.
Regarding claim 13, Sridharan teaches all the elements of claim 12, therefore is rejected for the same reasons as those presented for claim 12. The claim recites similar limitations corresponding to claim 4. Therefore, the claim is rejected for similar reasons as claim 4 using similar teachings and rationale.
Regarding claim 14, Sridharan teaches all the elements of claim 13, therefore is rejected for the same reasons as those presented for claim 13. The claim recites similar limitations corresponding to claim 5. Therefore, the claim is rejected for similar reasons as claim 5 using similar teachings and rationale.
Regarding claim 15, Sridharan teaches all the elements of claim 10, therefore is rejected for the same reasons as those presented for claim 10. The claim recites similar limitations corresponding to claim 6. Therefore, the claim is rejected for similar reasons as claim 6 using similar teachings and rationale.
Regarding claim 16, Sridharan teaches all the elements of claim 10, therefore is rejected for the same reasons as those presented for claim 10. The claim recites similar limitations corresponding to claim 7. Therefore, the claim is rejected for similar reasons as claim 7 using similar teachings and rationale.
Regarding claim 17, Sridharan teaches all the elements of claim 11, therefore is rejected for the same reasons as those presented for claim 11. The claim recites similar limitations corresponding to claim 8. Therefore, the claim is rejected for similar reasons as claim 8 using similar teachings and rationale.
Regarding claim 19, Sridharan teaches the following limitations:
A chip, comprising: a processor; and a data interface, wherein the processor reads, through the data interface, instructions stored in a memory, to perform a training method for a neural network model, wherein the training method is for implementing natural language processing (NLP) training, wherein the training method is applied to a plurality of accelerators, wherein the plurality of accelerators are included in a training apparatus, and wherein the training method comprises (Sridharan, [0164] “A machine learning application 1502 can be configured to train a neural network using a training dataset or to use a trained deep neural network to implement machine intelligence. The machine learning application 1502 can include training and inference functionality for a neural network and/or specialized software that can be used to train a neural network before deployment.” Paragraph [0143] “implementation of one or more processor or accelerator devices as described herein…can include one or more portions of any of the processor core(s), graphics processor(s), or other accelerator devices described herein. [0191] “in a distributed system in which each computational node includes multiple GPUs. Each node can have a complete instance of the model with separate GPUs within each node are used to train different portions of the model.” [0052] “In one embodiment the memory device 120 can operate as system memory for the system 100, to store data 122 and instructions 121 for use when the one or more processors 102 executes an application or process” [0268] “The external network interface 3018 can be a wireless data interface such as an LTE or GSM interface." [0211] “The applications 2104 include multi-modal fusion and decision-making applications that can process the input to enable machine learning tasks such as image understanding, video summarization, speech and natural language processing,”):
storing some weight coefficients, wherein the some weight coefficients stored in each accelerator of the plurality of accelerators form a complete weight coefficient (Sridharan, Paragraph [0143] “implementation of one or more processor or accelerator devices as described herein…can include one or more portions of any of the processor core(s), graphics processor(s), or other accelerator devices described herein. [0191] “in a distributed system in which each computational node includes multiple GPUs. Each node can have a complete instance of the model with separate GPUs within each node are used to train different portions of the model.” Sridharan, paragraph [0203] “ As shown in FIG. 20B, model parallelism can be implemented in which the model or set of weights is split across multiple nodes.“ Paragraph [0204] “or a layer of a neural network, the input data 2002, weight data 2004, and/or activation data 2006 is partitioned and distributed across multiple compute nodes (e.g., Node 0-Node 3). Paragraph [0202] “An allreduce operation 2005 is used to update the weights of each layer… an allgather operation is used after stochastic gradient descent to synchronize weights across nodes.” – teaches that the distributed training system is implemented using accelerators such as GPUs and partitioning the model’s weights such that each accelerator stores only a portion (“some weight coefficients”), with the complete weight coefficient obtained by combining the distributed portions via allgather/reduce operations.);
adding the some weight coefficients stored in each accelerator of the plurality of accelerators to obtain the complete weight coefficient (Sridharan, Paragraph [0143] “implementation of one or more processor or accelerator devices as described herein…can include one or more portions of any of the processor core(s), graphics processor(s), or other accelerator devices described herein. [0191] “in a distributed system in which each computational node includes multiple GPUs. Each node can have a complete instance of the model with separate GPUs within each node are used to train different portions of the model.” Paragraph [0203] “ As shown in FIG. 20B, model parallelism can be implemented in which the model or set of weights is split across multiple nodes.“ Sridharan, paragraph [0202] “An allreduce operation 2005 is used to update the weights of each layer for the next forward pass. In one embodiment, distributed weight update can be enabled in which a reduce_scatter is used calculate an average for gradients before stochastic gradient descent is performed and an allgather operation is used after stochastic gradient descent to synchronize weights across nodes.” – teaches distributed weight storage across multiple accelerators and then combining those distributed portions using collective communication operations such as allreduce, reduce_scatter, and allgather. These operations aggregate the partial weight data from each accelerator and produce a synchronized, complete set of weights. This directly corresponds to “adding the some weight coefficients stored in each accelerator…to obtain the complete weight coefficient,” because the system retrieves and aggregates partial weight values across accelerators to reconstruct the full weight coefficient.); and
training a neural network model based on input data received by the accelerator and the complete weight coefficient by generating a target model/rule, wherein the target model/rule is configured to implement NLP training, and wherein different input data is received by each different accelerator of the plurality of accelerators (Sridharan, paragraph [0143] “implementation of one or more processor or accelerator devices as described herein…can include one or more portions of any of the processor core(s), graphics processor(s), or other accelerator devices described herein. [0189] “In model parallelism 1902, different computational nodes in a distributed system can perform training computations for different parts of a single network. For example, each layer of a neural network can be trained by a different processing node of the distributed system.” [0190] “In data parallelism 1904, the different nodes of the distributed network have a complete instance of the model and each node receives a different portion of the data.” [0191] “in a distributed system in which each computational node includes multiple GPUs. Each node can have a complete instance of the model with separate GPUs within each node are used to train different portions of the model.” [0173] “Backpropagation of errors is a common method used to train neural networks…The network can then learn from those errors using an algorithm, such as the stochastic gradient descent algorithm, to update the weights of the of the neural network.” [0202] “ An allreduce operation 2005 is used to update the weights of each layer for the next forward pass. In one embodiment, distributed weight update can be enabled in which a reduce_scatter is used calculate an average for gradients before stochastic gradient descent is performed and an allgather operation is used after stochastic gradient descent to synchronize weights across nodes.” [0211] “The applications 2104 include multi-modal fusion and decision-making applications that can process the input to enable machine learning tasks such as image understanding, video summarization, speech and natural language processing” – teaches training, by each computational node (accelerator), a neural network model in a distributed system in which different computational nodes receive different portions of input data and perform training computations, which corresponds to training a neural network model based on input data received by each accelerator, wherein different input data is received by each different accelerator. Sridharan further teaches that training includes updating model weights using backpropagation and stochastic gradient descent. Sridharan additionally teaches that aggregation operations, including allreduce and allgather, are used to synchronize weights across nodes, which corresponds to the complete weight coefficient used in the training. Sridharan further teaches that the trained neural network models are used in applications including natural language processing.).
Regarding claim 20, Sridharan teaches all the elements of claim 1, therefore is rejected for the same reasons as those presented for claim 1. Sridharan further teaches:
wherein the input data is text data (Sridharan, paragraph [0160] “pattern recognition algorithms can be used to generate translated text or perform text to speech and/or speech recognition.” – teaches that machine learning algorithms, including neural networks, operate on data used for tasks such as generating translated text and performing speech recognition. These tasks use text data as input, which corresponds to the claimed input data being text data.).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 9 and 18 are rejected under the 35 U.S.C. 103 as being unpatentable over Sridharan et al., (Pub. No.: US 20190205745 A1 (Filed: 2019)) in view of Lopez (Pub. No.: US 20200125933 A1 (Filed: 2019)).
Regarding claim 9, Sridharan teaches all the elements of claim 3, therefore is rejected for the same reasons as those presented for claim 3. However, Sridharan does not teach but Sridharan in view of Lopez teaches:
wherein the some initial variables comprise initial variables assigned to the plurality of accelerators one by one after the complete initial variable is evenly divided (Sridharan, Paragraph [0143] “implementation of one or more processor or accelerator devices as described herein…can include one or more portions of any of the processor core(s), graphics processor(s), or other accelerator devices described herein. [0191] “in a distributed system in which each computational node includes multiple GPUs. Each node can have a complete instance of the model with separate GPUs within each node are used to train different portions of the model.” Sridharan [0203] “ As shown in FIG. 20B, model parallelism can be implemented in which the model or set of weights is split across multiple nodes.” Lopez, paragraph [0068] “S20 splits the mini-batches at each node into a number of sections X corresponding and equal to the number of nodes. Thus if there are 40 nodes 1 to 40, there are 40 sections 1 to 40 and section 3 corresponds to node 3 etc.” [0069] “The node retains the section of the mini-batch which has the same number as the node and sends all the other sections of the mini-batch to their corresponding nodes.” – Lopez teaches even division of a complete quantity and one-by-one assignment to each accelerator (node). Sridharan teaches distributing model parameters across nodes.).
Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, having Sridharan and Lopez before them, to incorporate Lopez’s teaching of assigning input data to accelerators in an individualized, accelerator-specific manner into the distributed training system of Sridharan. Sridharan describes both model parallelism and data parallelism among multiple accelerators, but does not specify how input data is dispatched to each accelerator. Lopez provides the missing functionality by teaching a CPU-directed mechanism in which input data is sent to different accelerators in a controlled stepwise fashion. One would have been motivated to adopt such an assignment scheme in Sridharan’s framework to manage data distribution more granularly, which is a predictable variation of data-parallel training. Implementing individualized (one-by-one) assignment of training inputs would merely refine the manner in which data is delivered to accelerators without altering the underlying distributed-training architecture.
Regarding claim 18, Sridharan teaches all the elements of claim 12, therefore is rejected for the same reasons as those presented for claim 12. However, Sridharan does not teach but Sridharan in view of Lopez teaches:
wherein the some initial variables comprise initial variables assigned to the plurality of accelerators one by one after the complete initial variable is evenly divided (Lopez, paragraph [0020] introduces a training method across multiple nodes. [0024] “Each node may include memory and processing capability, and is preferably an accelerator, such as a GPU.” [0068] “S20 splits the mini-batches at each node into a number of sections X corresponding and equal to the number of nodes. Thus if there are 40 nodes 1 to 40, there are 40 sections 1 to 40 and section 3 corresponds to node 3 etc.” [0069] “The node retains the section of the mini-batch which has the same number as the node and sends all the other sections of the mini-batch to their corresponding nodes.” Sridharan [0203] “ As shown in FIG. 20B, model parallelism can be implemented in which the model or set of weights is split across multiple nodes.” – Lopez teaches even division of a complete quantity and one-by-one assignment to each accelerator (node). Sridharan teaches distributing model parameters across nodes.).
Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, having Sridharan and Lopez before them, to incorporate Lopez’s teaching of assigning input data to accelerators in an individualized, accelerator-specific manner into the distributed training system of Sridharan. Sridharan describes both model parallelism and data parallelism among multiple accelerators, but does not specify how input data is dispatched to each accelerator. Lopez provides the missing functionality by teaching a CPU-directed mechanism in which input data is sent to different accelerators in a controlled stepwise fashion. One would have been motivated to adopt such an assignment scheme in Sridharan’s framework to manage data distribution more granularly, which is a predictable variation of data-parallel training. Implementing individualized (one-by-one) assignment of training inputs would merely refine the manner in which data is delivered to accelerators without altering the underlying distributed-training architecture.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action. Any inquiry concerning this communication or earlier communications from the examiner should be directed to Daravanh Phakousonh whose telephone number is (571)272-6324. The examiner can normally be reached Mon - Thurs 7 AM - 5 PM, Every other Friday 7 AM - 4PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Li B Zhen can be reached at 571-272-3768. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Daravanh Phakousonh/Examiner, Art Unit 2121
/Li B. Zhen/Supervisory Patent Examiner, Art Unit 2121
Prosecution Insights