Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
This action is in response to the amendments filed 03/27/2026. Claims 1-20 are currently pending.
Response to Arguments
Applicant’s arguments regarding the 101 rejection have been fully considered but they are not persuasive. Applicant argues on pages 10-11 that the claim limitation “generating N first gradient sets in a first BP calculation in a jth iteration of N deep learning models” does not recite a mathematical calculation, does not recite an abstract idea, is instead at most merely based on or involve mathematical calculations, and compares this limitation to the published USPTO Example 39. Examiner respectfully disagrees and notes that in contrast to Example 39, which recites “training the neural network”, Applicant’s claim explicitly recites “generating N first gradient sets in a first BP (i.e., backpropagation) calculation”. Examiner notes that section 2106.04(a)(2)(I)(C) of the MPEP states “a step of "determining" a variable or number using mathematical methods or "performing" a mathematical operation may also be considered mathematical calculations when the broadest reasonable interpretation of the claim in light of the specification encompasses a mathematical calculation”. Given the description of forward and back propagation calculations in at least paragraphs [0073]-[0081] and [0085]-[0088] as mathematical calculations, the broadest reasonable interpretation of this limitation includes a mathematical calculation.
Applicant also argues on pages 11-13 that the limitations directed to “adjusting a communication sequence of gradients. . .” and “correcting a parameter matrix. . .” improve a training system by reducing an iteration time of the deep learning model and increasing iteration efficiency of the deep learning model, and therefore integrate any claimed judicial exceptions into a practical application. Examiner respectfully disagrees and notes that these limitations were interpreted as judicial exceptions. Section 2106.05(a) of the MPEP states “the judicial exception alone cannot provide the improvement. . .the improvement can be provided by the additional element(s) in combination with the recited judicial exception”. Applicant has not clearly shown which additional elements, either on their own or in combination with claimed judicial exceptions, provide this improvement to integrate the claimed judicial exceptions into a practical application or amount to significantly more. Therefore, these arguments are considered unpersuasive and the rejection is maintained. The 101 rejections have been updated to include the amended limitations and to clarify any reasoning given for the limitations that were not amended where necessary.
Applicant’s arguments regarding the prior art rejection have been fully considered but they are not persuasive. Applicant argues on pages 14-15 that the combination of Que and Cui fails to disclose “sending, according to an adjusted communication sequence, gradients to a parameter storage space of a training system so that a last gradient on a last neuron at a last layer is sent within a forward propagation (FP) calculation time period after a first gradient”. Examiner respectfully disagrees and notes that, as Applicant’s disclosure does not further define “a forward propagation (FP) calculation time period after a first gradient”, the broadest reasonable interpretation of this time period includes the embodiment from at least fig. 5 and Section II B of Que wherein gradients are sent such that a gradient from a last layer is sent after a gradient from a first layer within a calculation time associated with a forward propagation. Therefore, these arguments are considered unpersuasive and the rejection is maintained. The prior art rejections have been updated to include the amended limitations and to clarify any reasoning given for the limitations that were not amended where necessary.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101. Claims 1-7 are directed to a method, claims 8-14 are directed to a system, and claims 15-20 are directed to a non-transitory computer-readable medium; therefore, claims 1-20 fall within one of the four statutory categories (i.e., process, machine, manufacture, or composition of matter). However, claims 1-20 fall within the judicial exception of an abstract idea, specifically the abstract ideas of “Mental Processes” (including observation, evaluation, and opinion) and “Mathematical Concepts (including mathematical calculations and relationships)”.
Claim 1:
Claim 1 is directed to a method; therefore, the claim does fall within one of the four statutory categories (i.e., process, machine, manufacture, or composition of matter).
Claim 1 recites the following abstract ideas:
wherein a training process of each of the N deep learning models comprises a plurality of iterations, wherein each of the iterations comprises a forward propagation (FP) calculation and a back propagation (BP) calculation, wherein N is a positive integer greater than 1, wherein n is a positive integer greater than 1 (a training process comprising a forward propagation calculation and a back propagation calculation is interpreted as a mathematical calculation);
generating N first gradient sets in a first BP calculation in a j-th iteration of the N deep learning models, wherein each of the N first gradient sets comprises a first gradient corresponding to a first parameter matrix of a first neuron at each of the n layers of one deep learning model of the N deep learning models, and wherein j is a positive integer greater than zero (generating a gradient set comprising a gradient corresponding to a parameter matrix from a backward propagation calculation is interpreted as a mathematical calculation. Wherein the gradient is generated by a deep learning model corresponding to a neuron of a layer of the model is interpreted as applying an abstract idea directed to a mathematical calculation using a generic computer component);
adjusting a communication sequence of gradients comprised in each of the N first gradient sets to obtain an adjusted communication sequence (mental step directed to observation, evaluation – a person could adjust a communication sequence of observed gradients in their mind to mentally determine the order in which those gradients should be sent to a storage parameter space);
correcting a second parameter matrix of a second neuron at each of the n layers of neurons of each of the N deep learning models based on a second gradient comprised in the second gradient set to perform a first FP calculation in a (j + 1)th iteration on each of the N deep learning models (correcting a second parameter matrix corresponding to a second neuron in a deep learning model by performing a forward propagation calculation is interpreted as a mathematical calculation. Wherein the forward propagation calculation is performed by a deep learning model is interpreted as applying an abstract idea directed to a mathematical calculation using a generic computer component).
Claim 1 recites the following additional elements:
a training system, wherein the training system comprises at least one model training server, wherein one model training is used as one computing node, wherein the training system comprises N deep learning models, wherein each of the N deep learning models comprises n layers of neurons; sending, according to the adjusted communication sequence, the gradients to a parameter storage space of the training system so that a last gradient on a last neuron at a last layer is sent within a FP calculation time period after the first gradient; and obtaining a second gradient set based on the N first gradient sets that are stored in the parameter storage space, wherein the second gradient set comprises an average value of the first gradient from each of the N first gradient sets. The training system, model training server, computing nodes, and deep learning models are interpreted as generic computer components used to perform the abstract ideas listed in the previous step; sending the gradients according to an adjusted communication so that a last gradient on a last neuron at a last layer is sent within a FP calculation time is interpreted as merely implementing the mental step of adjusting the communication sequence using a generic computer and transmitting data over a network. Obtaining a second gradient set stored in a parameter storage space is interpreted as retrieving information from memory, wherein the second gradient set comprises an average value of the first gradient sets is interpreted as a further description of the type of data being retrieved from memory. These additional elements do not integrate the abstract idea into a practical application or amount to significantly more than the abstract idea (see MPEP 2106.05(d)(II) and MPEP 2106.05(f)).
Claim 8 is a system claim and its limitation is included in claim 1. The only difference is that claim 8 requires a system. Therefore, claim 8 is rejected for the same reasons as claim 1.
Claim 15 is a non-transitory computer readable medium claim and its limitation is included in claim 1. The only difference is that claim 15 requires a non-transitory computer readable medium. Therefore, claim 15 is rejected for the same reasons as claim 1.
The independent claims are not patent eligible. Dependent claims 2-7, 9-14, and 16-20 when analyzed as a whole are held to be patent ineligible under 35 U.S.C. 101 because the additional recited limitations fail to establish that the claims are not directed to an abstract idea, as they recite further embellishment of the judicial exception.
Claim 2 recites wherein adjusting the communication sequence comprises adjusting a first sequence of sending a third gradient corresponding to a third parameter matrix of a third neuron at an a-th layer to the parameter storage space to be before a second sequence of sending a fourth gradient corresponding to a fourth parameter matrix of a fourth neuron at a b-th layer to the parameter storage space, wherein b is less than or equal to n, wherein a is less than b, and wherein a is a positive integer greater than zero. Adjusting a sequence of gradients such that a third gradient is sent before a second sequence is interpreted as an additional element directed to organizing data to be transmitted over a network and does not integrate the abstract ideas in claim 1 into a practical application or amount to significantly more than the abstract ideas in claim 1 (see MPEP 2106.05(d)(II)).
Claim 3 recites wherein adjusting the communication sequence comprises adjusting the communication sequence according to a gradient communication policy. Adjusting a communication sequence according to a gradient communication policy is interpreted as an additional element directed to organizing data to be transmitted over a network and does not integrate the abstract ideas in claim 1 into a practical application or amount to significantly more than the abstract ideas in claim 1 (see MPEP 2106.05(d)(II)).
Claim 4 recites setting the gradient communication policy based on a communication bandwidth between the one deep learning model and the parameter storage space. Setting a communication policy based on a communication bandwidth is interpreted as a mental step directed to observation, judgement – a person could decide in their mind to set a gradient communication policy based on an observed communication bandwidth.
Claim 5 recites setting the gradient communication policy based on a value of the first gradient. Setting a communication policy based on a value of a first gradient is interpreted as a mental step directed to observation, judgement – a person could decide in their mind to set a gradient communication policy based on an observed value of a first gradient.
Claim 6 recites setting the gradient communication policy based on a time required by the first neuron during a second FP calculation. Setting a communication policy based on a time required during a forward propagation calculation is interpreted as a mental step directed to observation, judgement – a person could decide in their mind to set a gradient communication policy based on an observed time required during a forward propagation calculation.
Claim 7 wherein adjusting the communication sequence comprises: adjusting the communication sequence according to a gradient communication policy; obtaining an iteration time of the one deep learning model, wherein the iteration time comprises a first time for a second BP calculation in an Lth iteration of the one deep learning model and a second time for a third FP calculation in an (L + 1)th iteration of the one deep learning model, and wherein L is a positive integer greater than j; and adjusting the gradient communication policy based on the iteration time. Adjusting a communication sequence according to a gradient communication policy and adjusting a gradient communication policy based on an iteration time are interpreted as additional elements directed to organizing data to be transmitted over a network. Obtaining an iteration time of a deep learning model comprising a time for a backward propagation and a forward propagation calculation is interpreted as receiving data over a network. These addition elements do not integrate the abstract ideas in claim 1 into a practical application or amount to significantly more than the abstract ideas in claim 1 (see MPEP 2106.05(d)(II)).
Claim 9 is a system claim and its limitation is included in claim 2. Claim 9 is rejected for the same reasons as claim 2.
Claim 10 is a system claim and its limitation is included in claim 3. Claim 10 is rejected for the same reasons as claim 3.
Claim 11 is a system claim and its limitation is included in claim 4. Claim 11 is rejected for the same reasons as claim 4.
Claim 12 is a system claim and its limitation is included in claim 5. Claim 12 is rejected for the same reasons as claim 5.
Claim 13 is a system claim and its limitation is included in claim 6. Claim 13 is rejected for the same reasons as claim 6.
Claim 14 is a system claim and its limitation is included in claim 7. Claim 14 is rejected for the same reasons as claim 7.
Claim 16 is a non-transitory computer readable medium claim and its limitation is included in claim 2. Claim 16 is rejected for the same reasons as claim 2.
Claim 17 is a non-transitory computer readable medium claim and its limitation is included in claim 3. Claim 17 is rejected for the same reasons as claim 3.
Claim 18 is a non-transitory computer readable medium claim and its limitation is included in claim 4. Claim 18 is rejected for the same reasons as claim 4.
Claim 19 is a non-transitory computer readable medium claim and its limitation is included in claims 5 and 6. Claim 19 is rejected for the same reasons as claims 5 and 6.
Claim 20 is a non-transitory computer readable medium claim and its limitation is included in claim 7. Claim 20 is rejected for the same reasons as claim 7.
Viewed as a whole, these additional claim elements do not provide meaningful limitations to transform the abstract idea into a patent eligible application of the abstract idea such that the claims amount to significantly more than the abstract idea itself. Therefore, the claims are rejected under 35 U.S.C. 101 as being directed to non-statutory subject matter.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Que et al (“Efficient Scheduling in Training Deep Convolutional Networks at Large Scale”, herein Que), in view of Cui et al (US 20200042362 A1, herein Cui).
Regarding claim 1, Que teaches a method applied to a training system, [wherein the training system comprises at least one model training server, wherein one model training is used as one computing node], the training system comprises N deep learning models, wherein each of the N deep learning models comprises n layers of neurons, wherein a training process of each of the N deep learning models comprises a plurality of iterations, wherein each of the iterations comprises a forward propagation (FP) calculation and a back propagation (BP) calculation, wherein N is a positive integer greater than 1, wherein n is a positive integer greater than 1 (section II para. 2 recites “Let N be the number of workers in a distributed training system, and the mini-batch size of each worker node is b. The total batch size of the training system is B = N x b. In a training iteration, each worker calculates gradients by performing forwarding and backpropagation on a set randomly selected b samples” (i.e., a number of distributed models, or workers, performing training iterations comprised of forward and back propagations)), and wherein the method comprises:
generating N first gradient sets in a first BP calculation in a j-th iteration of the N deep learning models, wherein each of the N first gradient sets comprises a first gradient corresponding to a first parameter matrix of a first neuron at each of the n layers of one deep learning model of the N deep learning models, and wherein j is a positive integer greater than zero (section II para. 1 recites “Training neural networks usually consists of three steps: forwarding, backpropagation and weight update. In the forwarding step, the activations, which are the outputs of neurons, are calculated layer by layer. Then, in backpropagation, the error terms are calculated and propagated in an inverse order. Using the error terms, we can calculate the gradients of weights at each layer. At the end of a training iteration, the parameter weights are updated as following: (EQ1) where Wi is the weight of parameters in iteration i, α is the learning rate, and ∆W is the gradient of the parameters”. Figure 3 depicts the order of back propagation in a neural network model (i.e., a back propagation step which generates gradients for a neuron in each layer of a model));
adjusting a communication sequence of gradients comprised in each of the N first gradient sets to obtain an adjusted communication sequence; sending, according to the adjusted communication sequence, the gradients to a parameter storage space of the training system so that a last gradient on a last neuron at a last layer is sent within a FP calculation time period after the first gradient (section II B para. 1-2 recite “As shown in Fig. 3, the order of AllReduce call follows backpropagation sequence from Softmax - Conv3 - Conv2 to Conv1. Since the order of backpropagation is reverse to forwarding, worker nodes always get what they need to start forwarding in the end of transmission, which causes delay and affects training performance when communication speed becomes bottleneck. To solve this problem, we break the AllReduce call into two separated routings, reducing and broadcasting. In the reducing procedure, worker nodes aggregate their gradients to root nodes. After receiving reduced gradients from workers, the root nodes cache the reduced gradients and schedule the order of broadcasting to minimize delay at worker nodes (i.e., caching, or storing the gradients). For example in Fig. 3, the integrated version of AllReduce broadcasts the reduced gradients from Softmax to Conv1. In contrast, our separated AllReduce can cache the reduced gradients and reschedule the broadcasting order from Conv1 to Softmax, thus reduces communication delay by helping the worker nodes to get what they need in the beginning of transmission”. Fig. 5 and section II B para. 7 recite “The reduced gradients of first two layers [0, 1] are sent in the backpropagation time of ith iteration, while gradients of the last four layers [2, 3, 4, 5] are sent sequentially in the forwarding time of (i + 1)th iteration. The scheduling proposal gets performance gain by overlapping broadcasting of reduced gradients [3, 4, 5] with forwarding computations in (i + 1)th iteration. Moreover, since broadcasting of gradients [3, 4, 5] is moved from ith iteration to (i + 1)th iteration, the reducing speed of ith iteration increases due to less contentions in this stage” (i.e., adjusting a communication sequence such that the last gradients are sent and stored in an adjusted order within the time required before a next forward propagation ends, or a FP calculation time period after a first gradient is received));
and correcting a second parameter matrix of a second neuron at each of the n layers of neurons of each of the N deep learning models based on a second gradient comprised in the second gradient set to perform a first FP calculation in a (j + 1)th iteration on each of the N deep learning models (section II para. 2 recites “In a training iteration, each worker calculates gradients by performing forwarding and backpropagation on a set randomly selected b samples. The gradients at each node are then accumulated and synchronized via AllReduce. Finally, the worker nodes calculate a new version of parameters based on the reduced gradients using (EQ1)”. Section II C para. 3 recites “After collecting information about slow nodes in the system, the turn-round algorithm decides whether to increase or decrease the value of M (i.e., the turn-around index) to reduce delay. For example, if the slow node is blocked by layers with indices smaller than M, we need to decrease M to reduce contentions in the backpropagation pass. Otherwise, we increase M to reduce communication loads in the forwarding pass” (i.e., based on an obtained new, or second gradient set, correcting, or updating parameters of a neuron in a next layer and performing forward propagation in a next iteration of the neural network model)).
However, while Que teaches that use of a parameter server with computing nodes is known the art (see at least section I para. 5), Que does not explicitly teach how its AllReduce algorithms would be applied to a training system such as a parameter server or model training server, wherein one model training is used as one computing node; and obtaining a second gradient set based on the N first gradient sets that are stored in the parameter storage space, wherein the second gradient set comprises an average value of the first gradient from each of the N first gradient sets.
Cui teaches wherein the training system comprises at least one model training server, wherein one model training is used as one computing node (para. [0005] recites “a method includes provisioning a plurality of accelerator resources on one or more server nodes of a computing system to execute a distributed deep learning model training process to train a deep learning model”. Para. [0016] recites “A distributed SGD DL (i.e., stochastic gradient descent deep learning) training process can be implemented in an HPC system using a data-parallel programming model in which the SGD training process is executed in parallel by a plurality of worker nodes executing worker processes (e.g., accelerator resources such as GPU resources) that are distributed over one or more compute nodes of the HPC system” (i.e., a model training server with a plurality of computing nodes)); and obtaining a second gradient set based on the N first gradient sets that are stored in the parameter storage space, wherein the second gradient set comprises an average value of the first gradient from each of the N first gradient sets (para. [0017] recites “In data parallel training, for each iteration of a backpropagation process, a mini-batch of data samples is partitioned and evenly distributed to a plurality of worker nodes, which can reside on the same or different server machines. The worker nodes perform the forward and backward propagation operations on their respective subsets of a given mini-batch dataset in parallel. The gradient parameters computed by all worker nodes for the given iteration are then aggregated/synchronized (e.g. averaged) and the averaged gradient parameters are pushed to each worker node so that each worker node can perform a parameter update process using the averaged gradient parameters to update the model parameters of the DL network model” (i.e., obtaining an updated, or second gradient based on the calculated average value of the first, or initial gradients from the plurality of nodes)).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine these teachings by modifying the method of updating node parameters from Que with the gradient averaging calculation technique from Cui. Que and Cui are both directed to distributed learning systems; therefore, it would be obvious to one of ordinary skill in the art that the known gradient averaging technique from Cui could be substituted for the known parameter update method from Que to yield a predictable result.
Regarding claim 2, the combination of Que and Cui teaches the method of claim 1, wherein adjusting the communication sequence comprises adjusting a first sequence of sending a third gradient corresponding to a third parameter matrix of a third neuron at an a-th layer to the parameter storage space to be before a second sequence of sending a fourth gradient corresponding to a fourth parameter matrix of a fourth neuron at a b-th layer to the parameter storage space, wherein b is less than or equal to n, wherein a is less than b, and wherein a is a positive integer greater than zero (Que section 1 para. 9 recites “we propose a novel algorithm which reduces communication delay and link contentions by jointly scheduling the priority of parameters and overlapping communication with both backpropagation and forwarding computations. Fig. 3 shows a simple convolutional neural networks with four layers. In current AllReduce implementations, the gradient updates are sent in the order of AllReduce call, which is the backward order from Softmax - Conv3 - Conv2 to Conv1. The problem is that, in a worker node, even gradient updates of the Softmax layer are received, the node cannot start to compute until it receives outputs from Conv3. On the contrary, Conv1's forwarding computation can be started immediately when the updated gradients of Conv1 are received, without waiting for Conv2 or other layers. In addition, the communication of gradient updates in subsequent layers such as Conv2 and Conv3 can be overlapped with the forwarding computation of Conv1” (i.e., adjusting a gradient communication sequence such that a gradient in a previous, or a-th, layer is sent and stored before a gradient in a later, or b-th, layer, wherein b > a)).
Regarding claim 3, the combination of Que and Cui teaches the method of claim 1, wherein adjusting the communication sequence comprises adjusting the communication sequence according to a gradient communication policy (Que section II B para. 3-4 recites “Formally, we formulate the reduced gradients to be broadcast as a vector [0, 1, . . ., K – 1], where the index k of the array stands for network forwarding order executed by solvers like Caffe. The order of gradients arriving at root node is reverse to forwarding, which is [K – 1, K – 2, . . ., 0]. The target of our scheduling algorithm is to find an optimal transmission order S D [s0, s1, . . ., sK–1], where sk is the time to broadcast layer k. Theoretically, we can build a model to calculate the optimal value of S if we know all the hardware configurations, network structures, batch sizes, etc.” (i.e., adjusting the communication sequence according to an optimal transmission order, or policy)).
Regarding claim 4, the combination of Que and Cui teaches the method of claim 3, further comprising setting the gradient communication policy based on a communication bandwidth between the one deep learning model and the parameter storage space (Que section II B para. 5-7 recite “To schedule packet transmissions and reduce delay, our algorithm tries to find a turn-round index M in the array, where gradients before M are broadcast in backpropagation order from M – 1 to 0 and gradients behind M are broadcast in forwarding order from M to K – 1. The algorithm overlaps broadcasting from M – 1 to 0 with backpropagation computations in the i-th iteration, while broadcasting from M to K – 1 is overlapped with forwarding computations in the (i + 1)th iteration. Intuitively, if M equals K, the turn-round algorithm falls back to the original AllReduce implementation and overlaps all the communications with only backpropagation computations. On the other hand, if M equals 0, the algorithm sequentially broadcasts all the reduced gradients in the forwarding pass. When a new version of reduced gradient is received by root node, if its index is less than M, it is broadcast immediately by the root node. Otherwise, the gradient is buffered in the root node, and the buffered gradients are sequentially sent out after the reduced gradients of layer 0 are received” (i.e., setting a communication policy for gradients based on the communication ability, or bandwidth, between the model and the cache, or parameter storage)).
Regarding claim 5, the combination of Que and Cui teaches the method of claim 3, further comprising setting the gradient communication policy based on a value of the first gradient (Que section II B para. 5-7 recite “To schedule packet transmissions and reduce delay, our algorithm tries to find a turn-round index M in the array, where gradients before M are broadcast in backpropagation order from M – 1 to 0 and gradients behind M are broadcast in forwarding order from M to K – 1. The algorithm overlaps broadcasting from M – 1 to 0 with backpropagation computations in the i-th iteration, while broadcasting from M to K – 1 is overlapped with forwarding computations in the (i + 1)th iteration. Intuitively, if M equals K, the turn-round algorithm falls back to the original AllReduce implementation and overlaps all the communications with only backpropagation computations. On the other hand, if M equals 0, the algorithm sequentially broadcasts all the reduced gradients in the forwarding pass. When a new version of reduced gradient is received by root node, if its index is less than M, it is broadcast immediately by the root node. Otherwise, the gradient is buffered in the root node, and the buffered gradients are sequentially sent out after the reduced gradients of layer 0 are received”. Section II C para. 3 recites “After collecting information about slow nodes in the system, the turn-round algorithm decides whether to increase or decrease the value of M to reduce delay. For example, if the slow node is blocked by layers with indices smaller than M, we need to decrease M to reduce contentions in the backpropagation pass. Otherwise, we increase M to reduce communication loads in the forwarding pass” (i.e., setting a communication policy for gradients based on the values of gradients for a given layer being computed too slowly)).
Regarding claim 6, the combination of Que and Cui teaches the method of claim 3, further comprising setting the gradient communication policy based on a time required by the first neuron during a second FP calculation (Que section II B para. 3-4 recites “Formally, we formulate the reduced gradients to be broadcast as a vector [0, 1, . . ., K – 1], where the index k of the array stands for network forwarding order executed by solvers like Caffe. The order of gradients arriving at root node is reverse to forwarding, which is [K – 1, K – 2, . . ., 0]. The target of our scheduling algorithm is to find an optimal transmission order S D [s0, s1, . . ., sK–1], where sk is the time to broadcast layer k. Theoretically, we can build a model to calculate the optimal value of S if we know all the hardware configurations, network structures, batch sizes, etc.” (i.e., adjusting the communication sequence based on the time required to broadcast the neurons in a layer associated with a given forward propagation iteration)).
Regarding claim 7, the combination of Que and Cui teaches the method of claim 1, wherein adjusting the communication sequence comprises: adjusting the communication sequence according to a gradient communication policy (Que section II B para. 1-2 recite “As shown in Fig. 3, the order of AllReduce call follows backpropagation sequence from Softmax - Conv3 - Conv2 to Conv1. Since the order of backpropagation is reverse to forwarding, worker nodes always get what they need to start forwarding in the end of transmission, which causes delay and affects training performance when communication speed becomes bottleneck. To solve this problem, we break the AllReduce call into two separated routings, reducing and broadcasting. In the reducing procedure, worker nodes aggregate their gradients to root nodes. After receiving reduced gradients from workers, the root nodes cache the reduced gradients and schedule the order of broadcasting to minimize delay at worker nodes (i.e., caching, or storing the gradients). For example in Fig. 3, the integrated version of AllReduce broadcasts the reduced gradients from Softmax to Conv1. In contrast, our separated AllReduce can cache the reduced gradients and reschedule the broadcasting order from Conv1 to Softmax, thus reduces communication delay by helping the worker nodes to get what they need in the beginning of transmission (i.e., adjusting a communication sequence according to a policy to reduce communication delay));
obtaining an iteration time of the one deep learning model, wherein the iteration time comprises a first time for a second BP calculation in an L-th iteration of the one deep learning model and a second time for a third FP calculation in an (L + 1)th iteration of the one deep learning model, and wherein L is a positive integer greater than j; and adjusting the gradient communication policy based on the iteration time (Que section II B para. 3-4 recites “Formally, we formulate the reduced gradients to be broadcast as a vector [0, 1, . . ., K – 1], where the index k of the array stands for network forwarding order executed by solvers like Caffe. The order of gradients arriving at root node is reverse to forwarding, which is [K – 1, K – 2, . . ., 0]. The target of our scheduling algorithm is to find an optimal transmission order S D [s0, s1, . . ., sK–1], where sk is the time to broadcast layer k. Theoretically, we can build a model to calculate the optimal value of S if we know all the hardware configurations, network structures, batch sizes, etc.” (i.e., adjusting the communication sequence based on iteration time associated with a given layer of the neural network)).
Claim 8 is a system claim and its limitation is included in claim 1. The only difference is that claim 8 requires a system. Therefore, claim 8 is rejected for the same reasons as claim 1.
Claim 9 is a system claim and its limitation is included in claim 2. Claim 9 is rejected for the same reasons as claim 2.
Claim 10 is a system claim and its limitation is included in claim 3. Claim 10 is rejected for the same reasons as claim 3.
Claim 11 is a system claim and its limitation is included in claim 4. Claim 11 is rejected for the same reasons as claim 4.
Claim 12 is a system claim and its limitation is included in claim 5. Claim 12 is rejected for the same reasons as claim 5.
Claim 13 is a system claim and its limitation is included in claim 6. Claim 13 is rejected for the same reasons as claim 6.
Claim 14 is a system claim and its limitation is included in claim 7. Claim 14 is rejected for the same reasons as claim 7.
Claim 15 is a non-transitory computer readable medium claim and its limitation is included in claim 1. The only difference is that claim 15 requires a non-transitory computer readable medium. Therefore, claim 15 is rejected for the same reasons as claim 1.
Claim 16 is a non-transitory computer readable medium claim and its limitation is included in claim 2. Claim 16 is rejected for the same reasons as claim 2.
Claim 17 is a non-transitory computer readable medium claim and its limitation is included in claim 3. Claim 17 is rejected for the same reasons as claim 3.
Claim 18 is a non-transitory computer readable medium claim and its limitation is included in claim 4. Claim 18 is rejected for the same reasons as claim 4.
Claim 19 is a non-transitory computer readable medium claim and its limitation is included in claims 5 and 6. Claim 19 is rejected for the same reasons as claims 5 and 6.
Claim 20 is a non-transitory computer readable medium claim and its limitation is included in claim 7. Claim 20 is rejected for the same reasons as claim 7.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20190258924 A1 (Hamidouche et al) teaches a method for optimizing parameter updates for an asynchronous stochastic gradient descent training process.
“LAG: Lazily Aggregated Gradient for Communication-Efficient Distributed Learning” (Chen et al) teaches a method for detecting slowly-varying gradients and adaptively skipping outdated gradient in a distributed machine learning system.
“Asynchronous Decentralized Parallel Stochastic Gradient Descent” (Lian et al) teaches a method for asynchronous decentralized stochastic gradient decent wherein workers do not wait for all others and only communicate in a decentralized fashion.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LEAH M FEITL whose telephone number is (571) 272-8350. The examiner can normally be reached on M-F 0900-1700 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Viker Lamardo can be reached on (571) 270-5871. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/L.M.F./ Examiner, Art Unit 2147
/VIKER A LAMARDO/Supervisory Patent Examiner, Art Unit 2147
Prosecution Insights