DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This office action is in response to the amendment filed on 07/16/2025. Claims 1-7, 9-12, and 14-20 remain pending in the application. Claims 1, 12, and 16 are independent.
Claim Objections
Applicant's amendment to claims corrects some of previous objections; therefore, some of previous objections are withdrawn. The remaining objection is raised in 112 rejection instead because Applicant does not add back a required limitation "decreasing the layer counter" in Claim 16, which is in the claim set filed on 08/21/2024 but missing in the claim set filed on 02/21/2025 without explicitly expressing to cross-out from the previous claim set.
Claim Rejections - 35 USC § 112
Applicant's amendment to claims corrects previous rejections; therefore, the previous rejections are withdrawn. Since Applicant does not address a part of concerned raised in the previous Claim objection, the following 112 rejection is raised instead.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 16-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim16 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being incomplete for omitting essential elements, such omission amounting to a gap between the elements. See MPEP § 2172.01. The omitted elements are: "decreasing/increasing/changing the layer counter" as part of iteratively performing operations because Claim 16 recites "… initializing a layer counter that represents a set of final layers in the neural network; until stop condition is met, iteratively performing operations comprising: applying the batch of input data to a number of layers associated with the layer counter to perform the forward-propagation; calculating a prediction error; comparing the prediction error to a threshold; using the back-propagation circuit to perform the back-propagation with at least some of the storage elements repurposed and distribute the prediction error to a layer that precedes a last layer of the number of layers; and resuming with receiving a next batch of input data and initializing the layer counter", and as currently claimed in Claim 16, the layer counter is not changed during iteratively performing operations so the number of layers are not changed during iteratively performing operations as well; and therefore it is unclear why there is a need to initialize the layer counter again for next batch of input data since the layer counter is not changed at all during the iteratively performing operations for the batch of input data.
Claims 17-20 are rejected for fully incorporating the deficiency of their respective base claims.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-7, 11-12, and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Goulding et al. (US 2019/0147342 A1, pub. date on 05/16/2019), hereinafter Goulding in view of Shirahata et al. ("Memory reduction method for deep neural network training", 2016 IEEE 26th International Workshop on Machine Learning for Signal Processing, Sept. 2016), hereinafter Shirahata.
Independent Claims 1 and 12
Goulding discloses a storage-efficient method for on-device machine learning (Goulding, ¶¶ [0012]-[0017] and [0019]-[0020] with FIG. 1: an artificial neural network (ANN) include a network of processing elements (PEs) with weight values associated with the interconnections between the PEs stored in a local or remote memory; the PE's computations can be performed and the results stored locally; implement ANNs on a larger scale than achievable via a purely local network of PEs and local memory; efficiently using SRAM and reducing use of DRAM; implement the DNN using a local and/or external memory and compute resources; the DNN 195 may include a plurality of PEs 165 and local static random access memory (SRAM) 170; the co-occurrence of the forward calculation and the backward propagation can help increase the use of limited FPGA memory resources, reduce power consumption, and reduce time to make a prediction or train a DNN; Interleaved backpropagation is a DNN (on-chip) machine learning method that can help reduce energy use in a system on chip (SoC) processor; only a portion of the DNN is implemented at any one time in an FPGA accelerator circuit), the method comprising:
in a forward-propagation phase of an on-device machine learning process, using storage elements in an inference memory to perform forward-propagation in one or more layers in a set of layers of a neural network, wherein the forward-propagation phase performs inferencing based on a set of input data (Goulding, ¶¶ [0012]-[0017], [0019]-[0024], [0026], and [0030]-[0031] with FIG. 1: training a DNN using large data sets and models leads to better accuracy and inference, but using large data sets also increases computation time; reduce DNN latency, make field training practical, and/or in-flight adaptation possible; the co-occurrence of the forward calculation and the backward propagation can help increase the use of limited FPGA memory resources, reduce power consumption, and reduce time to make a prediction or train a DNN; the PEs 165 may perform a forward propagation of an input in the DNN 195. Each of the PEs 165 in a first or input layer 180 may receive an input, such as from one or more external sources, and multiply the input by a weight factor; the weight factor may be different for each of the inputs; the weight factors for one or more of the PEs 165 may be stored in the DRAM 120 and may be retrieved by a PE 165 and/or stored in the SRAM 170 for retrieval by the PE 165; the PEs 165 within the input layer 180 may then work (in parallel) to perform a computation on the weighted inputs and output a result; each of the PEs 165 in the second and subsequent hidden layers 185 and the final or classifier layer 190 of the DNN 195 may receive an input from one or more PEs 165 of a prior hidden layer 185 or the input layer 180, and multiply the input by a weight factor 175; the weight factor 175 may be different for each of the inputs, and may be stored in the DRAM 120 and retrieved by the PE 165 and/or stored in the SRAM 170 for retrieval by the PE 165; the PEs 165 may then perform a computation on the weighted inputs and output a result; the PEs 165 of the second layer 185 may wait for all the PEs 165 of the input layer 180 to complete their computations prior to beginning computation; likewise, each of the subsequent layers 185 may wait for all the PEs 165 of its immediately prior layer 185 to complete prior to beginning computation; the PEs 165 of each layer of the DNN 195 may operate in parallel; computational results of the PEs 165 can be provided to the DRAM 120 to be stored; the SRAM 170 may include weight data for the PEs 165 of each layer 180, 185, 190 of the DNN 195 as well as input and output registers to permit the PEs 165 to step through each of the layers of the DNN 195 from the input layer 180 to the classifier layer 190 while the PEs 165 read input data from the SRAM 170 and write output data to the SRAM 170 corresponding to each layer of the DNN 195; the DNN 195 may be used to determine a prediction; e.g., based on at least a portion of an input image represented by the input data 140, a topology of the DNN 195, and weights determined according to training the DNN 195 to recognize a vehicle of a certain type, the CPU 115 may analyze the output data 145 to determine whether the input image represented by the input data 140 includes an image of a vehicle of the certain type; ¶¶ [0035]-[0037] with FIGS. 2-3: provide input 202 to the input layer 180 which performs operations 220, 230, and/or 240 on the input; retrieve, from the DRAM 120, weights 210A to be used by the input layer 180 in performing a calculation; the operation 220 can include multiplying the input (a) by a retrieved corresponding weight; the operation 230 can include applying a transfer function to result of the multiplication from operation 220; at operation 240, results from multiple PEs 165 of the input layer 180 can be pooled, such as by combining the results, mathematically and/or into a vector or matrix, of the operation 230; results of the operation 240 can be stored in the SRAM 170, which can be retrieved from the SRAM 170 and provided to the hidden layer 185 as input; the layers 185 and 190 perform operations similar to the input layer 180, with the layers 185 likely including different weights, different inputs, different transfer functions, and/or different pooling; ¶¶ [0048]-[0049] and [0051] with 810-820 in FIG. 8: retrieving for the forward propagation and from a remote memory, respective weight values associated with the last layer, at operation 810; storing the retrieved weight values in a local memory of the processing circuitry, at operation 820; multiplying a plurality of computational unit input values by corresponding weight values to produce a plurality of weighted input values, the plurality of computational unit input values received from a lower-numbered layer of the plurality of neural network layers when the respective weighted computational unit is not in the first layer or received from the input/output ports when the respective weighted computational unit is in the first layer; performing a computational function on the plurality of weighted input values to produce a plurality of computational function results; transmitting the plurality of computational function results to a higher-numbered layer of the plurality of neural network layers when the respective weighted computational unit is not in the last layer or to the input/output ports when the respective weighted computational unit is in the last layer; transmitting the plurality of computational function results to a higher-numbered layer for the respective weighted computational unit can include writing the plurality of results to a local memory; receiving the plurality of computational unit input values from a lower-numbered layer of the plurality of neural network layers when the respective weighted computational unit is not in the first layer is by reading the computational unit input values from the local memory);
in a back-propagation phase of the on-device machine learning process, using a back-propagation circuit coupled to the inference memory (Goulding, ¶¶ [0072]-[0074] and Claims 3-5: circuitry to backpropagate the plurality of error values to at least one neuron of the lower-numbered layer when the respective weighted computational unit is not in the first layer or to the input/output ports when the respective weighted computational unit is in the first layer; the circuitry to transmit the plurality of backpropagating results to a corresponding weighted computational unit of the lower numbered layer when the respective weighted computational unit is not in the first layer includes circuitry to write the plurality of backpropagating results to the local memory, and the circuitry to receive the plurality of error values from the higher-numbered layer when the respective weighted computational unit is not in the last layer includes circuitry to read the plurality of error values from the local memory; ¶ [0058]: a hardware-implemented module may comprise dedicated circuitry or logic that is permanently configured to perform certain operations) to share at least some of the storage elements to process a subset of layers in the set of layers in the neural network to at least partially train the neural network (Goulding, ¶¶ [0017]-[0018]: the DNN error can be backward propagated (i.e., run backwards), such as to correct weights of PEs, reusing the same hardware-limited accelerator circuit in FPGA and at the same time the parameters (used in the forward calculation) are available in SRAM (i.e., using at least some of the storage elements from forward propagation); the neural network is not run backward serially, rather the network waits until weights are in active memory for use in a forward propagation and backpropagates error values; ¶¶ [0025] and [0032] with FIG. 1: after an input has been through a forward propagation through the DNN 195, a backward propagation of error may be performed through the DNN 195; in a backward propagation modified weights associated with each of the PEs 165 are stored in the DRAM 120 or SRAM 170 for future reference or use; backward propagation of errors is interleaved with the forward propagation, such that they co-occur; when interleaved backward propagation is performed, it may begin with the classifier layer 190; calculate an error present in the classifier layer 190 and then initiate forward propagation beginning at the input layer 180; the forward propagation may halt at the layer 185 immediately prior to the classifier layer 190 (e.g., classifier layer- 1); the output buffer corresponding to the layer 185 immediately prior to the classifier layer 190 (e.g., classifier layer-1) may include data for calculating an adjustment to the weights of the classifier layer 190; after the weights of the classifier layer 190 have been adjusted, the weights may be transferred to the DRAM 120 and the processor may initiate forward propagation from the input layer 180 to layer immediately prior to the last layer at which the prior forward propagation halted (e.g., two layers 185 prior to the classifier layer 190, or classifier layer-2); the error for the layer immediately prior to the classifier layer 190 (e.g., classifier layer-1) may be calculated using the output data of the layer 185 that is two layers 185 prior to the classifier layer 190 ( e.g., classifier layer-2); the weights of the layer immediately prior to the classifier layer 190 ( e.g., classifier layer- 1) may be adjusted and transferred to the DRAM 120; this process may be performed a total of N-1 times, where N is the total number of layers from the input layer 180 to the classifier layer 190, until the classifier layer-n = the input layer 180, where n ranges from 1 to N-1; the weights of the input layer 180 may not be adjusted, and backward propagation may be considered complete at that point; i.e., only a subset of layers in DNN is trained/retrained in backward propagation to adjust weights for each forward propagation starting from the last layer (e.g., classify layer 190); ¶¶ [0038]-[0047] with FIGS. 4-7: for backward propagation, the processor 260 can determine a difference between the output 204 and a desired output 402, at operation 404A; the result, error "e" 408, can be provided to the classification layer 190 which can use the error as an input and propagate the error to the weight w3j or other input; the layer 190 can perform an operation 410A using the error, which includes an operation (e.g., a multiplication) that is a function of the output, error, and weight; the result of the operation 410A can be stored in the SRAM 170 and provided to the processor 260 which can determine an amount to adjust the weight w3j; the amount can be determined by performing an operation 406A on the result of the operation 410A, which can include multiplying by a constant, a function that depends on a change in the error and/or a change in the weight, or a combination thereof; the weight 210C can be adjusted by an amount equal to the result of the operation 406A, at operation 404B; a new weight 210C1 can be stored in place of the weight 210C, in the DRAM 120; the error can be further propagated to the layer 185 (for performing operation 410B and operation 406B to adjust the weight 210B, w2j, to a new weight 210B1 at operation 404C based on the result of the operation 406B) and then to the layer 180 (for performing operation 410C and operation 406C to adjust the weight 210A, w1j, to a new weight 210A1 at operation 404D based on the result of the operation 406C); performing operation 410A at a time that the weight 210C is locally available to the layer 190 (a time at which the weight w3j is in the SRAM 170), operation 410B at a time that the weight 210B is locally available to the layer 185 (a time at which the weight w2j is in the SRAM 170), and operation 410A at a time the weight 210A is locally available to the layer 180 (a time at which the weight w1j is in the SRAM 170); after a first few forward propagations and interleaved backward propagations, the weights 210A, 210B, and 210C for each layer can be updated with every backward propagation; in FIG. 7, the weights 210A-210C can be updated every backward propagation after the third input; for DNNs with more/fewer layers or different configurations, this number can be different; i.e., each SRAM 170 in FIG. 7 is shared by forward propagation (for inferencing or making prediction) and backward propagation (for training or distributing errors); ¶¶ [0048]-[0055] with 830-840 in FIG. 8: while a weight of the respective weight values associated with the last layer is still in the local memory from the forward propagation, backward propagating an output error value from an output of the last layer to an input of the last layer, at operation 830; and providing a result of the backward propagating to the input/output ports, at operation 840; receiving a plurality of error values from the higher-numbered layer when the respective weighted computational unit is not in the last layer or from the input/output ports when the respective weighted computational unit is in the last layer, each of the plurality of error values corresponding to a different weight value of the respective weighted computational unit; backpropagating the plurality of error values to the lower-numbered layer when the respective weighted computational unit is not in the first layer or to the input/output ports when the respective weighted computational unit is in the first layer; multiplying the plurality of error values by the corresponding weight values of the respective weighted computational unit to produce a plurality of backpropagating results; transmitting the plurality of backpropagating results to corresponding ones of the plurality of weighted computational units of the lower-numbered layer when the respective weighted computational unit is not in the first layer or to the input/output ports when the respective weighted computational unit is in the first layer; transmitting the plurality of backpropagating results to corresponding ones of the plurality of weighted computational units of the lower-numbered layer when the respective weighted computational unit is not in the first layer is by writing the plurality of backpropagating results to the local memory; receiving the plurality of error values from the higher-numbered layer when the respective weighted computational unit is not in the last layer is by reading the plurality of error values from the local memory; subtracting a corresponding predefined desired result from each of the plurality of computational function results transmitted to the input/output ports by the plurality of weighted computational units in the last layer to determine the plurality of error values for the plurality of weighted computational units in the last layer; transmitting the plurality of error values to the processing circuitry for the plurality of weighted computational units in the last layer; for each of the plurality of weighted computational units, multiplying the plurality of backpropagating results transmitted by the respective weighted computational unit by a multiplication factor to determine a plurality of multiplying results, and subtracting the plurality of multiplying results from the corresponding weights for the respective weighted computational unit to determine updated weights for the respective weighted computational unit; computing updated weight values for each of the plurality of weighted computational units according to the backpropagated output error values; transmitting the updated weight values to the plurality of weighted computational units, wherein the backpropagation of the output error values from the last layer to the first layer is performed when weight values for the respective weighted computational units are in active memory of the respective weighted computational units, the weight values used to perform computations by the weighted computational units during both the forward propagation and the backward propagation), wherein the back-propagation phase performs back-propagation that comprise storing intermediate parameters corresponding to a prediction error of the inferencing based on the set of input data; (Goulding, ¶¶ [0014]-[0015]: training a DNN using large data sets and models leads to better accuracy and inference, but using large data sets also increases computation time; reduce DNN latency, make field training practical, and/or in-flight adaptation possible; the co-occurrence of the forward calculation and the backward propagation can help increase the use of limited FPGA memory resources, reduce power consumption, and reduce time to make a prediction or train a DNN; ¶ [0017]: intermediate results held in SRAM; the DNN error can be backward propagated (i.e., run backwards), such as to correct weights of PEs, reusing the same hardware-limited accelerator circuit in FPGA and at the same time the parameters (used in the forward calculation) are available in SRAM; ¶¶ [0025]-[0026]: after an input has been through a forward propagation through the DNN 195, a backward propagation of error may be performed through the DNN 195; in a backward propagation modified weights associated with each of the PEs 165 are stored in the DRAM 120 or SRAM 170 for future reference or use; backward propagation of errors is interleaved with the forward propagation, such that they co-occur; the DNN 195 may be used to determine a prediction); and
iterating between the forward-propagation phase and the back-propagation phase until a stop condition is reached (Goulding, ¶ [0032]: this process may be performed a total of N-1 times, where N is the total number of layers from the input layer 180 to the classifier layer 190, until the classifier layer-n = the input layer 180, where n ranges from 1 to N-1; the weights of the input layer 180 may not be adjusted, and backward propagation may be considered complete at that point).
Goulding further discloses a storage-efficient system (Goulding, 195 in FIG. 1; 200 in FIGS. 2-3; 400 in FIG. 4; and 500 in FIG. 5 with ¶¶ [0002], [0019], [0034], [0037]-[0038], [0041], [0043]-[0044] : a deep neural network (DNN) system) for on-device machine learning (Goulding, ¶¶ [0012]-[0017] and [0019]-[0020] with FIG. 1: an artificial neural network (ANN) include a network of processing elements (PEs) with weight values associated with the interconnections between the PEs stored in a local or remote memory; the PE's computations can be performed and the results stored locally; implement ANNs on a larger scale than achievable via a purely local network of PEs and local memory; efficiently using SRAM and reducing use of DRAM; implement the DNN using a local and/or external memory and compute resources; the DNN 195 may include a plurality of PEs 165 and local static random access memory (SRAM) 170; the co-occurrence of the forward calculation and the backward propagation can help increase the use of limited FPGA memory resources, reduce power consumption, and reduce time to make a prediction or train a DNN; Interleaved backpropagation is a DNN (on-chip) machine learning method that can help reduce energy use in a system on chip (SoC) processor; only a portion of the DNN is implemented at any one time in an FPGA accelerator circuit)), the system comprising: an inference memory, the inference memory including storage elements (Goulding, 170 in FIGS. 1-7 with ¶¶ [0013]-[0017], [0020], [0024], and [0034]: local static random access memory (SRAM) or on-chip SRAM 170); a processor, the processor comprising circuitry configured as a back-propagation circuit, and wherein at least a portion of the storage elements are coupled to the back-propagation circuit (Goulding, ABSTRACT and ¶¶ [0070]-[0074]: each of the plurality of neural network layers including a plurality of weighted computational units having circuitry to interleave forward propagation of computational unit input values from the first layer to the last layer and backward propagation of output error values from the last layer to the first layer; circuitry to backpropagate the plurality of error values to at least one neuron of the lower-numbered layer when the respective weighted computational unit is not in the first layer or to the input/output ports when the respective weighted computational unit is in the first layer; the circuitry to backpropagate the plurality of error values includes circuitry to multiply the plurality of error values by the corresponding weight values of the respective weighted computational unit to produce a plurality of backpropagating results, and circuitry to transmit the plurality of backpropagating results to a corresponding weighted computational unit of the lower-numbered layer when the respective weighted computational unit is not in the first layer or to the input/output ports when the respective weighted computational unit is in the first layer; the circuitry to transmit the plurality of backpropagating results to a corresponding weighted computational unit of the lower numbered layer when the respective weighted computational unit is not in the first layer includes circuitry to write the plurality of backpropagating results to the local memory, and the circuitry to receive the plurality of error values from the higher-numbered layer when the respective weighted computational unit is not in the last layer includes circuitry to read the plurality of error values from the local memory; ¶ [0017]: the DNN error can be backward propagated (i.e., run backwards), such as to correct weights of PEs, reusing the same hardware-limited accelerator circuit in FPGA and at the same time the parameters (used in the forward calculation) are available in SRAM; ¶¶ [0019]-[0020]: hardware processors, such as central processing units (CPUs), graphics processing units (GPUs), microcontrollers, field programmable gate arrays (FPGAs), or the like; ¶¶ [0034]-[0036] with 250/260 in FIGS. 2-7: processors 250 and 260; the processor 250 manages an FPGA accelerator circuit (via configuring memory-mapped register settings, direct memory access (DMA), linked-lists, etc.) to implement the neural network layers 180, 185, and 190; the processor 260 can manage output, error calculation, or the like; ¶¶ [0048] and [0052] with FIG. 8: storing the retrieved weight values in a local memory of the processing circuitry, at operation 820; while a weight of the respective weight values associated with the last layer is still in the local memory from the forward propagation, backward propagating an output error value from an output of the last layer to an input of the last layer, at operation 830; ¶ [0058]: hardware-implemented module may comprise dedicated circuitry or logic that is permanently configured (e.g., as a special-purpose processor, such as an FPGA or an ASIC) to perform certain operations); and a non-transitory computer-readable medium (Goulding, ¶¶ [0057] and [0065]: non-transitory machine readable medium; 120 and 170 in FIGS. 1-7 with ¶¶ [0020], [0027], and [0034]: local static random access memory (SRAM) 170, dynamic random access memory (DRAM) 120, flash memory, nonvolatile random access memory (NVRAM), nonvolatile memory (NVM), etc.) comprising instructions (Goulding, ¶ [0066]: computer program) that, when executed by the processor, cause operations described above to be performed (Goulding, ¶ [0057]: one or more computer systems (e.g., a standalone, client or server computer system) or one or more processors may be configured by software ( e.g., an application or application portion) as a hardware-implemented module that operates to perform certain operations; ¶ [0067]: operations may be performed by one or more programmable processors executing a computer program to perform functions by operating on input data and generating output).
Goulding fails to explicitly disclose to repurpose at least some of the storage elements to process a subset of layers in the set of layers in the neural network to at least partially train the neural network.
Shirahata teaches a system and a method relating to training deep neural networks (Shirahata, ABSTRACT), wherein repurpose at least some of the storage elements to process a subset of layers in the set of layers in the neural network to at least partially train the neural network (Shirahata, ABSTRACT in Page 1: training deep neural networks requires a large amount of memory, making very deep neural networks difficult to fit on accelerator memories; present a method to reduce the amount of memory for training a deep neural network; the method enables to suppress memory increase during the backward pass, by reusing the memory regions allocated for the forward pass; enable training speedup by increasing the mini batch size up to double; Section I in Page 1: in the case of training, DNNs require over twice as many memory as DNNs for inference, because the training processing is composed of not only forward propagation (i.e. the inference phase) but also backwards propagation of errors and parameter updates; in order to process larger DNNs on accelerators, propose a memory reduction method for deep neural network training; reduce the amount of memory by reusing the memory region of the forward propagation (i.e., the inference phase) for the backward propagation (i.e., reuse the inference memory for training – reuse the memory for different purposes: inferencing and training); introduce the computation order decision algorithm for reducing memory usage considering mini-batch sizes; in contrast to the existing lossy compression-based techniques, this technique does not affect training accuracy (i.e. lossless compression); Section 2 with FIG. 1 in Pages 1-2: call input and output data of each layer as neuron data and synapses as parameters; the parameters are initialized inside the accelerator before the computation starts; then, a set of input data is loaded from a disk to the accelerator, and the training computation for the set of input data is executed; when the training is finished, the output parameters are stored to the disk; Section 2.1-2.3 with FIGS. 2-3 in Pages 2-3: training feed-forward networks consists of three phases; forward pass, backward pass, and parameter updates; the forward pass classifies a set of images into the given number of categories (i.e. executes the inference phase); the backward pass computes gradients of the neuron data and the parameters for each layer based on the errors obtained in the forward pass, and propagates the gradients to the earlier layer; the parameter update phase updates the parameters by subtracting the gradients obtained in the backward pass multiplied by a learning rate; these three phases are iterated until a convergent criteria is met; the backward pass computes the backward propagation for the corresponding forward pass; the backpropagation method is an iterative optimization method which gradually minimizes errors by propagating the error gradients of neuron data to earlier layers; the backward pass requires about twice as much memory as the forward pass, since the backward pass uses not only gradients of neurons and parameters for each layer (except the first layer) but also neuron data and parameters; parameters are updated using the gradients of parameters for all the layers; Figure 3 describes the memory usage and the workflow of forward pass and backward pass of the example CNN described in Figure 2; Section 3, 3.1, and 3.2 with FIG. 4 in Page 4: propose a method to reduce memory size on DNN training; the main idea is to reuse the memory regions used at the forward pass (i.e., the inference phase) for the backward pass (i.e., for training phase), by overwriting the memory regions used for neuron data and parameters when running the backward pass; allocate as small memory as possible considering reusable memory regions; before beginning training, calculate backward computation order, given neuron data size and parameter size for each layer, and mini batch size; compute the forward pass; compute the backward pass according to the backward computation order decided in the step 1, reusing the same region used at the forward pass when possible; compute the parameter updates according to the parameter update algorithm; continue the steps 2 to 4 until the convergence criteria is met; Figure 4 describes an example of overview of the memory usage and workflow using proposed method, wherein the dotted memory regions can be reduced since these regions are never used during the whole training; proposed parameter update algorithm updates some parameters during the backward pass for each layer, while the standard algorithm updates after the backward pass of all the layers finished; i.e., starting from the last layer, parameters are updated during the backward pass for each layer before the backward pass of all the layers finished; for each layer, overwrite ΔWt to the memory region used for Wt for computing Wt+1 by eqn. (6) during the backward pass of the layer, when compute parameter gradients after the neuron gradients at the layer; by applying proposed parameter update algorithm, do not need to allocate memory regions for the gradients of the parameters (i.e. ΔWt) for the layers where the gradients of parameters are overwritten).
Goulding and Shirahata are analogous art because they are from the same field of endeavor, a system and a method relating to training deep neural networks. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to apply the teaching of Shirahata to Goulding. Motivation for doing so would reduce the amount of memory for training a deep neural network without affecting training accuracy and enable training speedup (Shirahata, ABSTRACT in Page 1; Section 1 in Page 1; Section 2.2 in Page 3; and Section 5 in Page 5).
Claim 2
Goulding in view of Shirahata discloses all the elements as stated in Claim 1 and further discloses wherein the subset of layers comprises N final layers in the set of layers (Goulding, ¶ [0032] with FIG. 1: when interleaved backward propagation is performed, it may begin with the classifier layer 190; calculate an error present in the classifier layer 190 and then initiate forward propagation beginning at the input layer 180; the forward propagation may halt at the layer 185 immediately prior to the classifier layer 190 (e.g., classifier layer- 1); the output buffer corresponding to the layer 185 immediately prior to the classifier layer 190 (e.g., classifier layer-1) may include data for calculating an adjustment to the weights of the classifier layer 190; after the weights of the classifier layer 190 have been adjusted, the weights may be transferred to the DRAM 120 and the processor may initiate forward propagation from the input layer 180 to layer immediately prior to the last layer at which the prior forward propagation halted (e.g., two layers 185 prior to the classifier layer 190, or classifier layer-2); the error for the layer immediately prior to the classifier layer 190 (e.g., classifier layer-1) may be calculated using the output data of the layer 185 that is two layers 185 prior to the classifier layer 190 ( e.g., classifier layer-2); the weights of the layer immediately prior to the classifier layer 190 ( e.g., classifier layer- 1) may be adjusted and transferred to the DRAM 120; this process may be performed a total of N-1 times, where N is the total number of layers from the input layer 180 to the classifier layer 190, until the classifier layer-n = the input layer 180, where n ranges from 1 to N-1; the weights of the input layer 180 may not be adjusted, and backward propagation may be considered complete at that point; ¶¶ [0044]-[0047] with FIG. 7: after a first few forward propagations and interleaved backward propagations, the weights 210A, 210B, and 210C for each layer can be updated with every backward propagation; in FIG. 7, the weights 210A-210C can be updated every backward propagation after the third input; for DNNs with more/fewer layers or different configurations, this number can be different; i.e., only a subset of layers (e.g., one layer) in DNN is trained/retrained in backward propagation to adjust weights for each forward propagation starting from the last layer (e.g., classify layer 190)) (Shirahata, Sections 3.1-3.2 with FIG. 4 in Page 4: proposed parameter update algorithm updates some parameters during the backward pass for each layer, while the standard algorithm updates after the backward pass of all the layers finished; i.e., starting from the last layer, parameters are updated during the backward pass for each layer before the backward pass of all the layers finished).
Claim 3
Goulding in view of Shirahata discloses all the elements as stated in Claim 2 and further discloses wherein at least partially training the neural network comprises replacing weight parameter data in at least some of the N final layers (Goulding, ¶ [0032] with FIG. 1: when interleaved backward propagation is performed, it may begin with the classifier layer 190; calculate an error present in the classifier layer 190 and then initiate forward propagation beginning at the input layer 180; the forward propagation may halt at the layer 185 immediately prior to the classifier layer 190 (e.g., classifier layer- 1); the output buffer corresponding to the layer 185 immediately prior to the classifier layer 190 (e.g., classifier layer-1) may include data for calculating an adjustment to the weights of the classifier layer 190; after the weights of the classifier layer 190 have been adjusted, the weights may be transferred to the DRAM 120 and the processor may initiate forward propagation from the input layer 180 to layer immediately prior to the last layer at which the prior forward propagation halted (e.g., two layers 185 prior to the classifier layer 190, or classifier layer-2); the error for the layer immediately prior to the classifier layer 190 (e.g., classifier layer-1) may be calculated using the output data of the layer 185 that is two layers 185 prior to the classifier layer 190 ( e.g., classifier layer-2); the weights of the layer immediately prior to the classifier layer 190 ( e.g., classifier layer- 1) may be adjusted and transferred to the DRAM 120; this process may be performed a total of N-1 times, where N is the total number of layers from the input layer 180 to the classifier layer 190, until the classifier layer-n = the input layer 180, where n ranges from 1 to N-1; the weights of the input layer 180 may not be adjusted, and backward propagation may be considered complete at that point; ¶¶ [0044]-[0047] with FIG. 7: after a first few forward propagations and interleaved backward propagations, the weights 210A, 210B, and 210C for each layer can be updated with every backward propagation; in FIG. 7, the weights 210A-210C can be updated every backward propagation after the third input; for DNNs with more/fewer layers or different configurations, this number can be different) (Shirahata, Sections 3.1-3.2 with FIG. 4 in Page 4: proposed parameter update algorithm updates some parameters during the backward pass for each layer, while the standard algorithm updates after the backward pass of all the layers finished; i.e., starting from the last layer, parameters are updated during the backward pass for each layer before the backward pass of all the layers finished).
Claim 4
Goulding in view of Shirahata discloses all the elements as stated in Claim 3 and further discloses using at least some of the weight parameter data to generate an inference result (Goulding, ¶ [0032] with FIG. 1: when interleaved backward propagation is performed, it may begin with the classifier layer 190; calculate an error present in the classifier layer 190 and then initiate forward propagation beginning at the input layer 180; the forward propagation may halt at the layer 185 immediately prior to the classifier layer 190 (e.g., classifier layer- 1); the output buffer corresponding to the layer 185 immediately prior to the classifier layer 190 (e.g., classifier layer-1) may include data for calculating an adjustment to the weights of the classifier layer 190; after the weights of the classifier layer 190 have been adjusted, the weights may be transferred to the DRAM 120 and the processor may initiate forward propagation from the input layer 180 to layer immediately prior to the last layer at which the prior forward propagation halted (e.g., two layers 185 prior to the classifier layer 190, or classifier layer-2); the error for the layer immediately prior to the classifier layer 190 (e.g., classifier layer-1) may be calculated using the output data of the layer 185 that is two layers 185 prior to the classifier layer 190 ( e.g., classifier layer-2); the weights of the layer immediately prior to the classifier layer 190 ( e.g., classifier layer- 1) may be adjusted and transferred to the DRAM 120; this process may be performed a total of N-1 times, where N is the total number of layers from the input layer 180 to the classifier layer 190, until the classifier layer-n = the input layer 180, where n ranges from 1 to N-1; the weights of the input layer 180 may not be adjusted, and backward propagation may be considered complete at that point; ¶¶ [0043]-[0047] with FIGS. 6-7: with an additional input 202C and propagated to the layer 190 and the error 408 further backpropagated to the layer 185; the error 408 can first be backpropagated to the layer 190; at a time in which the weight 210B, w2j, is available in the SRAM 170, the layer 185 can backpropagate the error to its input(s); the weight 210C can be updated based on the backpropagation to a new weight 210C1; additional weight retrievals are not needed for backpropagation; adjusted weights 210C1 is used in forward propagation for input 202C at classifier layer 190 to generate inference result as shown in FIG. 6; after a first few forward propagations and interleaved backward propagations, the weights 210A, 210B, and 210C for each layer can be updated with every backward propagation; in FIG. 7, the weights 210A-210C can be updated every backward propagation after the third input; for DNNs with more/fewer layers or different configurations, this number can be different).
Claim 5
Goulding in view of Shirahata discloses all the elements as stated in Claim 1 and further discloses wherein the forward-propagation is performed in response to completing at least one of the back-propagation (Goulding, ¶ [0032] with FIG. 1: interleaved backward propagation a processor may calculate an error present in the classifier layer 190 and then initiate forward propagation beginning at the input layer 180).
Claim 6
Goulding in view of Shirahata discloses all the elements as stated in Claim 1 and further discloses wherein the forward-propagation commences at an initial layer of the set of layers and terminates prior to a final convolutional layer of the set of layers (Goulding, ¶ [0032] with FIG. 1: interleaved backward propagation a processor may calculate an error present in the classifier layer 190 and then initiate forward propagation beginning at the input layer 180; the forward propagation may halt at the layer 185 immediately prior to the classifier layer 190 (e.g., classifier layer-1)).
Claim 7
Goulding in view of Shirahata discloses all the elements as stated in Claim 1 and further discloses wherein a second iteration of forward-propagation terminates earlier than a first iteration of forward-propagation by at least one layer in the one or more layers (Goulding, ¶¶ [0032] and [0043]-0047] with FIGS. 1 and 6-7: when interleaved backward propagation is performed, it may begin with the classifier layer 190; calculate an error present in the classifier layer 190 and then initiate forward propagation beginning at the input layer 180; the forward propagation may halt at the layer 185 immediately prior to the classifier layer 190 (e.g., classifier layer- 1); the output buffer corresponding to the layer 185 immediately prior to the classifier layer 190 (e.g., classifier layer-1) may include data for calculating an adjustment to the weights of the classifier layer 190; after the weights of the classifier layer 190 have been adjusted, the weights may be transferred to the DRAM 120 and the processor may initiate forward propagation from the input layer 180 to layer immediately prior to the last layer at which the prior forward propagation halted (e.g., two layers 185 prior to the classifier layer 190, or classifier layer-2); the error for the layer immediately prior to the classifier layer 190 (e.g., classifier layer-1) may be calculated using the output data of the layer 185 that is two layers 185 prior to the classifier layer 190 ( e.g., classifier layer-2); the weights of the layer immediately prior to the classifier layer 190 ( e.g., classifier layer- 1) may be adjusted and transferred to the DRAM 120; this process may be performed a total of N-1 times, where N is the total number of layers from the input layer 180 to the classifier layer 190, until the classifier layer-n = the input layer 180, where n ranges from 1 to N-1; the weights of the input layer 180 may not be adjusted, and backward propagation may be considered complete at that point).
Claims 11 and 15
Goulding in view of Shirahata discloses all the elements as stated in Claims 1 and 12 respectively and further discloses wherein the neural network receives the set of input data that comprises at least one of audio data or image sensor data (Goulding , ¶¶ [0014] and [0026]: the DNN 195 may be used to determine a prediction; e.g., based on at least a portion of an input image represented by the input data 140, a topology of the DNN 195, and weights determined according to training the DNN 195 to recognize a vehicle of a certain type; FIG. 3; ¶ [0037]: the inputs 202A-202C can include one or more images, audio recordings, sensor readings, or the like).
Claims 9 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Goulding in view of Shirahata as applied to Claims 1 and 12 respectively above, and further in view of Hasan et al. ("On-chip Training of Memristor Based Deep Neural Networks", 2017 International Joint Conference on Neural Networks (IJCNN), May 2017), hereinafter Hasan.
Claims 9 and 14
Goulding in view of Shirahata discloses all the elements as stated in Claims 1 and 12 respectively and further discloses wherein one or more of the storage elements are coupled to the back-propagation circuit (Goulding, ABSTRACT and ¶ [0070]: each of the plurality of neural network layers including a plurality of weighted computational units having circuitry to interleave forward propagation of computational unit input values from the first layer to the last layer and backward propagation of output error values from the last layer to the first layer; ¶¶ [0013], [0015]-[0017], [0019]-[0020], and [0071]-[0074] with FIG. 1; the local processing circuitry can include static random access memory (SRAM) to reduce both training and execution time of the ANN; a processor of the ANN (e.g., a central processing unit (CPU)) may be part of a low size, weight, and power (low-SWaP) system on chip (SoC) implementation incorporating the CPU, FPGA, on-chip SRAM, and external DRAM for training and execution of DNN; interleaving a forward calculation of an input with a backward propagation of errors, such as in a field programmable gate array (FPGA) (e.g., an accelerator circuit of the FPGA); interleav