DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The amendment filed on December 23rd, 2025 has been entered and Claims 1-20 is/are pending. Applicant’s amendments to the Claims overcome the 35 U.S.C 112(b) rejection(s) and U.S.C 101 rejection(s) previously set forth in the Non-Final Action mailed on September 25th, 2025.
Response to Arguments
Applicant's arguments filed December 23rd, 2025 have been fully considered but they are not persuasive.
In regards to applicant’s argument starting on page 31, applicant argues that neither Bajic-1 nor Bajic-2 teaches “generating a summary of the set of execution data of the execution of the neural network, the summary of the set of execution data comprising a reduced-dimension representation of the set of execution data”. This is not persuasive. Respectfully, the previous rejection is based on Bajic-1 in view of Bajic-2, and not Bajic-1 alone.
Applicant’s arguments with respect to Bajic-2’s “does not teach a summary of a set of execution data of a neural network”. This is not persuasive. Applicant’s SPEC [0009] The computation data for the execution of an ANN can include the input data to the ANN, the execution data (e.g., activation data that is passed from one layer of the network to the next)…[0012], The execution data is data generated during the execution of the ANN. Thus, in the previous rejection, Bajic-2, ([0033], “For example, the execution data can be a set of execution data values such as outputs of each hidden layer in an ANN.”) Under the broadest reasonable interpretation (BRI), execution data is interpreted to include outputs produced during execution of a neural network.
Applicant’s arguments with respect to claim(s) 1, 17, 19, have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-6, 9, 11, 16, 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bajic et al., (US10318317B2) (hereinafter “Bajic-1”) in view of Bajic et al., (US20180293486A1) (hereinafter “Bajic-2”).
Regarding claim 1:
Bajic-1 teaches:
A computer-implemented method, conducted during an execution of a neural network, comprising: (Col 4, lines 12-17, The execution of an ANN is an example of the class of problems described in the prior paragraph. The execution of an ANN [execution of a neural network] for purposes of generating an inference from an input generally involves a large number of component computations that do not significantly contribute to the value expressed by the generated inference.)
generating a set of execution data; (Col 4, lines 46-50, “The computation of the outputs of one layer of the ANN, or sub-sections of one layer of the ANN, can be considered a component computation to a composite computation [a set execution data] consisting of the overall execution of the ANN.”)
generating a prediction using the summary of the set of neural network data and the summary of the set of execution data; (Col 5, lines 59-62, “The partial execution will generate an estimated output [prediction] which may not be equivalent to the precise output that would have been generated by a complete execution of the composite computation.”)…(Col 6, lines 45-48, “FIG. 2 includes two data flow diagrams that illustrate the generation of an estimated output for a composite computation. In diagram 200, input tensors [X] and [Y] [neural network data] are subject to a complete execution of a computation.”)…(Col 7, lines 48-66, “Data flow diagram 210 also includes optional step 212 of generating a reference value [execution data]...Step 212 can utilize one or both of input tensors [X] and [Y]. The reference value can be utilized to more accurately determine the priority that any given component computation should be given. For example, if tensors [X] and [Y] included values with widely varying orders of magnitude, the reference value could be set equal to the largest order of magnitude occupied by the values of the input tensors, and component computation approximations that were not at or above a fraction of that order of magnitude would be suppressed. The reference value used in evaluating a component computation approximation can also be derived from other component computation approximations in the same composite computation.”)
executing a composite computation, wherein the composite computation is required for the execution of the neural network; and (Col 2, lines 21- 23, “A complete execution of the composite computation generates a precise output and requires execution of a set of component computations”)
suppressing a set of component computations of the composite computation, (Col 6, lines 33-35, “The contribution estimate can be used to suppress individual component computations while preserving the fidelity of the partial execution to the complete execution.”)
wherein the set of component computations is at least partly determined by the prediction. (Col 4, lines 53-60, “Individual sum-of-products computations, convolutions of a filter or set of filters with a set of input data, and multi-product computations such as matrix multiplications are all examples of composite computations that are amenable to optimization because they consist essentially of component computations that can alternatively be suppressed or fully executed based on an estimate [prediction] of their contribution”)
Bajic-1 does not explicitly teach:
generating a summary of a set of neural network data of the neural network;
generating a summary of a set of execution data of the execution of the neural network;
Bajic-2 teaches:
generating a summary of a set of neural network data of the neural network; ([0030], “The deriving of the simplified version of a neural network could include replacing a set of weight values, filter values, or any other value used in the computation of the neural network,”)
generating a summary of the set of execution data of the execution of the neural network ([0033], “For example, the execution data can be a set of execution data values such as the outputs of each hidden layer in an ANN (i.e., wherein under the broadest reasonable interpretation (BRI), execution data is interpreted to include outputs produced during execution of a neural network, thus a summary of execution data is generated)”
the summary of the set of execution data comprising a reduced-dimension representation of the set of execution data; ([0027], “The simplified version of the directed graph may be a down-sampled version of the directed graph or a version in which individual values in the directed graph were rounded, replaced by more basic data structures, or otherwise simplified. An example of the replacement of a value with a more basic data structure is the replacement of a high precision floating point data structure with a low precision fixed point data structure. The simplified version of the directed graph could also exhibit more dramatic differences as compared to the original directed graph. For example, the simplified version could have vertices and edges associated with tensors of lower rank or dimensionality [comprising a reduced-dimension representation] (i.e., wherein under the broadest reasonable interpretation (BRI) reduced-dimension representation is interpreted to be lower rank or dimensionality, (Applicants’ SPEC [0016], “transform to the data to reduce the dimensionality of the data structure. For example, the transform could produce a lower rank approximation”) than those corresponding with the respective vertices and edges of the original directed graph”)
Bajic-2 and Bajic-1 are both related to the same field of endeavor (i.e., neural network computation efficiency). In view of the teachings of Bajic-2 it would have been obvious for a person of ordinary skill in the art to apply the teachings of Bajic-2 to Bajic-1 before the effective filing date of the claimed invention in order to improve the efficiency of a neural network by reducing computation (Bajic-2, [0003], “A traditional ANN takes in an input vector, conducts calculations using the input vector and a set of weight vectors, and produces an output vector. Each weight vector in the set of weight vectors is often referred to as a layer of the network, and the output of each layer serves as the input to the next layer. In a traditional network, the layers are fully connected, which requires every element of the input vector to be involved in a calculation with every element of the weight vector. Therefore, the number of calculations involved increases with a power law relationship to the size of each layer. Furthermore, this aspect of machine intelligence algorithms make them difficult to parallelize because the calculations for each layer depend on the output of the prior layer.”)
Regarding claim 2:
Bajic-1 as modified by Bajic-2 teaches the method of claim 1.
Bajic-1 further teaches:
wherein: execution of the neural network requires execution of a layer of the neural network; the composite computation is required for the execution of the layer of the neural network; (Col 1, lines 44-54, “A traditional ANN takes in an input vector, conducts calculations using the input vector and a set of weight vectors, and produces an output vector. Each weight vector in the set of weight vectors is often referred to as a layer of the network, and the output of each layer serves as the input to the next layer. In a traditional network, the layers are fully connected, which requires every element of the input vector to be involved in a calculation with every element of the weight vector. Therefore, the number of calculations involved increases with a power law relationship to the size of each layer.”)
the composite computation is a matrix multiply operation; (Col 4, lines 54-56, “convolutions of a filter or set of filters with a set of input data, and multi-product computations such as matrix multiplications are all examples of composite computations”)
the set of component computations are multiplication operations; (Col 7, lines 38-47, “For example, if the component computation was a 16-bit multiplication of two numbers, step 211 could involve an 8-bit multiplication of those numbers. In other approaches, step 211 will involve evaluating the operands to the component computation to determine if any of the operands are highly influential to the outcome of the operation. For example, step 211 could determine that one of the operands to a multiplication operation was a zero and the multiplication operation could be replaced by the delivery of a zero as the output of the multiplication.”)
the set of neural network data is a set of weights of the layer of the neural network; and the set of execution data is a set of output data from a prior layer of the neural network. (Col 1, lines 44-54, “A traditional ANN takes in an input vector, conducts calculations using the input vector and a set of weight vectors [neural network data], and produces an output vector [execution data]. Each weight vector in the set of weight vectors is often referred to as a layer of the network [set of weights], and the output of each layer serves as the input to the next layer. In a traditional network, the layers are fully connected, which requires every element of the input vector to be involved in a calculation with every element of the weight vector [execution data is a set of output data from a prior layer of the neural network].”)
The motivation for claim 2 is the same as the motivation of claim 1.
Regarding claim 3:
Bajic-1 as modified by Bajic-2 teaches the method of claim 1.
Bajic-1 further teaches:
generating, after generating the additional summaries, additional predictions using the additional summaries; (Col 16, lines 15-17, “The reference value could be continuously updated as additional approximations or partial execution outputs were generated for a given composite computation.”)
executing additional composite computations; and (Col 1, lines 44-54, “ANN takes in an input vector, conducts calculations using the input vector and a set of weight vectors, and produces an output vector. Each weight vector in the set of weight vectors is often referred to as a layer of the network, and the output of each layer serves as the input to the next layer. In a traditional network, the layers are fully connected, which requires every element of the input vector to be involved in a calculation with every element of the weight vector. Therefore, the number of calculations involved increases with a power law relationship to the size of each layer.”)
suppressing additional sets of component computations of the additional composite computations, (Col 2, line 65, “The method also includes alternatively suppressing and executing the component computations in the set of component computations, while executing the partial execution, based on the at least one reference value and the set of component computation approximations.”)
wherein the additional sets of component computations are at least partly determined by the additional predictions. (Col 16, lines 15-17, “The reference value could be continuously updated as additional approximations or partial execution outputs were generated for a given composite computation.”)
Barjic-1 does not explicitly teach:
iteratively: generating, after executing the composite computation, at least one of: (i) additional summaries of additional sets of neural network data of the neural network; and (ii) additional summaries of additional sets of execution data of the neural network;
Barjic-2 further teaches:
iteratively: generating, after executing the composite computation, at least one of: (i) additional summaries of additional sets of neural network data of the neural network; and (ii) additional summaries of additional sets of execution data of the neural network; ([0043], “The simplified version of the directed graph can also be recalculated if the directed graph is modified such as by training or some other update. In the specific example of an ANN, the simplified version of the directed graph can be automatically updated after each training session or after the activations have changed by a given delta.”)
The motivation for claim 3 is the same as the motivation of claim 1.
Regarding claim 4:
Bajic-1 as modified by Bajic-2 teaches the method of claim 1.
Bajic-1 does not explicitly teach:
wherein: the summary of the set of neural network data is generated using a first sampling operation; and the summary of the set of execution data is generated using a second sampling operation.
Bajic-2 further teaches:
wherein: the summary of the set of neural network data is generated using a first sampling operation; and ([0028], “The down-sampling could also be conducted in any dimension by any rate. The down-sampling can use basic averaging, a sync filter approach, or polynomial interpolation. In the specific approach of an ANN, the deriving of the simplified version of a neural network could include down-sampling a set of weight values, filter values, or any other value used in the computation of the neural network by a sampling factor using the above referenced approaches.”)
the summary of the set of execution data is generated using a second sampling operation. ([0024], “For example, the execution data can be used to modify the directed graph before the input tensor is applied by changing the values associated with the vertices or edges of the graph. In the example of a machine intelligence system, such an approach could involve rounding or down-sampling the values associated with the weights or filters of the system prior to the application of an input to the system.”)
The motivation for claim 4 is the same as the motivation of claim 1.
Regarding claim 5:
Bajic-1 as modified by Bajic-2 teaches the method of claim 4.
Bajic-1 does not explicitly teach:
wherein: the first sampling operation and the second sampling operation are average pooling operations.
Bajic-2 further teaches:
wherein: the first sampling operation and the second sampling operation are average pooling operations. ([0028], “The down-sampling can use basic averaging, a sync filter approach, or polynomial interpolation. In the specific approach of an ANN, the deriving of the simplified version of a neural network could include down-sampling a set of weight values, filter values, or any other value used in the computation of the neural network by a sampling factor using the above referenced approaches.”)
The motivation for claim 5 is the same as the motivation of claim 1.
Regarding claim 6:
Bajic-1 as modified by Bajic-2 teaches the method of claim 1.
Bajic-1 further teaches:
wherein: the prediction is generated using a predictor computation with the summary of the set of neural network data and the summary of the set of execution data as operands; (Col 8, lines 9-18, “For example, the step could determine if any of the operands to a multiplication [execution data] were zero, or determine that one of the approximations was far lower [predictor computation] than the average approximation generated for a set of component computations [neural network data]. The purpose of the comparison is to filter component computations based on their level of importance to the overall composite computation. As such, the comparison can output [prediction] a binary value which sorts the components into computations that should be fully executed and ones that should be suppressed”)
and the composite computation and the predictor computation are a same type of computation. (Col 7, lines 63-66, “The reference value used in evaluating a component computation approximation can also be derived from other component computation approximations in the same composite computation.” i.e., wherein the computation is derived from the same composite computation.)
The motivation for claim 6 is the same as the motivation of claim 1.
Regarding claim 9:
Bajic-1 as modified by Bajic-2 teaches the method of claim 6.
Bajic-1 further teaches:
evaluating the prediction to determine a set of salient outputs of the composite computation; and (Col 5, lines 14-25, “These approaches are particularly beneficial in that a single quanta of processing resources is consumed to both provide a value for determining if a computation should be pruned, and to provide a rough estimate of what the computation would have produced if it had been executed with high fidelity which can be used as a replacement for the suppressed computation. As illustrated by FIG. 1, pruning out composite computations is difficult from an ex ante perspective because it is not immediately clear which component computations are low value and which are important.”)
determining a set of salient neural network data from the set of neural network data using the set of salient outputs; and (Col 6, lines 29-39, “In some approaches, determining which computations can be suppressed also utilizes a reference value [salient neural network data]. The reference value can be evaluated along with the approximations to determine a contribution estimate for the component computation. The contribution estimate [salient outputs] can be used to suppress individual component computations while preserving the fidelity of the partial execution to the complete execution. For example, if the approximation is less than the reference value, such that the impact of the computation is likely to be nominal, the associated component computation will be suppressed and not executed.” (i.e., wherein the contribution estimate is used to determine important data))
wherein the suppressing suppresses component computations in the composite computation that do not use the set of salient neural network data. (Col 6, lines 29-39, “In some approaches, determining which computations can be suppressed also utilizes a reference value [salient neural network data]. The reference value can be evaluated along with the approximations to determine a contribution estimate for the component computation”...(Col 8, lines 40-56, “Suppressing computations can be conducted in various ways. For example, suppressing the computation may involve striking the computation [component computation] entirely from the composite computation [composite computation], providing a value from memory in lieu of conducting the computation, conducting a lower resolution version of the computation, or otherwise replacing the component computation with a simplified operation. (i.e., wherein the reference value is not used))
The motivation for claim 9 is the same as the motivation of claim 1.
Regarding claim 11:
Bajic-1 as modified by Bajic-2 teaches the method of claim 6.
Bajic-1 further teaches:
evaluating the prediction to determine a set of salient outputs of the composite computation; and (Col 6, lines 30-35, “The reference value can be evaluated along with the approximations [prediction] to determine a contribution estimate [salient output] for the component computation. The contribution estimate can be used to suppress individual component computations while preserving the fidelity of the partial execution to the complete execution.”)
Bajic-1 does not explicitly teach:
determining a set of salient execution data from the set of execution data using the set of salient outputs; and
wherein the suppressing suppresses component computations in the composite computation that do not use the set of salient execution data.
Barjic-2 further teaches:
determining a set of salient execution data from the set of execution data using the set of salient outputs; and ([0036], “The execution data can also be used to generate specific instructions for a later execution of the directed graph. For example, in the same way that the execution data can be used to generate a tag to indicate that a specific edge of the directed graph is of “low” priority [salient outputs], the execution data can also be used to generate an instruction to reduce the fidelity of the calculations associated with that edge of the directed graph [salient execution data]” (i.e., wherein the salient outputs are used to determine salient execution data))
wherein the suppressing suppresses component computations in the composite computation that do not use the set of salient execution data. ([0036], “The execution data can also be used to generate specific instructions for a later execution of the directed graph. For example, in the same way that the execution data can be used to generate a tag to indicate that a specific edge of the directed graph is of “low” priority, the execution data can also be used to generate an instruction to reduce the fidelity of the calculations associated with that edge of the directed graph [salient execution data], or to suppress the calculations associated with that edge of the directed graph”…([0041], “As another example, the computation could be suppressed altogether without even the need for providing a masked value.” (i.e., wherein the suppression does not use salient execution data))
The motivation for claim 11 is the same as the motivation of claim 1.
Regarding claim 16:
Bajic-1 as modified by Bajic-2 teaches the method of claim 1.
Bajic-1 further teaches:
wherein: the execution of the neural network is conducted after training the neural network. ([Col 4, lines 12-30, “The execution of an ANN is an example of the class of problems described in the prior paragraph. The execution of an ANN for purposes of generating an inference from an input generally involves a large number of component computations that do not significantly contribute to the value expressed by the generated inference. This is because ANNs are generally over-parameterized for any given inference. They are trained to work with many different potential inputs, but only process one input at a time. For example, an ANN may be able to recognize multiple subjects in an input image, but only a small portion of the ANN may respond in a meaningful way to any one subject. In the example of FIG. 1, a single ANN 100 is provided with two different inputs. Input 101 is an input tensor that encodes an image of a cat. After a full execution, ANN 100 produces an inference 110 from input 101 in the form of the text string “CAT.” Input 102 is an input tensor that encodes an image of a dog. After a full execution, ANN 100 produces an inference 120 from input 102 in the form of the text string “DOG”.”)
The motivation for claim 16 is the same as the motivation of claim 1.
Regarding claim 19:
Bajic-1 further teaches:
generating a summary of a set of execution data of the execution of the directed graph; (Col 16, lines 11-14, “In such an approach, component product computations with corresponding component approximations less than a fraction of that reference value would be suppressed during a partial execution. The reference value could be continuously updated as additional approximations or partial execution outputs were generated for a given composite computation.”)
executing a composite computation, wherein the composite computation is required for the execution of the directed graph; and (Col 2, lines 21- 23, “A complete execution of the composite computation generates a precise output and requires execution of a set of component computations”)
selectively executing a set of component computations of the composite computation, wherein the set of component computations is selected based on the prediction. (Col 18, lines 34-48 , “The two comparisons, the one using the reference value and the other using the component computation, can be conducted in series or in parallel. If the comparisons are conducted in series, the second comparison (in which approximations [prediction] are compared on a relative basis) can itself be suppressed if there are no conflicts between subsets of computations. FIG. 10 is a block diagram 1000 of a hardware implementation for a suppression scheme in which the suppressed component computations are suppressed while providing a substitute value in place of the output of the suppressed computation…the substituted values are the component computation approximations that were used to select which component computations should be executed and which should be suppressed [selectively executing a set of component computations of the composite computation]”)
Bajic-1 does not explicitly teach:
A computer-implemented method, conducted during an execution of a directed graph, comprising:
generating a set of execution data;
generating a summary of a set of directed graph data of the directed graph;
generating a summary of the set of execution data of the execution of the neural network
the summary of the set of execution data comprising a reduced-dimension representation of the set of execution data;
generating a prediction using the summary of the set of directed graph data and the summary of the set of execution data;
Barjic-2 further teaches:
A computer-implemented method, conducted during an execution of a directed graph, comprising: ([0020], “These steps are conducted to evaluate the response of the simplified version of the directed graph in order to determine which portions of the graph have less of an impact on the overall execution.”)
generating a set of execution data; ([0020], “a collection of execution data is obtained during the application of the pilot input tensor.”)
generating a summary of a set of directed graph data of the directed graph; ([0019], “The flow chart begins with step 201 of deriving a simplified version of the directed graph. The simplified version of the graph can be executed by the processor more efficiently than the directed graph itself.”)
generating a summary of the set of execution data of the execution of the neural network ([0033], “For example, the execution data can be a set of execution data values such as the outputs of each hidden layer in an ANN (i.e., wherein under the broadest reasonable interpretation (BRI), execution data is interpreted to include outputs produced during execution of a neural network, thus a summary of execution data is generated)”
the summary of the set of execution data comprising a reduced-dimension representation of the set of execution data; ([0027], “The simplified version of the directed graph may be a down-sampled version of the directed graph or a version in which individual values in the directed graph were rounded, replaced by more basic data structures, or otherwise simplified. An example of the replacement of a value with a more basic data structure is the replacement of a high precision floating point data structure with a low precision fixed point data structure. The simplified version of the directed graph could also exhibit more dramatic differences as compared to the original directed graph. For example, the simplified version could have vertices and edges associated with tensors of lower rank or dimensionality [comprising a reduced-dimension representation] (i.e., wherein under the broadest reasonable interpretation (BRI) reduced-dimension representation is interpreted to be lower rank or dimensionality, (Applicants’ SPEC [0016], “transform to the data to reduce the dimensionality of the data structure. For example, the transform could produce a lower rank approximation”) than those corresponding with the respective vertices and edges of the original directed graph”)
generating a prediction using the summary of the set of directed graph data and the summary of the set of execution data; ([0020], “The flow chart continues with steps 202 and 203 in which a pilot input tensor is applied to the simplified version of directed graph [directed graph], and a collection of execution data [execution data] is obtained during the application of the pilot input tensor. These steps are conducted to evaluate the response of the simplified version of the directed graph in order to determine which portions of the graph have less of an impact on the overall execution”…([0033], “For example, the execution data values could each uniquely correspond with a set of vertices in the directed graph, each vertex in the set of vertices could product a contribution to the inference tensor [prediction] produced by the directed graph, and each execution data value cold be proportional in magnitude to the contribution to the inference tensor [prediction] of each vertex.”(i.e., wherein the directed graph and execution data contributes to the prediction))
The motivation for claim 19 is the same as the motivation of claim 1.
Regarding claim 20:
Bajic-1 as modified by Bajic-2 teaches the method of claim 19.
Bajic-1 further teaches:
the summary of the set of execution data is generated using a second average pooling operation; and (Col 9, lines 23-32, “The values are generated by component computation approximations in accordance with this disclosure because an integer multiplication is less computationally intensive than a multiplication involving decimal numbers. In this example, the direct calculation output 302 is the average of the component computation approximations. Note that this situation is one in which the reference value will be common for a set of component computations and is determined after the set of approximations has been calculated.”)
the prediction is generated using a same type of operation as the composite computation with the summary of the set of directed graph data and the set of execution data as operands. (Col 6-7, line 64, “FIG. 2 provides an illustration of one class of approaches for reducing the computational complexity of a composite computation. FIG. 2 includes two data flow diagrams that illustrate the generation of an estimated output for a composite computation. In diagram 200, input tensors [X] and [Y] are subject to a complete execution of a computation 201. In the illustrated case, the complete execution 201 involves a dot product of the two tensors to produce an output tensor [Z]. For example, the two tensors could be one dimensional vectors and the complete execution could involve a sum-of-products composite computation. The tensors [X] and [Y] could be extremely large such that complete execution 201 was computational intensive. In such a situation, data flow diagram 210 represents a more efficient process as it will generate an estimated output [Z′] [prediction] which is approximately equivalent to output [Z], while consuming less computation resources in terms of both power, time, and hardware. The tensors [X] and [Y] could alternatively be multi-dimensional and the computation could be any tensor computation including a convolution, dot product, or cross product [same type of operation])
Bajic-1 may not explicitly teach:
the summary of the set of directed graph data is generated using a first average pooling operation;
Bajic-2 further teaches:
the summary of the set of directed graph data is generated using a first average pooling operation; ([0028], “In situations in which the simplified version is derived via a down-sampling process [pooling operation], the down-sampling of the directed graph can be conducted in numerous ways. Generally, the deriving of the simplified version of the directed graph in step 201 would involve down-sampling the directed graph by a sampling factor, S. The simplified version of directed graph [the summary] would thereby be a down-sampled version of the directed graph. For example, tensors associated with the vertices and edges of the directed graph could be down-sampled by a factor of S or by taking S neighboring elements along any number of dimensions and averaging them [average]”)
The motivation for claim 20 is the same as the motivation of claim 19.
Claim(s) 7-8, 10, 12, 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bajic-1, as modified by Barjic-2, further in view of Gao et al., Non-Patent Literature (“Dynamic Channel Pruning: Feature Boosting and Suppression”).
Regarding claim 7:
Bajic-1 as modified by Bajic-2 teaches the method of claim 6.
Bajic-1 as modified by Bajic-2, does not explicitly teach:
evaluating the prediction to determine a set of salient outputs of the composite computation; wherein the set of salient outputs are generated by a set of salient component computations of the composite computation; and wherein the set of component computations and the set of salient component computations are mutually exclusive.
Gao teaches:
evaluating the prediction to determine a set of salient outputs of the composite computation; wherein the set of salient outputs are generated by a set of salient component computations of the composite computation; and wherein the set of component computations and the set of salient component computations are mutually exclusive. (Page 4, section 3, “We start with a high-level illustration (Figure 2) of how FBS accelerates a convolutional layer with batch normalization (BN). The auxiliary components (in red) predict the importance of each output channel based on the input features, and amplify the output features accordingly. Moreover, certain output channels are predicted to be entirely suppressed (or zero-valued as represented by ), such output sparsity information can advise the convolution operation to skip the computation of these channels, as indicated by the dashed arrow. It is notable that the expensive convolution can be doubly accelerated by skipping the inactive channels from both the input features and the predicted output channel saliencies.”)
Gao and Bajic-1 are both related to the same field of endeavor (i.e., neural network computation efficiency). In view of the teachings of Gao it would have been obvious for a person of ordinary skill in the art to apply the teachings of Gao to Bajic-1 before the effective filing date of the claimed invention in order to improve the efficiency of a neural network by reducing computation (Gao, Abstract, “Making deep convolutional neural networks more accurate typically comes at the cost of increased computational and memory resources. In this paper, we reduce this cost by exploiting the fact that the importance of features computed by convolutional layers is highly input-dependent, and propose feature boosting and suppression (FBS), a new method to predictively amplify salient convolutional channels and skip unimportant ones at run-time.”)
Regarding claim 8:
Bajic-1 as modified by Bajic-2 and Gao teaches the method of claim 7.
Bajic-1 as modified by Bajic-2, does not explicitly teach:
wherein: the evaluating uses a top-K operation to select the set of salient outputs.
Gao further teaches:
wherein: the evaluating uses a top-K operation to select the set of salient outputs. (Page 5, Section 3.3, “Here, wtak(z) is a k-winners-take-all function, i.e. it returns a tensor identical to z, except that we zero out entries in z that are smaller than the k largest entries in absolute magnitude.”)
The motivation for claim 8 is the same as the motivation of claim 7.
Regarding claim 10:
Bajic-1 as modified by Bajic-2 teaches the method of claim 9.
Bajic-1 as modified by Bajic-2, does not explicitly teach:
the determining of the set of salient neural network data uses a transpose of the summary of the set of execution data.
wherein: the evaluating uses a top-K operation to select the set of salient outputs;
Gao further teaches:
the determining of the set of salient neural network data uses a transpose of the summary of the set of execution data. (Section 3.4, “To avoid significant computational cost in gl , we subsample xl−1 by reducing the spatial dimensions of each channel to a scalar using the following function ss : R C×H×W → R C : ss(xl−1) = 1 HW h s x [1] l−1 s x [2] l−1 · · · s x [C] l−1 i , (7) where s x [c] l−1 reduces the c th channel of z to a scalar using, for instance, the ` 1 -norm kx [c] l−1 k1, ` 2 -norm, `∞-norm, or the variance of x [c] l−1 . The results in Section 4 use the ` 1 - norm by default, which is equivalent to global average pooling for the ReLU activated xl−1. We then design gl(xl−1) to predict channel saliencies with a fully connected layer following the subsampled activations ss(xl−1), where φl ∈ R C l×C l−1 is the weight tensor of the layer: gl (xl−1) = (ss(xl−1) φl + ρl)+.” i.e. wherein the matrix is manipulated to determine saliency.)
wherein: the evaluating uses a top-K operation to select the set of salient outputs; and (Page 5, Section 3.3, “Here, wtak(z) is a k-winners-take-all function, i.e. it returns a tensor identical to z, except that we zero out entries in z that are smaller than the k largest entries in absolute magnitude.”)
The motivation for claim 10 is the same as the motivation of claim 7.
Regarding claim 12:
Bajic-1 as modified by Bajic-2 teaches the method of claim 11.
Bajic-1 does not explicitly teach:
the determining of the set of salient execution data uses a transpose of the summary of the set of neural network data.
wherein: the evaluating uses a top-K operation to select the set of salient outputs;
Gao further teaches:
the determining of the set of salient execution data uses a transpose of the summary of the set of neural network data. (Section 3.4, “To avoid significant computational cost in gl , we subsample xl−1 by reducing the spatial dimensions of each channel to a scalar using the following function ss : R C×H×W → R C : ss(xl−1) = 1 HW h s x [1] l−1 s x [2] l−1 · · · s x [C] l−1 i , (7) where s x [c] l−1 reduces the c th channel of z to a scalar using, for instance, the ` 1 -norm kx [c] l−1 k1, ` 2 -norm, `∞-norm, or the variance of x [c] l−1 . The results in Section 4 use the ` 1 - norm by default, which is equivalent to global average pooling for the ReLU activated xl−1. We then design gl(xl−1) to predict channel saliencies with a fully connected layer following the subsampled activations ss(xl−1), where φl ∈ R C l×C l−1 is the weight tensor of the layer: gl (xl−1) = (ss(xl−1) φl + ρl)+.” i.e. wherein the matrix is manipulated to determine saliency.)
wherein: the evaluating uses a top-K operation to select the set of salient outputs; (Page 5, Section 3.3, “Here, wtak(z) is a k-winners-take-all function, i.e. it returns a tensor identical to z, except that we zero out entries in z that are smaller than the k largest entries in absolute magnitude.”)
The motivation for claim 12 is the same as the motivation of claim 7.
Regarding claim 15:
Bajic-1 as modified by Bajic-2 teaches the method of claim 1.
Bajic-1 as modified by Bajic-2, does not explicitly teach:
wherein: the execution of the neural network is conducted while training the neural network; and the set of execution data is part of a training input.
Gao further teaches:
the execution of the neural network is conducted while training the neural network; (Section 3.3, paragraph 1, “FBS similarly generates a channel-wise importance measure. Yet contrary to them, instead of using the constant BN scaling factors γl , we predict channel importance and dynamically amplify or suppress channels with a parametric function π(xl−1) dependent on the output from the previous layer xl−1. Here, we propose to replace the layer definition fl (xl−1) for each of l ∈ [1, L] with ˆfl (xl−1) which employs dynamic channel pruning:”)
and the set of execution data is part of a training input. (Introduction, paragraph 4, “the flow of information of each output channel can be amplified or restricted under the control of a “valve”. This allows salient information to flow freely while we stop all information from unimportant channels and skip their computation. Unlike pruning statically, the valves use features from the previous layer to predict the saliency of output channels. With conventional stochastic gradient descent (SGD) methods, the predictor can learn to adapt itself by observing the input and output features of the convolution operation.”)
The motivation for claim 15 is the same as the motivation of claim 7.
Claim(s) 13-14, 17-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bajic-1, as modified by Barjic-2, further in view of Baum et al., (US20180285727A1) and Heaton et al., (US11561833B1).
Regarding claim 13:
Bajic-1 as modified by Bajic-2 teaches the method of claim 1.
Bajic-1 as modified by Bajic-2 further in view of Baum, does not explicitly teach:
prior to the execution of the neural network: compiling a set of instructions for the execution of the neural network using a set of at least two processing cores;
assigning instructions from the set of instructions for generating the prediction to a first processing core in the set of at least two processing cores; and
assigning instructions from the set of instructions for executing the composite computation to the first processing core.
Heaton teaches:
prior to the execution of the neural network: compiling a set of instructions for the execution of the neural network using a set of at least two processing cores; (Col 3, lines 32-36, “The compiler may generate the set of instructions [compiling a set of instructions] upon receiving a request (e.g., from a host device) to perform the neural network processing, and provide the set of instructions to the neural network processor for execution [execution of the neural network]”…(Col 13, lines 29-31, “Computer system 302 may include a memory device 313, a runtime engine 314, a DMA engine 318, and one or more network processors 320 [a set of at least two processing cores]”)
assigning instructions from the set of instructions for generating the prediction to a first processing core in the set of at least two processing cores; and (“At operation 612, runtime engine 314 select one or more neural network processors to execute the set of instructions [assigning instructions] (i.e., wherein assigning ‘select’ a neural network processor, thereafter interpreted as the selected one is the ’first processing core’). The selection can be based on, for example, for example, whether the architecture of the neural network processors supports the execution of the set of instructions included in the instruction file, the availability of the neural network processor, the priority of the neural network processing operations (versus other neural network processing operations), an expected completion time of the neural network processing operations, etc. In some examples, runtime engine 314 can also suspend the neural network operations (e.g., for a different neural network model, for a different host device, etc.) that are underway at the selected neural network processor(s), and control the selected neural network processor(s) to execute the set of instructions. Runtime engine 314 may also move the execution of the set of instructions to a different neural network processor [a first processing core in the set of at least two processing cores] (i.e., wherein ‘move the instructions to a different neural network processor’ is interpreted that the neural network processor previously mentioned is seen as the ‘first’) based on the criteria described above, to create a computing environment in which the instructions for neural network processing can be relocated among different neural network processors. In some examples, runtime engine 314 may split the set of instructions among a plurality of neural network processors, to enable parallel execution of the instructions. With these arrangements, the execution of neural network processing operations can become more flexible and can be adapted dynamically to an operation state of a computing environment including the neural network processors.”)
assigning instructions from the set of instructions for executing the composite computation to the first processing core. (Col 22-23, lines 61-3, “At operation 614, runtime engine 314 can execute the set of instructions at the selected neural network processor using the staging area of the selected memory device, and the execution can be based on the first mapping and second mapping created at operation 610. For example, runtime engine 334 may provide the first mapping and the second mapping to a DMA engine, which can then perform access to the selected memory device on behalf of the selected neural network processor [first processing core] (i.e., wherein the selected neural network processor is interpreted as the ‘first’ previously mentioned) when the neural network processor executes instructions including the memory references.”…(Col 23, lines 8- 17, “At operation 624, runtime engine 314 may control an internal memory of the neural network processor (e.g., state buffer 322) to retrieve first data of the input data from the input data [executing the composite computation] area of an external memory device (e.g., memory device 313) based on a first memory reference (of the first set of memory references) included in the first instruction. In some examples, retrieval of the first data can be through the DMA engine which receives the first mapping and second mapping information from runtime engine 314 prior to the execution of the first instruction.”)
A person of ordinary skill in the art would reasonably find the teachings of Heaton to be helpful in solving the problem of managing tasks of a neural network present in Bajic-1. In view of the teachings of Heaton it would have been obvious for a person of ordinary skill in the art to apply the teachings of Heaton to Bajic-1 before the effective filing date of the claimed invention in order to improve the computation of a neural network (Heaton, Col 2, lines 34-45, “Moreover, the runtime engine may perform resource placement based on the set of instructions and upon receiving a request from a host device to perform the neural network processing operations. For example, upon receiving the request, the runtime engine may select the aforementioned neural network processor and the memory device from a set of available neural network processors and memory devices to provide, respectively, the computation and memory resources for the requested neural network processing. The runtime engine can allocate a memory space in the selected memory device to become a staging area for the neural network processing operations.”)
Regarding claim 14:
Bajic-1 as modified by Bajic-2 teaches the method of claim 1.
Bajic-1 as modified by Bajic-2 and further in view of Heaton, does not explicitly teach:
the set of neural network data is a block of neural network data; the block of neural network data: (i) holds a set of network data elements of the neural network; (ii) is larger than a single network data element; and (iii) is smaller than a layer of the neural network; and the composite computation uses the block of neural network data as an operand.
Baum teaches:
the set of neural network data is a block of neural network data; the block of neural network data: ([0101], "FIG. 5. The NN processing engine or core 60 comprises several hierarchical computation units [block of neural network data]. The lowest hierarchical level is the processing element (PE) 76 with its own dedicated internal Layer 1 or L1 memory 78 in which individual neurons are implemented [neural network data].)"… and holds a set of network data elements of the neural network; ([0111], "A first feature comprises the compute fabric (or compute capability) provided by the computation units that are organized into various aggregation levels or hierarchical levels, such as PEs, subclusters, clusters, NN cores as described in the example system disclosed herein. The compute fabric comprises the basic compute elements [network data elements] that are configured to address the special nature of the computational needs of ANNs [neural network])"… and (ii) is larger than a single network data element; and (iii) is smaller than a layer of the neural network ([0105], "In another embodiment, the NN processor is capable of implementing multiple ANNs in parallel, where each ANN has one or more network layers. The NN processor is adapted to simultaneously process one or more input data [larger than a single network data element] (i.e., wherein one or more input used) streams associated with the ANNs. Since the architecture of the NN device resembles the structure of an ANN, multiple ANNs can be viewed as a single wide ANN...In this case, due to the enormous parallelism of the device, each set of resources grouped within a ‘layer’ of the ANN is independent from each other. In addition, the computation elements of the NN processor are operative to function at any desired granularity of a subset of the input data stream [smaller than a layer of the neural network] (i.e., wherein the subset of input data is seen as smaller than a layer) thereby trading off memory element usage versus latency, as described in more detail infra. The NN processor of the present invention uses several design principles in its implementation including: (1) just in time usage of system resources; (2) dynamic allocation of system resources per need; (3) leveraging both the time-domain and the space-domain to optimize utilization and efficiency; and (4) balanced load over available system resources.)"
the composite computation uses the block of neural network data as an operand ([0111], “A first feature comprises the compute fabric (or compute capability) provided by the computation units [block of neural network data] that are organized into various aggregation levels or hierarchical levels, such as PEs, subclusters, clusters, NN cores as described in the example system disclosed herein. The compute fabric comprises the basic compute elements that are configured to address the special nature of the computational needs of ANNs [composite computation]. Several features of the compute fabric include: (1) a lean circuit architecture thereby allowing a relatively large number of physical entities to be implemented; (2) a large number of multiply and accumulate operations at once, where additions are performed as accumulations; (3) flexibility of number representation, including integer and floating point as well as different bit widths; (4) quad-multiplier support allowing for higher resolution computations; and (5) N-way ALU support to provide the capability of optimizing memory bandwidth, i.e. instead of performing a single operation per cycle such as y←y+w*x, a more complex operation such as y←y+w1*x1+w2*x2 can be implemented [as an operand] (i.e., wherein values of w1*x1+w2*x2) which reflects a trade-off between an increase in silicon complexity and reduced memory access required.”
Baum and Bajic-1 are both related to the same field of endeavor (i.e., neural network computation efficiency). In view of the teachings of Baum it would have been obvious for a person of ordinary skill in the art to apply the teachings of Baum to Bajic-1 before the effective filing date of the claimed invention in order to improve the efficiency of a neural network by reducing computation (Baum, Abstract, “A novel and useful neural network (NN) processing core adapted to implement artificial neural networks (ANNs) and incorporating processing circuits having compute and local memory elements. The NN processor is constructed from self-contained computational units organized in a hierarchical architecture. The homogeneity enables simpler management and control of similar computational units, aggregated in multiple levels of hierarchy. Computational units are designed with minimal overhead as possible, where additional features and capabilities are aggregated at higher levels in the hierarchy. On-chip memory provides storage for content inherently required for basic operation at a particular hierarchy and is coupled with the computational resources in an optimal ratio.”)
Regarding claim 17:
Barjic-1 further teaches:
the method comprising: generating a set of execution data; (Col 4, lines 46-50, “The computation of the outputs of one layer of the ANN, or sub-sections of one layer of the ANN, can be considered a component computation to a composite computation [a set execution data] consisting of the overall execution of the ANN.”)
generating a prediction using the summary of the set of neural network data and the summary of the set of execution data; (Col 5, lines 59-62, “The partial execution will generate an estimated output [prediction] which may not be equivalent to the precise output that would have been generated by a complete execution of the composite computation.”)…(Col 6, lines 45-48, “FIG. 2 includes two data flow diagrams that illustrate the generation of an estimated output for a composite computation. In diagram 200, input tensors [X] and [Y] [neural network data] are subject to a complete execution of a computation.”)…(Col 7, lines 48-66, “Data flow diagram 210 also includes optional step 212 of generating a reference value [execution data]...Step 212 can utilize one or both of input tensors [X] and [Y]. The reference value can be utilized to more accurately determine the priority that any given component computation should be given. For example, if tensors [X] and [Y] included values with widely varying orders of magnitude, the reference value could be set equal to the largest order of magnitude occupied by the values of the input tensors, and component computation approximations that were not at or above a fraction of that order of magnitude would be suppressed. The reference value used in evaluating a component computation approximation can also be derived from other component computation approximations in the same composite computation.”)
executing a composite computation, wherein the composite computation is required for the execution of the neural network; and (Col 2, lines 21- 23, “A complete execution of the composite computation generates a precise output and requires execution of a set of component computations”)
suppressing a set of component computations of the composite computation, (Col 6, lines 33-35, “The contribution estimate can be used to suppress individual component computations while preserving the fidelity of the partial execution to the complete execution.”)
wherein the set of component computations is at least partly determined by the prediction. (Col 4, lines 53-60, “Individual sum-of-products computations, convolutions of a filter or set of filters with a set of input data, and multi-product computations such as matrix multiplications are all examples of composite computations that are amenable to optimization because they consist essentially of component computations that can alternatively be suppressed or fully executed based on an estimate [prediction] of their contribution”)
Bajic-2 further teaches:
generating a summary of a set of neural network data of the neural network; ([0030], “The deriving of the simplified version of a neural network could include replacing a set of weight values, filter values, or any other value used in the computation of the neural network,”)
generating a summary of the set of execution data of the execution of the neural network ([0033], “For example, the execution data can be a set of execution data values such as the outputs of each hidden layer in an ANN (i.e., wherein under the broadest reasonable interpretation (BRI), execution data is interpreted to include outputs produced during execution of a neural network, thus a summary of execution data is generated)”
the summary of the set of execution data comprising a reduced-dimension representation of the set of execution data; ([0027], “The simplified version of the directed graph may be a down-sampled version of the directed graph or a version in which individual values in the directed graph were rounded, replaced by more basic data structures, or otherwise simplified. An example of the replacement of a value with a more basic data structure is the replacement of a high precision floating point data structure with a low precision fixed point data structure. The simplified version of the directed graph could also exhibit more dramatic differences as compared to the original directed graph. For example, the simplified version could have vertices and edges associated with tensors of lower rank or dimensionality [comprising a reduced-dimension representation] (i.e., wherein under the broadest reasonable interpretation (BRI) reduced-dimension representation is interpreted to be lower rank or dimensionality, (Applicants’ SPEC [0016], “transform to the data to reduce the dimensionality of the data structure. For example, the transform could produce a lower rank approximation”) than those corresponding with the respective vertices and edges of the original directed graph”)
Barjic-1 as modified by Barjic-2 and further in view of Heaton, does not explicitly teach:
A system comprising: a set of processing cores;
a network communicatively connecting the processing cores;
a neural network stored on a set of memories across the set of processing cores;
a set of processing pipelines on the set of processing cores; and
a set of instructions stored on the set of memories across the set of processing cores, wherein the set of instructions, when executed by the set of processing pipelines, cause the system to execute the neural network and conduct a method during the execution of the neural network,
Baum further teaches:
A system comprising: a set of processing cores; ([0099], “A high-level block diagram illustrating an example system on chip (SoC) NN processing system comprising one or more NN processing cores is shown in FIG. 4. The SoC NN processing system, generally referenced 100, comprises at least one NN processor integrated circuit (or core)”)
a network communicatively connecting the processing cores; ([0101], “A plurality of bus interfaces 86 (i.e. chip-to-chip interfaces) couple the NN processor to other off-chip NN processor chips for additional network capacity. Bus interface 84 (i.e. chip-to-chip interface) couples the NN processor to a conventional rule based machine (RBM) co-processor 88 comprising a CPU 90, instruction memory 92 and data memory 94.”)
a neural network stored on a set of memories across the set of processing cores; ([0101], “The lowest hierarchical level is the processing element (PE) 76 with its own dedicated internal Layer 1 or L1 memory 78 in which individual neurons are implemented. A plurality of N PEs 76 along with dedicated Layer 2 or L2 memory 74 make up the next hierarchical level termed a subcluster 70. A plurality of M subclusters 70 along with dedicated Layer 3 or L3 memory 72, a plurality of activation function circuits 80, and a plurality of layer controller (LC) circuits 82 make up a cluster 66. A plurality of L clusters along with dedicated Layer 4 or L4 memory 64 are in the NN processor core 60”)
a set of processing pipelines on the set of processing cores; and ([0103], “the NN manager 62 is a specialized processor that controls two data pipes: one parallel and one serial along with functions to drive the network fabric. This processor carries out special purpose operations that are native to the control plane of the neural network. Example operations includes, but are not limited to, Infer, Train, Load weights, and Update weights. Load balancing and resource allocation are handled by an external software tool chain, which includes a set of tools including a compiler, mapper, and allocator, that address these tasks.”)
a set of instructions stored on the set of memories across the set of processing cores, ([0068], “The invention may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.”)
wherein the set of instructions, when executed by the set of processing pipelines, cause the system to execute the neural network and conduct a method during the execution of the neural network, ([0103]-[0105], “In one embodiment, the NN manager 62 is a specialized processor that controls two data pipes: one parallel and one serial along with functions to drive the network fabric. This processor carries out special purpose operations that are native to the control plane of the neural network. Example operations includes, but are not limited to, Infer, Train, Load weights, and Update weights. Load balancing and resource allocation are handled by an external software tool chain, which includes a set of tools including a compiler, mapper, and allocator, that address these tasks. In one embodiment, the NN processor includes shared memory for the storage of weights and dedicated memory elements are for storing contexts thereby enabling relatively high data processing bandwidth. In addition, the NN processor includes data and control planes that are strictly separate from each other and that provide out of band control to the computation elements. Moreover, the NN processor includes a configurable interconnect between aggregation levels to yield a dynamic and programmable data pipeline. In another embodiment, the NN processor is capable of implementing multiple ANNs in parallel, where each ANN has one or more network layers. The NN processor is adapted to simultaneously process one or more input data streams associated with the ANNs.”)
The motivation for claim 17 is the same motivation for claim 14.
Regarding claim 18:
Bajic-1 as modified by Bajic-2 and further in view of Baum teaches the method of claim 17.
Bajic-1 as modified by Bajic-2 and further in view of Baum, does not explicitly teach:
a compiler programmed to conduct a second method prior to the execution of the neural network, the second method comprising: compiling the set of instructions;
assigning instructions from the set of instructions for generating the prediction to a first processing core in the set of processing cores; and
assigning instructions from the set of instructions for executing the composite computation to the first processing core.
Heaton further teaches:
a compiler programmed to conduct a second method prior to the execution of the neural network, the second method comprising: compiling the set of instructions; (Col 3, lines 32-36, “The compiler [a compiler] may generate the set of instructions [a set of instructions] upon receiving a request (e.g., from a host device) to perform the neural network processing, and provide the set of instructions to the neural network processor for execution [prior to the execution of the neural network].”
assigning instructions from the set of instructions for generating the prediction to a first processing core in the set of at least two processing cores; and (“At operation 612, runtime engine 314 select one or more neural network processors to execute the set of instructions [assigning instructions] (i.e., wherein assigning ‘select’ a neural network processor, thereafter interpreted as the selected one is the ’first processing core’). The selection can be based on, for example, for example, whether the architecture of the neural network processors supports the execution of the set of instructions included in the instruction file, the availability of the neural network processor, the priority of the neural network processing operations (versus other neural network processing operations), an expected completion time of the neural network processing operations, etc. In some examples, runtime engine 314 can also suspend the neural network operations (e.g., for a different neural network model, for a different host device, etc.) that are underway at the selected neural network processor(s), and control the selected neural network processor(s) to execute the set of instructions. Runtime engine 314 may also move the execution of the set of instructions to a different neural network processor [a first processing core in the set of at least two processing cores] (i.e., wherein ‘move the instructions to a different neural network processor’ is interpreted that the neural network processor previously mentioned is seen as the ‘first’) based on the criteria described above, to create a computing environment in which the instructions for neural network processing can be relocated among different neural network processors. In some examples, runtime engine 314 may split the set of instructions among a plurality of neural network processors, to enable parallel execution of the instructions. With these arrangements, the execution of neural network processing operations can become more flexible and can be adapted dynamically to an operation state of a computing environment including the neural network processors.”)
assigning instructions from the set of instructions for executing the composite computation to the first processing core. (Col 22-23, lines 61-3, “At operation 614, runtime engine 314 can execute the set of instructions at the selected neural network processor using the staging area of the selected memory device, and the execution can be based on the first mapping and second mapping created at operation 610. For example, runtime engine 334 may provide the first mapping and the second mapping to a DMA engine, which can then perform access to the selected memory device on behalf of the selected neural network processor [first processing core] (i.e., wherein the selected neural network processor is interpreted as the ‘first’ previously mentioned) when the neural network processor executes instructions including the memory references.”…(Col 23, lines 8- 17, “At operation 624, runtime engine 314 may control an internal memory of the neural network processor (e.g., state buffer 322) to retrieve first data of the input data from the input data [executing the composite computation] area of an external memory device (e.g., memory device 313) based on a first memory reference (of the first set of memory references) included in the first instruction. In some examples, retrieval of the first data can be through the DMA engine which receives the first mapping and second mapping information from runtime engine 314 prior to the execution of the first instruction.”)
The motivation for claim 18 is the same as the motivation of claim 14.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AMINA BENOURAIDA whose telephone number is (571)272-4340. The examiner can normally be reached Monday-Friday 8:30am-5pm ET..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael J. Huntley can be reached at (303) 297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AMINA MORENO BENOURAIDA/Examiner, Art Unit 2129
/MICHAEL J HUNTLEY/Supervisory Patent Examiner, Art Unit 2129
Prosecution Insights