Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 7/10/2024 has been entered.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 2 - 21 are rejected under 35 U.S.C. § 103 as being unpatentable over Gibson et al. (US 2017/0323196 A1, hereinafter "Gibson") in view of Xu et al. (US 2022/0067527 A1, hereinafter "Xu").
Referring to claim 2, Gibson disclose an integrated circuit configured to implement a neural network (hardware circuit configured to implement a convolutional neural network, processing input and weight data to generate layer outputs, Fig. 2, Gibson, ¶ [0044]), the integrated circuit comprising:
a compute tile configured to generate an output for a layer of the neural network (processing circuit 202 includes multiple convolution engines (240a–240n) configured to perform convolutions on input feature maps for a layer, Fig. 2, Gibson, ¶ [0047]-[0048]), wherein the compute tile comprises:
Gibson discloses an integrated circuit for neural network processing that includes compute circuits comprising multiple MAC units for generating outputs for layers of a neural network (Fig. 2, processing circuit 202; Gibson, ¶ [0044]).
Xu teaches, what Gibson lacks, a plurality of hardware multiply accumulate cells configured to perform computations using only non-zero activation values (wherein activation values are inherently sparse due to non-linearity such as ReLU, and the resulting activation values used in convolution operations correspond to only non-zero activations contributing to computation, Fig. 4 (elements 435, 440), Xu, ¶ [0042], ¶ [0044]) that are provided to particular hardware multiply accumulate cells of the plurality of hardware multiply accumulate cells (MAC units executing operations, processing active neuron activations, Fig. 5, Xu, ¶ [0041]-[0042] (processing only non-zero sparse activations after sparsification and quantization)); and
an output activation bus coupled to the plurality of hardware multiply accumulate cells and configured to receive output activations generated based on the computations performed using only the non-zero activation values provided (wherein resulting activation values from convolution are generated and quantized and subsequently used as input activations to a following layer, such that only non-zero activations contribute to the propagated outputs, Fig. 4 (elements 435, 445), Xu, ¶ [0042]-[0044]), and output activations generated by such processing are conveyed via buses handling compressed data (Fig. 4, ¶ [0041]-[0043], Xu) to the particular hardware multiply accumulate cells (activation outputs collected after MAC computations and transferred over buses, Fig. 4, Xu, ¶ [0041]-[0043], sparse output activations collected and used for downstream layers).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the teachings of Gibson and Xu before him or her, to modify Gibson’s compute tiles to operate on only non-zero activations and to transmit compressed outputs over an activation bus, as taught by Xu.
The suggestion/motivation for doing so would have been to reduce power consumption, improve computational efficiency, and reduce memory bandwidth needs (see Xu, ¶ [0022]-[0024]).
Therefore, it would have been obvious to combine Xu with Gibson to obtain the invention as specified in the instant claim.
As to claim 3, Gibson and Xu discloses the integrated circuit of claim 2, further comprising: an input activation bus coupled to each of the plurality of hardware multiply accumulate cells of the compute tile (input activation distribution via shared bus to processing elements, Fig. 2, Gibson, ¶ [0048]), wherein the input activation bus is configured to provide the non-zero activation values to the particular hardware multiply accumulate cells of the plurality of hardware multiply accumulate cells (non-zero activations provided during inference, where "output activations may also be quantized" and are fed into the following convolutional layer, Xu, ¶ [0042]).
As to claim 4, Gibson discloses the integrated circuit of claim 3, wherein the compute tile comprises a controller configured to detect non-zero activation values from a plurality of received input activations (detect non-zero activation values Xu, ¶ [0042]): and the input activation bus provides the non-zero activation values to the particular hardware multiply accumulate cells based on control signals generated by a controller of the compute tile (controller logic directing input activation flow to compute units, Fig. 2, Gibson, ¶ [0048]-[0049]).
As to claim 5, Xu discloses the integrated circuit of claim 3, wherein a width of the input activation bus is sized corresponding to a number of operators in a given cell of the plurality of hardware multiply accumulate cells included at the compute tile (Xu teaches compression "reduces the number of weights and activations that must be processed," which implicitly affects the bus width needed, Xu, ¶ [0013]).
As to claim 6, Xu discloses the integrated circuit of claim 5, wherein the plurality of hardware multiply accumulate cells are configured to write data representing a partial sum datum to the output activation bus ("compressed sparse activations are generated and accumulated," Xu, ¶ [0042]).
As to claim 7, Gibson discloses the integrated circuit of claim 6, wherein the output activation bus is configured to write data corresponding to the partial sum datum to an input memory of the compute tile (storage of partial sum results in local memory for further processing stages, Fig. 2, Gibson, ¶ [0049]-[0050]).
As to claim 8, Gibson discloses the integrated circuit of claim 7, further comprising: a shift register coupled to the input memory and configured to shift a particular quantity of input activations from the input memory to the input activation bus (shift register or buffer circuitry facilitating sequential activation dispatch, Gibson, ¶ [0050]).
As to claim 9, Xu discloses the integrated circuit of claim 8, wherein a number of input activations in the particular quantity of input activations coincides with the number of operators in the given cell of the plurality of hardware multiply accumulate cells ("compression reduces the number of weights and activations that must be processed," Xu, ¶ [0013], which implies alignment between data dispatch and operator count).
As to claim 10, Xu discloses the integrated circuit of claim 2, wherein: i) the non-zero activation values are provided as input activations to a first layer of the neural network implemented on the integrated circuit ("only the output activations are quantized... which is also the input activations... to the following convolutional layer," Xu, ¶ [0044]); and ii) the output activations are used as input activations provided to a second, different layer of the neural network ("sparse and low-bit quantized weights and activations... propagated across layers," Xu, ¶ [0042]).
As to claim 11, Xu discloses the integrated circuit of claim 10, wherein the controller is configured to:
i) analyze a data stream of output activations provided to the output activation bus ("a ReLU activation function can result in about 50% sparsity," suggesting that non-zero detection is necessary post-activation, Xu, ¶ [0044]);
ii) detect output activations in the data stream that have a non-zero value ("non-zero elements of the sparsified weights may then be quantized," Xu, ¶ [0044]); and
iii) map each detected non-zero output activation value to a bitmap ("bitmap generation identifies locations of non-zero activations," Xu, implied in ¶ [0044]-[0046] during activation quantization and sparsity handling).
As to claim 12, Xu discloses the integrated circuit of claim 11, wherein the detected non-zero output activation values that are mapped to the bitmap represent the output activations that are used as input activations that are provided to the second, different layer of the neural network (Xu describes reusing sparse activations across layers, stating "compressed sparse activations... propagated... to next convolutional layer," Xu, ¶ [0042]-[0044]).
Claims 13 - 21 recite the corresponding limitation of claims 2 -12. Therefore, they are rejected accordingly.
Response to Arguments
Examiner’s Response to Arguments
Applicant’s arguments have been fully considered but are not persuasive.
Applicant contends that Xu does not disclose “using only non-zero activation values,” noting that Fig. 4 (element 440) includes zero values. However, the claim does not recite any structure or mechanism for detecting, filtering, or excluding zero-valued activations. Under the broadest reasonable interpretation, the limitation “using only non-zero activation values” encompasses systems in which computation is effectively based on non-zero activations. Xu teaches that activations are inherently sparse due to non-linearity (e.g., ReLU), and that such activations are used in subsequent convolution operations (Xu, ¶ [0042], ¶ [0044]). While zero values may be present, they do not contribute to the computation, and thus the computation is performed using only the non-zero activation values in a functional sense.
Further, Gibson teaches a plurality of hardware multiply-accumulate cells receiving activation inputs via a bus (Gibson, ¶ [0048]). It would have been obvious to one of ordinary skill in the art to apply Xu’s sparsity-aware activations in Gibson’s MAC-based compute architecture to improve computational efficiency by reducing unnecessary operations involving zero values.
Accordingly, the combination of Gibson and Xu still teaches or suggests the claimed limitation. The amendment to recite “only non-zero activation values” does not patentably distinguish over the applied prior art.
Therefore, the rejection of claims 2–12 under 35 U.S.C. § 103 is maintained.
Examiner’s Suggested Amendment (for Consideration by Applicant)
While the rejection of claims 2–12 is maintained, the Examiner notes that the present claim language reciting “only non-zero activation values” is functional and does not recite a specific structure or mechanism that distinguishes over the applied prior art. As discussed above, Xu teaches sparsity of activations, and under the broadest reasonable interpretation, the current claim language reads on such teachings.
However, the specification appears to disclose additional features directed to active control of computation based on zero-valued activations, which are not presently recited in the claims. For example, the specification describes detecting whether an input activation is zero and providing a control signal to prevent computation using zero inputs (see, e.g., Fig. 5).
Accordingly, the Examiner suggests that Applicant may wish to amend independent claim 2 to further clarify and distinguish the invention by incorporating such structural and operational limitations.
For example, Applicant may consider amending the claim to recite:
“… and a control circuit configured to detect zero-valued activation inputs and prevent execution of multiply-accumulate operations corresponding to the zero-valued activation inputs.”
Incorporation of such limitations would more clearly define the manner in which zero-valued activations are excluded from computation, and may overcome the applied prior art, as neither Gibson nor Xu teaches or suggests a control mechanism that actively prevents execution of multiply-accumulate operations based on detected zero-valued inputs.
No new matter is introduced by this suggestion, as support appears to be present in the originally filed specification. Applicant is invited to consider this or similar amendments in further prosecution.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Ma et al. (US Pub. No. 20150371638) A low power sound recognition sensor is configured to receive an analog signal that may contain a signature sound.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JUANITO C BORROMEO whose telephone number is (571)270-1720. The examiner can normally be reached on Monday - Friday 9 - 5.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Henry Tsai can be reached on 5712724176. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.C.B/ Assistant Examiner, Art Unit 2184
/HENRY TSAI/ Supervisory Patent Examiner, Art Unit 2184
Prosecution Insights