DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013 is being examined under the first inventor to file provisions of the AIA .
Status of the Application
This action is a first action on the merits in response to the application filed on 02/07/2025.
Status of Claims
Claims 1-13, 16, and 21-25 filed on 07/09/2026 are currently pending and have been examined in this application.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 02/07/2025 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Allowable Subject Matter
Claims 10 and 24 objected to as being dependent upon a rejected base claim, but it appears they would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or non-obviousness.
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claims 1-9, 11-13, 16, 21-23, and 25 are rejected under 35 U.S.C. 103 as being un-patentable over Han et al. EIE: Efficient Inference Engine on Compressed Deep Neural Network, 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture in view of Lamb et al. (US 20190087713 A1)
Regarding claim 1. Han teaches A deep neural network (DNN) parallel processor that is flexible, hardware programmable, scalable, and reconfigurable, the DNN parallel processor comprising a plurality of neural networks processing units (NPUs) configured to process artificial intelligence (AI)/machine learning (ML) input data, each of the plurality of NPUs comprising: [Han teaches in page 248 column 2 last para. “output activation read and input activation multiply (in parallel), shift and add, and output activation write” wherein DNN parallel processor. Page 244 column 1 second para. Han teaches “EIE is a scalable array of processing elements (PEs) Every PE stores a partition of network in SRAM and performs the computations associated with that part” wherein processing units (NPUs)] an activation (A) map memory; a weight (W) map memory; a compressed A memory; a pruned W memory; [Han teaches “pruning the redundant connections and having multiple connections share the same weight. We propose an energy efficient inference engine (EIE) that performs inference on this compressed network model and accelerates the resulting sparse matrix-vector multiplication with weight sharing” see abstract. Han also teaches in page 247 column 1 4th para. “The Activation Read/Write Unit contains two activation register files that accommodate the source and destination activation values respectively during a single round of FC layer computation”] a control logic block coupled to outputs of the A map memory and the W map memory and to inputs of the compressed A memory and the pruned W memory; [Han teaches, page 246 column 2 3rd para. “Figure 4 shows the architecture of EIE. A Central Control Unit (CCU) controls an array of PEs that each computes one slice of the compressed network. The CCU also receives non-zero input activations from a distributed leading nonzero detection network and broadcasts these to the PEs.” Wherein the CCU is equivalent to control logic block] a routing multiplexer coupled to outputs of the control logic block, the compressed A memory, and the pruned W memory; [Han teaches, in page 247 column 1 3th para. “A bypass path is provided to route the output of the adder to its input if the same accumulator is selected on two adjacent cycles” Wherein output router. The router multiplexer is inherent in sparse architectures to direct irregularly spaced non-zero activations and weights to matching execution units]
Han does not specifically teach, however, Lamb teaches a multiplier-accumulator (MAC) coupled to an output of the routing multiplexer; and a rectified linear unit (ReLU) or other non-linear function coupled to an output of the MAC; wherein the plurality of NPUs are connected together to perform DNN functions Lamb, para. 0133, Lamb teaches “The multiplier 706 may perform a multiplication operation on the selected activation and a weight of the sparse weight vector that corresponds with the selected activation.” Further, a multiplier-accumulator (MAC)... and a ReLU or other non-linear function... are Standard components of any fundamental baseline processing element (PE) array or Neural Processing Unit (NPU) that would have been obvious to a skilled in the art to modify/combine with the teaching of Han.
Regarding claim 2. Han in view of Lamb teaches all of the limitations of claim 1 (as above). Further, Han teaches The DNN parallel processor of claim 1, wherein the DNN functions comprise ML DNN functions [Han teaches, in page 243 figure 1 “Efficient inference engine that works on the compressed deep neural network model for machine learning applications”].
Regarding claim 3. Han in view of Lamb teaches all of the limitations of claim 1 (as above). Further, Han teaches The DNN parallel processor of claim 2, wherein the control logic block is configured to calculate an amount of memory locations in the compressed A memory and the pruned W memory to skip based on DNN accelerator (DNA) words [Han teaches, in page 243 column 1 second para. “Exploiting sparsity saves 10×; Weight sharing gives 8×; Skipping zero activations from ReLU saves another 3×” and page 246 column 2 first para. teaches “Columns corresponding to zeros in a are completely skipped”].
Regarding claim 4. Han in view of Lamb teaches all of the limitations of claim 1 (as above). Further, Han teaches wherein the control logic block is configured to skip all the pruned W memory and the compressed A memory zero- multiplied locations such that all zero multiplications are removed from DNN calculations for full sparsity removal [Han teaches, in page 243 column 1 second para. “Exploiting sparsity saves 10×; Weight sharing gives 8×; Skipping zero activations from ReLU saves another 3×” and page 246 column 2 first para. teaches “Columns corresponding to zeros in a are completely skipped” wherein all zero multiplications are removed from DNN calculations].
Regarding claims 5-6. wherein the A map memory receives inputs from an internal memory, an external memory, an input/output (I/O) interface, or the ReLU. Wherein the W map memory receives inputs from an internal memory, an external memory, or an input/output (I/O) interface. Storing map components or raw weights/activations in an internal SRAM cache, external DRAM, or streaming them over an I/O bus interface is an ubiquitous feature of standard memory hierarchies in edge and server computing. As a result, claims 5-6 do not serve a specific technical purpose and cannot contribute to the invention's technical character. Claims 5-6 do not produce any technical effect beyond the provision of their computer implementation. Features that do not contribute to an invention's technical character cannot support the presence of an inventive step. For these reasons, when confronted with the objective technical problem, the skilled person would implement the subject matter of claims 5-6 in the context of closest prior art Han and arrive at the subject matter of claims 5-6 without exercising any inventive skill.
Regarding claims 7-8. wherein: each of the DNA words comprises a W_RNA word from the W map memory and an A_RNA word from the A map memory; and each of the W_RNA word and the A_RNA word have a same length of n bits, wherein the control logic block is configured to: receive DNN accelerator (DNA) words as inputs; calculate a next address for each of the compressed A memory and the pruned W memory based on the corresponding DNA word; and calculate connectivity of the routing multiplexer based on the corresponding DNA word. Splitting index tracking into bitwise segments of length \(n\) (Claims 7-8) is a common design pattern for memory alignment. As a result, claims 7-8 do not serve a specific technical purpose and cannot contribute to the invention's technical character. Claims 7-8 do not produce any technical effect beyond the provision of their computer implementation. Features that do not contribute to an invention's technical character cannot support the presence of an inventive step. For these reasons, when confronted with the objective technical problem, the skilled person would implement the subject matter of claims 7-8 in the context of closest prior art Han and arrive at the subject matter of claims 5-6 without exercising any inventive skill.
Regarding claim 9. Han in view of Lamb teaches all of the limitations of claim 1 (as above). Further, Han teaches further comprising: a first activation-lookup table (ALUT) memory coupled between the output of the compressed A memory of a first NPU of the plurality of NPUs and the routing multiplexer of the first NPU and between the output of the pruned W memory of the first NPU and the routing multiplexer of the first NPU, a weight-lookup table (W_LUT) memory coupled to an input of the routing multiplexer of the first NPU and connected to an input from an input/output (I/O) interface, internal memory, or external memory [Han teaches, in page 247 column 1 third para. “Because v is stored in 4-bit encoded form, it is first expanded to a 16-bit fixed point number via a table look up” wherein look-up table. Further, Han teaches in page 248 column 2 second para “The power/area breakdown is shown in Table II” wherein the power/area breakdown is equivalent to the weight look-up table].
Regarding claim 11. Han in view of Lamb teaches all of the limitations of claim 1 (as above). Han does not specifically teach, however, Lamb teaches wherein: the plurality of NPUs includes a first NPU and a second NPU; the compressed A memory of the first NPU is a first compressed A memory; the pruned W memory of the first NPU is a first pruned W memory; the MAC of the first NPU is a first MAC; the MAC of the second NPU is a second MAC; the routing multiplexer enables routing different pairs of outputs from the first compressed A memory and the first pruned W memory of the first NPU into inputs of one or both of the first MAC of the first NPU or the second MAC of the second NPU [Lamb, para. 0035, Lamb teaches “FIG. 2 illustrates an example implementation of a system 200 in accordance with certain aspects of the present disclosure. As illustrated in FIG. 2, the system 200 may have multiple local processing units 202 that may perform various operations of methods described herein. Each local processing unit 202 may include a local state memory 204 and a local parameter memory 206 that may store parameters of a neural network” wherein plurality of NPUs]
It would have been obvious for one having ordinary skill in the art before the effective filing date of the claimed invention to modify/combine teaching of Han with the multiple NPUs of Lamb since the claimed invention is merely a combination of old elements, and in the combination each element merely would have performed the same function as it did separately, with the predictable results of optimizing neural network processing performance.
Regarding claim 12. Han in view of Lamb teaches all of the limitations of claim 1 (as above). Further, Han teaches wherein: the ReLU is active only at an end of an activation function tensor multiplication calculation; the ReLU is coupled to an input of the A map memory and an input of the compressed A memory; and an output of the ReLU is stored at the compressed A memory and map bit of the output of the ReLU is stored at the A map memory [Han teaches, in page 245 column 1 second para. “b = f(Wa + v) Where a is the input activation vector, b is the output activation vector, v is the bias, W is the weight matrix, and f is the non-linear function, typically the Rectified Linear Unit (ReLU) [22] in CNN and some RNN” wherein an activation function tensor multiplication calculation].
Regarding claim 13. Han in view of Lamb teaches all of the limitations of claim 1 (as above). Further, Han teaches wherein: the ReLU is active only at an end of an activation function tensor multiplication calculation; and an output of the ReLU is stored at an internal memory, an external memory, or an input/output (I/O) interface [Han teaches, in page 245 column 1 second para. “b = f(Wa + v) Where a is the input activation vector, b is the output activation vector, v is the bias, W is the weight matrix, and f is the non-linear function, typically the Rectified Linear Unit (ReLU) [22] in CNN and some RNN” wherein an activation function tensor multiplication calculation. See also claims 5-6 for standard memory hierarchies].
Regarding claim 16, the claim recites analogous limitations to claims 1-2 above, and is therefore rejected on the same premise.
Regarding claim 21, the claim recites analogous limitations to claims 1 and 3 above, and is therefore rejected on the same premise.
Regarding claim 22, the claim recites analogous limitations to claim 4 above, and is therefore rejected on the same premise.
Regarding claim 23, the claim recites analogous limitations to claim 8 above, and is therefore rejected on the same premise.
Regarding claim 25, the claim recites analogous limitations to claim 11 above, and is therefore rejected on the same premise.
Conclusion
The following prior arts made of record and not relied upon are considered pertinent to applicant's disclosure. McClelland (US-20140279601A1), Shahi et al. (US-20090144095A1), Turakhia et al. (US 20180164866 A1). Turakhia teaches a method of reducing power consumption of a neural network, comprising: retrieving, from a tag storage, at least one tag value of a first tag value for a weight in the neural network or a second tag value for an activation in the neural network, the first tag value indicating whether the weight is zero and the second tag value indicating whether the activation is zero, wherein the weight and the activation are to be loaded to a multiplier of a multiplier-accumulator (MAC) as a pair of operands; determining whether the at least one tag value indicates a zero value; and disabling loading the weight and the activation to the multiplier when the at least one tag value indicates the zero value
Any inquiry concerning this communication from the examiner should be directed to Abdallah El-Hagehassan whose contact information is (571) 272-0819 and Abdallah.el-hagehassan@uspto.gov The examiner can normally be reached on Monday- Friday 8 am to 5 pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Rutao Wu can be reached on (571) 272-6045. The fax phone number for the organization where this application or proceeding is assigned is (571) 273-3734.
Information regarding the status of an application may be obtained from the patent application information retrieval (PAIR) system. Status information of published applications may be obtained from either private PAIR or public PAIR. Status information of unpublished applications is available through private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have any questions on access to the private PAIR system, contact the electronic business center (EBC) at (866) 271-9197 (toll-free). If you would like assistance from a USPTO customer service representative or access to the automated information system, call (800) 786-9199 (in US or Canada) or (571) 272-1000.
/ABDALLAH A EL-HAGE HASSAN/
Primary Examiner, Art Unit 3623
Prosecution Insights