Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Election/Restrictions
Restriction to one of the following inventions is required under 35 U.S.C. 121:
I. Claims 1-13 are drawn to DNN accelerator comprising plurality of NPU(s) . The method claim was classified in classified in G06N3/04.
II. Claims 14-15 are drawn to DNN parallel comprising the pluraliity of NPU(s) and the NPU comprising a weight lookup table (WLUT) memory; a plurality of multiplier accelerators (MAs), each of the MAs comprising:an accumulator coupled to an output of an activation (A) memory; and were classified in G06N5/022.
The inventions are independent or distinct, each from the other because:
Invention I and II are related product inventions. The related inventions are distinct if: (1) the inventions as claimed are either not capable of use together or can have a materially different design, mode of operation, function, or effect; (2) the inventions do not overlap in scope, i.e., are mutually exclusive; and (3) the inventions as claimed are not obvious variants. See MPEP § 806.05(j). In the instant case, the inventions as claimed can have a materially different design. In the invention I requires the an activation sparsity removal (ASR) block coupled to the AMM; a redundancy removal (RR) block coupled to the AMM; The invention II (14-15) requires a weight lookup table (WLUT) memory; a plurality of multiplier accelerators (MAs), each of the MAs comprising: an accumulator coupled to an output of an activation (A) memory. Furthermore, the inventions as claimed do not encompass overlapping subject matter and there is nothing of record to show them to be obvious variants.
Restriction for examination purposes as indicated is proper because all the inventions listed in this action are independent or distinct for the reasons given above and there would be a serious search and/or examination burden if restriction were not required because one or more of the following reasons apply:
a) The inventions have acquired a separate status in the art in view of their different classification for example, the invention I require the searching for an activation sparsity removal (ASR) block coupled to the AMM; a redundancy removal (RR) block coupled to the AMM, while the search for Invention II would not.
b) The inventions require a different field of search (for example, searching different employing different search queries, the invention II requires the searching for a weight lookup table (WLUT) memory; a plurality of multiplier accelerators (MAs), an accumulator coupled to an output of an activation (A) memory, while the search for Invention I would not.
Subsequent to a telephone conversation with Attorney for Applicant, Paul Johnson, Registration No.58539, on 03/27/2026, a provisional election was made without traverse to prosecute the invention of group I, claims 1-13. Affirmation of this election must be made by applicant in replying to this Office action. Claims 14-15 are withdrawn from further consideration by the examiner, 37 CFR l.142(b), as being drawn to a non-elected invention.
Applicant is reminded that upon the cancellation of claims to a non-elected invention, the inventorship must be corrected in compliance with 37 CFR 1.48(a) if one or more of the currently named inventors is no longer an inventor of at least one claim remaining in the application. A request to correct inventorship under 37 CFR 1.48(a) must be accompanied by an application data sheet in accordance with 37 CFR 1.76 that identifies each inventor by his or her legal name and by the processing fee required under 37 CFR l.17(i).
DETAILED ACTION
This office action is in response to the claims filed on 06/02/2023.
Claims 1-13 are presented for examination.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-13 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 is rejected on the basis that it contains an improper Markush grouping of alternatives. See In re Harnisch, 631 F.2d 716, 721-22 (CCPA 1980) and Ex parte Hozumi, 3 USPQ2d 1059, 1060 (Bd. Pat. App. & Int. 1984). A Markush grouping is proper if the alternatives defined by the Markush group (i.e., alternatives from which a selection is to be made in the context of a combination or process, or alternative chemical compounds as a whole) share a “single structural similarity” and a common use. A Markush grouping meets these requirements in two situations. First, a Markush grouping is proper if the alternatives are all members of the same recognized physical or chemical class or the same art-recognized class, and are disclosed in the specification or known in the art to be functionally equivalent and have a common use. Second, where a Markush grouping describes alternative chemical compounds, whether by words or chemical formulas, and the alternatives do not belong to a recognized class as set forth above, the members of the Markush grouping may be considered to share a “single structural similarity” and common use where the alternatives share both a substantial structural feature and a common use that flows from the substantial structural feature. See MPEP § 2117.
The Markush grouping of “each NPU including: one of: an activation sparsity removal (ASR) block coupled to the AMM; a redundancy removal (RR) block coupled to the AMM; or both an ASR block coupled to the AMM and an RR block coupled to an output of the ASR block; a multiply accumulator (MAC) block coupled to the output of the ASR block or an output of the RR block; and a non-linear unit coupled to an output of the MAC block.” is improper because the alternatives defined by the Markush grouping do not share a common use for the following reasons: the members of the alternatives are not functionally equivalent and do not have a common use. See at least paragraph [Par.0033, 0045], disclosed the “multiply accumulator (MAC) block” and “ a non-linear unit “, [0050] which discloses the Activation Sparsity Removal “Each Activation Sparsity Removal block may implement a non-zero Activation jump algorithm similar or identical to the RNA (one bitstream of the DNA) algorithm/architecture as described herein”, [Par.0051] disclosed the “a redundancy removal (RR) block”, that execute different functions. Therefore, the members of the alternatives are substitutable, one for the other, with the expectation that the same intended result would be achieved.
To overcome this rejection, Applicant may set forth each alternative (or grouping of patentably indistinct alternatives) within an improper Markush grouping in a series of independent or dependent claims and/or present convincing arguments that the group members recited in the alternative within a single claim in fact share a single structural similarity as well as a common use.
The claim 2-13 are dependent of the claim 1 and are likewise rejected as that contains an improper Markush grouping of alternatives.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35
U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1,2, 5, 6, 7 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by LAMB et al. (Pub. No. US 20190087713– hereinafter, LAMB).
Regarding claim 1, Lamb teaches a scalable deep neural networks (DNN) accelerator (sDNA) (LAMB, [Par.0009-0010], “As described herein, a neural network may consume resources of a device on which the neural network is implemented. As neural networks increase in complexity and capability, neural networks may begin to exceed the available resources of some devices. That is, some neural networks may consume an amount of data, processor capacity, and/or power that may typically exceed on-chip and/or power supply resources of some devices. ... [0010] A weight matrix may include a plurality of weights that are input to artificial neurons or nodes in order to influence the output (e.g., activation) of those nodes. A row of a weight matrix may be a weight vector, which may include weights for a layer of the neural network. In some aspects, one or more weights may be eliminated (e.g., replaced with a zero) in order to reduce overhead (e.g., processing power) without appreciably degrading the accuracy of the neural network. For example, weights of a layer of an exemplary DCN may be thirty to seventy percent “sparse”—e.g., such that thirty to seventy percent of the weights of a weight vector for the exemplary layer are replaced with zeros.” Examiner’ s note, reducing the power of the operation of the neural network by replace the sparse weight the zero do reduce the power processing, that is considered as the scalable deep neural networks (DNN) accelerator (sDNA)).
comprising:a plurality of address generators (LAMB, [Par.0083-0084], “0083] Now with reference to FIG. 5, a memory 500 is illustrated, which may be a portion of memory 118 of FIG. 1... The memory 500 may include a plurality of addresses, including addresses 582a-d. According to an example, a compressed set of sparse weight vectors (e.g., the compressed set of sparse weight vectors 480) may be linearly mapped into memory (e.g., memory 118). In this example, the memory 500 is illustrated as including a portion of the compressed set of sparse weight vectors 480. [0084] As indicated with respect to FIG. 4E, one or more inserted weights 468 may be added, in addition to the weights of the compressed set of sparse weight vectors. For example, beginning at the first address 582a, a first weight of a first row and first column of the compressed set of sparse weight vectors may be mapped to the memory 500. Next, a second weight of a second row and the first column of the compressed set of sparse weight vectors may be mapped to memory 500, and so forth.” Examiner’s note, the first weight of a first row and first column of the compressed set of sparse weight vectors (output ) at the first address of the plurality addresses, that is mapped to the memory.);
an activation memory matrix (AMM) coupled to outputs of the plurality of address generators (LAMB, [Par.0083-0084], “0083] Now with reference to FIG. 5, a memory 500 is illustrated, which may be a portion of memory 118 of FIG. 1... The memory 500 may include a plurality of addresses, including addresses 582a-d. According to an example, a compressed set of sparse weight vectors (e.g., the compressed set of sparse weight vectors 480) may be linearly mapped into memory (e.g., memory 118). In this example, the memory 500 is illustrated as including a portion of the compressed set of sparse weight vectors 480. [0084] As indicated with respect to FIG. 4E, one or more inserted weights 468 may be added, in addition to the weights of the compressed set of sparse weight vectors. For example, beginning at the first address 582a, a first weight of a first row and first column of the compressed set of sparse weight vectors may be mapped to the memory 500. Next, a second weight of a second row and the first column of the compressed set of sparse weight vectors may be mapped to memory 500, and so forth.” Examiner’s note, the first weight of a first row and first column of the compressed set of sparse weight vectors (output ) at the first addresser of the plurality addressers, that is mapped to the memory., therefore, the first weight is considered as the output of the addresser. The process of the output of the sparse weight compression from the current layer is inputted into the next layer of the neural network that is corresponding to the activation memory matrix. Therefore, the activation memory matrix coupled to outputs of the plurality of address generators., as ti can be seen at [Par.0112-0112], “At operation 806, a neural network may be operated based on the compressed set of sparse weight vectors. …7, the controller 720 may obtain an input for a neural network. The input may be processed through the neural network such that the controller 720 provides an input vector to the inputs 702a-b. At least one activation of an input vector of the set of the input vectors may be selected based on the activation selection value 714, and the selected activation may be aligned with corresponding weights of the compressed set of sparse weight vectors 716. A set of partial sums may be accumulated based on the selected activation aligned with the corresponding weights of the compressed set of sparse weight vectors 716. The set of partial sums may be provided to the PSH 740, which may calculate activations for one or more nodes the neural network (e.g., nodes of a current layer of the neural network) and provide those activations for one or more other nodes of the neural network (e.g., nodes of a next layer of the neural network). The neural network may provide an output (e.g., at an output layer of the neural network),” )
a plurality of network processing units (NPUs) coupled to outputs of the AMM (LAMB, [Par.0031], “Variables (e.g., neural signals and synaptic weights), system parameters associated with a computational device (e.g., neural network with weights), delays, frequency bin information, and task information may be stored in a memory block associated with one or more Neural Processing Units (NPUs) 108” Examiner’s note, the NPU associated/coupled with the neural network weight, wherein, the neural network weight is considered as the output of the AMM, since the neural operation is processing by inputting an output weight from one layer into the next layer. ).
each NPU including: one of: an activation sparsity removal (ASR) block coupled to the AMM; a redundancy removal (RR) block coupled to the AMM; or both an ASR block coupled to the AMM and an RR block coupled to an output of the ASR block; a multiply accumulator (MAC) block coupled to the output of the ASR block or an output of the RR block; and a non-linear unit coupled to an output of the MAC block (LAMB , [par.0012], “In an aspect of the disclosure, a first method, a first computer-readable medium, and a first apparatus for operating a neural network are provided. For example, the first apparatus for operating a neural network may be configured to receive a set of sparse weight vectors, and at least a first sparse weight vector of the set of sparse weight vectors includes at least one zero weight element and at least one non-zero weight element. The apparatus may be configured to compress the set of sparse weight vectors to produce a compressed set of sparse weight vectors by removing one or more of the at least one zero weight element of at least the first sparse weight vector of the set of sparse weight vectors and combining at least the first sparse weight vector with at least a second sparse weight vector of the set of sparse weight vectors. The apparatus may be configured to operate the neural network based on the compressed set of sparse weight vectors.”, “[Par.0031], “Variables (e.g., neural signals and synaptic weights), system parameters associated with a computational device (e.g., neural network with weights), delays, frequency bin information, and task information may be stored in a memory block associated with one or more Neural Processing Units (NPUs) 108” And{par.0063-0064], “] As shown in FIG. 4B, a set of sparse weight vectors 420 is illustrated. In an aspect, a plurality of weights may be eliminated or removed, so that one or more of the set of sparse weight vectors 420 includes zero-weight elements 422a-b (e.g., the zero-weight elements 422a-b may be values set to zero). With the zero-weight elements 422a-b, the set of sparse weight vectors 420 may include eighty-six weights (e.g., thirty-three percent sparsity with sixty-seven percent occupancy)…[0064] In introducing the zero-weight elements 422a-b, the number of MAC operations may be correspondingly reduced (e.g., because no MAC operations need to be performed for zero-weight elements 422a-b). For example, one weight vector 426a of the set of sparse weight vectors 420 may include five non-zero weights (e.g., for activations 406a-b, 406f-h) and, therefore, five MAC operations may be performed (instead of eight MAC operations for a respective one of the row 402 of the set of weight vectors 400). Therefore, instead of consuming sixteen clock cycles for 128 calculations (e.g., for eight MAC operations per clock cycle), MAC operations for the set of sparse weight vectors 420 may be completed in eleven clock cycles (e.g., eighty-six weights of the set of sparse weight vectors 420 divided by eight MAC operations per clock cycle equals 10.75, adding one “dummy” operation to consume the eleven total clock cycles).” Examiner’s note, zero-weight elements 422a-b (e.g., the zero-weight elements 422a-b may be values set to zero) of the set of sparse weight vectors 420 maybe removed in the neural network operation, therefore, the set of sparse weight vectors 420 block is considered as the activation sparsity removal (ASR) block. NPU associated with the neural network weight, wherein, the weight includes the sparse weight, therefore, the NPU including: one of: an activation sparsity removal (ASR) block coupled to the AMM.).
Regarding claim 2, LAMB teaches the sDNA of claim 1, wherein the AMM comprises a plurality of activation memories (LAMB, [0087-0088], “…Each of the activations 602a-c may be a respective activation sample to be provided to each of the first banks 612a and each of the second banks 612b of each of the MAC elements 610a-h. Accordingly, each of the activations 602a-c may be provided (e.g., broadcast) to each of the MAC elements 610a-h for each weight of a compressed set of sparse weight vectors. Each of the MAC elements 610a-h may be replicated for each pairing of one of the activations 602a-c with a weight. [0088] For each of the activations 602a-c, the MAC elements 610a-h may compute a respective one of the outputs 604a-c, which may include partial sums that may be accumulated over the activations 602a-c. Each of the outputs 604a-c may correspond to a compressed set of sparse weight vectors (e.g., the compressed set of sparse weight vectors 480).” And [Par.0112-0112], “At operation 806, a neural network may be operated based on the compressed set of sparse weight vectors. …7, the controller 720 may obtain an input for a neural network. The input may be processed through the neural network such that the controller 720 provides an input vector to the inputs 702a-b. At least one activation of an input vector of the set of the input vectors may be selected based on the activation selection value 714, and the selected activation may be aligned with corresponding weights of the compressed set of sparse weight vectors 716. A set of partial sums may be accumulated based on the selected activation aligned with the corresponding weights of the compressed set of sparse weight vectors 716. The set of partial sums may be provided to the PSH 740, which may calculate activations for one or more nodes the neural network (e.g., nodes of a current layer of the neural network) and provide those activations for one or more other nodes of the neural network (e.g., nodes of a next layer of the neural network). The neural network may provide an output (e.g., at an output layer of the neural network),” Examiner’s note, the process of the output of the sparse weight compression from the current layer is inputted into the next layer of the neural network that is corresponding to the activation memory matrix. each of the Mac element is considered as the activation memory since the activations is provided to each of the first banks 612a and each of the second banks 612b of each of the MAC elements.),
each activation memory coupled to a different address generator of the plurality of address generators (LAMB, [Par.0084-0085], “As indicated with respect to FIG. 4E, one or more inserted weights 468 may be added, in addition to the weights of the compressed set of sparse weight vectors. For example, beginning at the first address 582a, a first weight of a first row and first column of the compressed set of sparse weight vectors may be mapped to the memory 500. Next, a second weight of a second row and the first column of the compressed set of sparse weight vectors may be mapped to memory 500, and so forth. After each weight of the first column is mapped to memory 500, a weight of the first row and second column may be mapped to memory 500. Accordingly, each weight of the compressed set of sparse weight vectors may be mapped to memory 500. The mapping of the weights to memory 500 may be performed at compile time when generating a static DDR image for the neural network.[0085] While the mapping of weights to memory 500 may interleave weights associated with activations 406a-h, any address (e.g., one of the addresses 582a-d) may be either one of two possible activations 406a-h, as the result of the address modulo the number of MAC elements (e.g., eight MAC elements 464a-h). Accordingly, each MAC element may select one between two of the activations 406a-h, for example, based on an activation selection bit(s). In some aspects, mapping may reorder the series of weights according to an activation 406a-h (e.g., according to filter number), and further interleave two series corresponding to sequential activations 406a-h.” Examiner’s note, each activation memory is associate with the particular address of the plurality addresses, since the weight associates with one or two activations of each of the MAC element.).
Regarding claim 5, Lamb teaches the sDNA of claim 1, wherein each of the ASR blocks: implements a non-zero Activation jump algorithm; uses multiple first in first out (FIFO) memories to store non-zero activations read from the AMM; and/or uses an adder tree (LAMB , [par.0012], “In an aspect of the disclosure, a first method, a first computer-readable medium, and a first apparatus for operating a neural network are provided. For example, the first apparatus for operating a neural network may be configured to receive a set of sparse weight vectors, and at least a first sparse weight vector of the set of sparse weight vectors includes at least one zero weight element and at least one non-zero weight element. The apparatus may be configured to compress the set of sparse weight vectors to produce a compressed set of sparse weight vectors by removing one or more of the at least one zero weight element of at least the first sparse weight vector of the set of sparse weight vectors and combining at least the first sparse weight vector with at least a second sparse weight vector of the set of sparse weight vectors. The apparatus may be configured to operate the neural network based on the compressed set of sparse weight vectors.” And [0064] In introducing the zero-weight elements 422a-b, the number of MAC operations may be correspondingly reduced (e.g., because no MAC operations need to be performed for zero-weight elements 422a-b). For example, one weight vector 426a of the set of sparse weight vectors 420 may include five non-zero weights (e.g., for activations 406a-b, 406f-h) and, therefore, five MAC operations may be performed (instead of eight MAC operations for a respective one of the row 402 of the set of weight vectors 400). Therefore, instead of consuming sixteen clock cycles for 128 calculations (e.g., for eight MAC operations per clock cycle), MAC operations for the set of sparse weight vectors 420 may be completed in eleven clock cycles (e.g., eighty-six weights of the set of sparse weight vectors 420 divided by eight MAC operations per clock cycle equals 10.75, adding one “dummy” operation to consume the eleven total clock cycles).”Examiner’s note, the ASR blocks include the zero weight are removed and the MAC only performs the non-zero weight block, therefore , each of the sparse weight vectors 420 (activation sparsity removal block) implements the non-zero Activation jump algorithm.).
Regarding claim 6, LAMB teaches the sDNA of claim 1, wherein each of the MAC blocks is configured to implement machine learning tensor multiplications (Lamb, [Par.0057-0059], “] FIGS. 4A-E are diagrams illustrating weights to be implemented by a neural network. FIG. 4A illustrates a set of weight vectors 400, including sixteen weight vectors for eight activations 406a-h. Each of the rows 402 of the set of weight vectors 400 may correspond to a different weight vector (e.g., sixteen eight-element weight vectors). For example, in the context of a CNN, a weight vector may be an output filter, which may be convolved with an input (e.g., an eight-channel input for the eight activations 406a-h). Each of the columns 404 of the set of weight vectors 400 may correspond to each of the set of activations 406a-h that may be needed to perform a set of calculations, e.g., for a channel i. An activation may be a value that is output from a node of one layer (e.g., a prior layer) that serves as input for a node of another layer (e.g., a next layer) of the neural network. Each of the numerical values in each of the set of weight vectors 400 may correspond to a different weight (e.g., at least one weight 1 for one filter and a first activation 406a, at least one weight 2 for a second filter and a first activation 406a, . . . , at least one weight 16 for a sixteenth filter and a first activation 406a, etc.) (n.b., a value of each at least one weight may not be the illustrated numerical value)… According to various aspects, one or more MAC operations may consume a clock cycle of a system architecture (e.g., clock cycles of the SOC 100). For example, a system architecture may support eight MAC operations per clock cycle. Therefore, 128 calculations (e.g., sixteen weight vectors, each having eight elements, multiplied with an eight-channel input) may be performed in sixteen clock cycles (e.g., 128 calculations divided by eight MAC operations per clock cycle).” Examiner’s note, 128 calculation corresponds to sixteen weight vectors, each having eight elements, multiplied with an eight-channel input is performed in sixteen clock cycles, each cycle includes 8MAC, wherein, each cycle is performed by the neural network.).
Regarding claim 7, LAMB teaches the sDNA of claim 1, wherein the non-linear unit implements a non-linear function (Lamb, [Par.0051], “The values in the feature map may be further processed with a non-linearity, such as a rectification, max(0,x). Values from adjacent neurons may be further pooled, which corresponds to down sampling, and may provide additional local invariance and dimensionality reduction. Normalization, which corresponds to whitening, may also be applied through lateral inhibition between neurons in the feature map.”). .
Regarding claim 8, LAMB teaches the sDNA of claim 1, wherein the non-linear unit comprises a rectified linear unit (ReLU), (Lamb, [Par.0056], “A neural network may include a plurality of layers, and each layer may include a set of nodes (also known as “units” or “artificial neurons”). Each node may receive inputs. A first input may be provided as input to the neural network (e.g., at an input layer of node(s)) or as an output of a node from another layer (e.g., at hidden layer(s) of node(s)). A first input may be associated with a weight element (or “weight”), which may introduce a bias associated with the first input (e.g., based on relative importance of the first input). Some nodes may receive a weight (e.g., a “bias”) that is associated with the node, e.g., in order to affect output or activation of the node. Each node may apply an activation function (e.g., a sigmoid function, a rectified linear unit (ReLU) function, etc.) to the inputs in order to generate an output or activation. The output or activation may be provided at an output layer of the neural network or may be provided as an input to another node. Accordingly, the weights may be stored in memory, such as in a set of weight vectors.”.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 3, 4, 9-13 are rejected under 35 U.S.C. 103 as being unpatentable over LAMB et al. (Pub. No. US 20190087713– hereinafter, LAMB) in view of AFZAI et al. (Patent. No 11537853-hereinafter, AFZAI).
Regarding claim 3, LAMB teaches the sDNN of the claim 2, but it does not teach wherein each of the plurality of NPUs is coupled to each of the plurality of activation memories,
On the other hand, AFZAI teaches wherein each of the plurality of NPUs is coupled to each of the plurality of activation memories (Afzai, [Col. 7, lines 24-46], “The activation buffer access unit 120 is configured to access one or more activation buffers in the local memory buffers 140. Similarly, the weight buffer access unit 122 and the output buffer access unit 130 are configured to access one or more weight buffers and one or more output buffers, respectively. The activations stored in the activation buffer(s) correspond to activations produced by one or more layers of a neural network being executed on the NNA 100. The weights stored in the weight buffer(s) are synaptic weights associated with edges between a node of one layer and a node of another layer. Activation and weights are used for certain computations, including for instructions executed by the compute engine 116. The output buffers can store final results or intermediate results (e.g., partial sums) for access by the host processor or the system memory. The NPUs 124, 126, and 128 perform numerical operations using the activations and weights stored in the local memory buffers 140. Each NPU is configured to perform all or part of a compute instruction. Although FIG. 1 depicts the NPUs 124, 126, and 128 as block components, the NPUs 124, 126, and 128 are not necessarily identical. For example, as described in connection with FIG. 2, the operations of one NPU may differ from the operations performed by another NPU.”).
LAMB and AFZAI are analogous in arts because they have the same field of endeavor of generating the neural network component.
Accordingly, it would have been obvious to one of the ordinary skills in the art before the effective filing date of the claimed invention to have modified the sDNN, as taught by LAMB, to include the wherein each of the plurality of NPUs is coupled to each of the plurality of activation memories, as taught by AFZAI. The modification would have been obvious because one of the ordinary skills in art would be motivated to store the activations produced by one or more layer of a neural network being executed on the NNA, (AFZAI, [Col. 7, lines 24-38], “The activation buffer access unit 120 is configured to access one or more activation buffers in the local memory buffers 140. Similarly, the weight buffer access unit 122 and the output buffer access unit 130 are configured to access one or more weight buffers and one or more output buffers, respectively. The activations stored in the activation buffer(s) correspond to activations produced by one or more layers of a neural network being executed on the NNA 100. The weights stored in the weight buffer(s) are synaptic weights associated with edges between a node of one layer and a node of another layer. Activation and weights are used for certain computations, including for instructions executed by the compute engine 116. The output buffers can store final results or intermediate results (e.g., partial sums) for access by the host processor or the system memory.”).
Regarding the claim 4, LAMB teaches the sDNA of claim 1, but it does not teach wherein the AMM supports at least one of: a multiple points (pixels) parallel scheme, a lines parallel scheme, a multiple input channels parallel scheme, or a multiple output channels parallel scheme,
On the hand, AFZAI teaches wherein the AMM supports at least one of: a multiple points (pixels) parallel scheme, a lines parallel scheme, a multiple input channels parallel scheme, or a multiple output channels parallel scheme (Afzai, [Col.7, lines 16-47], “The processor 114 is an optional general purpose processor for performing certain types of processing in parallel with processing performed by the NPUs 124, 126, and 128. For example, processor 114 may include a floating point unit or other arithmetic logic unit for performing general arithmetic operations in parallel with matrix operations performed by the NPUs 124, 126, and 128. The activation buffer access unit 120 is configured to access one or more activation buffers in the local memory buffers 140. Similarly, the weight buffer access unit 122 and the output buffer access unit 130 are configured to access one or more weight buffers and one or more output buffers, respectively. The activations stored in the activation buffer(s) correspond to activations produced by one or more layers of a neural network being executed on the NNA 100. The weights stored in the weight buffer(s) are synaptic weights associated with edges between a node of one layer and a node of another layer. Activation and weights are used for certain computations, including for instructions executed by the compute engine 116. The output buffers can store final results or intermediate results (e.g., partial sums) for access by the host processor or the system memory. The NPUs 124, 126, and 128 perform numerical operations using the activations and weights stored in the local memory buffers 140. Each NPU is configured to perform all or part of a compute instruction. Although FIG. 1 depicts the NPUs 124, 126, and 128 as block components, the NPUs 124, 126, and 128 are not necessarily identical. For example, as described in connection with FIG. 2, the operations of one NPU may differ from the operations performed by another NPU.”),
LAMB and AFZAI are analogous in arts because they have the same field of endeavor of generating the neural network component.
Accordingly, it would have been obvious to one of the ordinary skills in the art before the effective filing date of the claimed invention to have modified the sDNN, as taught by LAMB, to include the wherein the AMM supports at least one of: a multiple points (pixels) parallel scheme, a lines parallel scheme, a multiple input channels parallel scheme, or a multiple output channels parallel scheme, as taught by AFZAI.
The modification would have been obvious because one of the ordinary skills in art would be motivated to process the parallel with processing performed by the plurality NPUs, AFZAI, [Col.7, lines 16-47], “The processor 114 is an optional general purpose processor for performing certain types of processing in parallel with processing performed by the NPUs 124, 126, and 128. For example, processor 114 may include a floating point unit or other arithmetic logic unit for performing general arithmetic operations in parallel with matrix operations performed by the NPUs 124, 126, and 128. The activation buffer access unit 120 is configured to access one or more activation buffers in the local memory buffers 140. Similarly, the weight buffer access unit 122 and the output buffer access unit 130 are configured to access one or more weight buffers and one or more output buffers, respectively.).
Regarding claim 9, LAMB teaches the sDNA of claim 1, but it does not teach wherein each of the NPUs comprises a sequential execution NPU, a concurrent execution NPU, or a combination of sequential execution NPU and concurrent execution NPU.
On the other hand, AFZAI teaches, wherein each of the NPUs comprises a sequential execution NPU, a concurrent execution NPU, or a combination of sequential execution NPU and concurrent execution NPU (Afzal, [Col.7, lines 17-23], “The processor 114 is an optional general purpose processor for performing certain types of processing in parallel with processing performed by the NPUs 124, 126, and 128. For example, processor 114 may include a floating point unit or other arithmetic logic unit for performing general arithmetic operations in parallel with matrix operations performed by the NPUs 124, 126, and 128.” Examiner’s note, the plurality of NPU(s) are performed in parallel that corresponds to the concurrent execution NPU.).
LAMB and AFZAI are analogous in arts because they have the same field of endeavor of generating the neural network component.
Accordingly, it would have been obvious to one of the ordinary skills in the art before the effective filing date of the claimed invention to have modified the sDNN, as taught by LAMB, to include the w wherein each of the NPUs comprises a sequential execution NPU, a concurrent execution NPU, or a combination of sequential execution NPU and concurrent execution NPU, as taught by AFZAI. The modification would have been obvious because one of the ordinary skills in art would be motivated to process the parallel with processing performed by the plurality NPUs, (AFZAI, [Col.7, lines 16-47], “The processor 114 is an optional general purpose processor for performing certain types of processing in parallel with processing performed by the NPUs 124, 126, and 128. For example, processor 114 may include a floating point unit or other arithmetic logic unit for performing general arithmetic operations in parallel with matrix operations performed by the NPUs 124, 126, and 128. The activation buffer access unit 120 is configured to access one or more activation buffers in the local memory buffers 140. Similarly, the weight buffer access unit 122 and the output buffer access unit 130 are configured to access one or more weight buffers and one or more output buffers, respectively.).
Regarding claim 10, Lamb teaches the sDNA of claim 9, wherein: each sequential execution NPU is configured to store back (feedback) an output of each neural network layer to a current AMM layer (Lamb, [Par.0031], “Variables (e.g., neural signals and synaptic weights), system parameters associated with a computational device (e.g., neural network with weights), delays, frequency bin information, and task information may be stored in a memory block associated with one or more Neural Processing Units (NPUs) 108” and [par.0039], “Neural networks may be designed with a variety of connectivity patterns. In feed-forward networks, information is passed from lower to higher layers, with each neuron in a given layer communicating to neurons in higher layers. A hierarchical representation may be built up in successive layers of a feed-forward network, as described above. Neural networks may also have recurrent or feedback (also called top-down) connections. In a recurrent connection, the output from a neuron in a given layer may be communicated to another neuron in the same layer. A recurrent architecture may be helpful in recognizing patterns that span more than one of the input data chunks that are delivered to the neural network in a sequence. A connection from a neuron in a given layer to a neuron in a lower layer is called a feedback (or top-down) connection. A network with many feedback connections may be helpful when the recognition of a high-level concept may aid in discriminating the particular low-level features of an input”);
However, LAMB does not teach and each concurrent execution NPU is configured to allocate different hardware resources to different DNN layers to process the DNN layers in parallel (concurrently),
On the other hand, AFZAI teaches and each concurrent execution NPU is configured to allocate different hardware resources to different DNN layers to process the DNN layers in parallel (concurrently) (Afzal, [Col.6, lines 64-67 and Col.7, lines 1-23], “…. For example, the DME 150 could receive and execute a data move instruction, but the NPUs 124, 126, and 128 could ignore the data move instruction. Because instructions can execute concurrently in different components, it is useful to have a synchronization mechanism to handle any dependencies between instructions. The predicate register can be used to implement such a synchronization mechanism and, in certain embodiments, is a global register visible to internal components of the NNA 100, as well as visible to external entities such as the host processor. Synchronization also helps to prevent conflicts in accessing the local memory buffers 140. The processor 114 is an optional general purpose processor for performing certain types of processing in parallel with processing performed by the NPUs 124, 126, and 128. For example, processor 114 may include a floating point unit or other arithmetic logic unit for performing general arithmetic operations in parallel with matrix operations performed by the NPUs 124, 126, and 128.),
LAMB and AFZAI are analogous in arts because they have the same field of endeavor of generating the neural network component.
Accordingly, it would have been obvious to one of the ordinary skills in the art before the effective filing date of the claimed invention to have modified the sDNN, wherein each sequential execution NPU is configured to store back (feedback) an output of each neural network layer to a current AMM layer as taught by LAMB, to include the and each concurrent execution NPU is configured to allocate different hardware resources to different DNN layers to process the DNN layers in parallel (concurrently), as taught by AFZAI. The modification would have been obvious because one of the ordinary skills in art would be motivated to process the parallel with processing performed by the plurality NPUs, (AFZAI, [Col.7, lines 16-47], “The processor 114 is an optional general purpose processor for performing certain types of processing in parallel with processing performed by the NPUs 124, 126, and 128. For example, processor 114 may include a floating point unit or other arithmetic logic unit for performing general arithmetic operations in parallel with matrix operations performed by the NPUs 124, 126, and 128. The activation buffer access unit 120 is configured to access one or more activation buffers in the local memory buffers 140. Similarly, the weight buffer access unit 122 and the output buffer access unit 130 are configured to access one or more weight buffers and one or more output buffers, respectively.).
Regarding claim 11, LAMB teaches the sDNA of claim 9, wherein:each sequential execution NPU is configured to reuse hardware resources to calculate different layers of the same neural network (Lamb, [Par.0031], “Variables (e.g., neural signals and synaptic weights), system parameters associated with a computational device (e.g., neural network with weights), delays, frequency bin information, and task information may be stored in a memory block associated with one or more Neural Processing Units (NPUs) 108” and [par.0039], “Neural networks may be designed with a variety of connectivity patterns. In feed-forward networks, information is passed from lower to higher layers, with each neuron in a given layer communicating to neurons in higher layers. A hierarchical representation may be built up in successive layers of a feed-forward network, as described above. Neural networks may also have recurrent or feedback (also called top-down) connections. In a recurrent connection, the output from a neuron in a given layer may be communicated to another neuron in the same layer. A recurrent architecture may be helpful in recognizing patterns that span more than one of the input data chunks that are delivered to the neural network in a sequence. A connection from a neuron in a given layer to a neuron in a lower layer is called a feedback (or top-down) connection. A network with many feedback connections may be helpful when the recognition of a high-level concept may aid in discriminating the particular low-level features of an input”);
However, LAMB does not teach each concurrent execution NPU provides results of each DNN layer to another hardware logic that executes a next DNN layer,
On the other hand, AFZAI teaches and each concurrent execution NPU provides results of each DNN layer to another hardware logic that executes a next DNN layer, (Afzal, [Col.6, lines 64-67 and Col.7, lines 1-23], “…. For example, the DME 150 could receive and execute a data move instruction, but the NPUs 124, 126, and 128 could ignore the data move instruction. Because instructions can execute concurrently in different components, it is useful to have a synchronization mechanism to handle any dependencies between instructions. The predicate register can be used to implement such a synchronization mechanism and, in certain embodiments, is a global register visible to internal components of the NNA 100, as well as visible to external entities such as the host processor. Synchronization also helps to prevent conflicts in accessing the local memory buffers 140. The processor 114 is an optional general purpose processor for performing certain types of processing in parallel with processing performed by the NPUs 124, 126, and 128. For example, processor 114 may include a floating point unit or other arithmetic logic unit for performing general arithmetic operations in parallel with matrix operations performed by the NPUs 124, 126, and 128.),
LAMB and AFZAI are analogous in arts because they have the same field of endeavor of generating the neural network component.
Accordingly, it would have been obvious to one of the ordinary skills in the art before the effective filing date of the claimed invention to have modified the sDNN, wherein each sequential execution NPU is configured to store back (feedback) an output of each neural network layer to a current AMM layer as taught by LAMB, to include the and each concurrent execution NPU provides results of each DNN layer to another hardware logic that executes a next DNN layer, as taught by AFZAI. The modification would have been obvious because one of the ordinary skills in art would be motivated to process the parallel with processing performed by the plurality NPUs, (AFZAI, [Col.7, lines 16-47], “The processor 114 is an optional general purpose processor for performing certain types of processing in parallel with processing performed by the NPUs 124, 126, and 128. For example, processor 114 may include a floating point unit or other arithmetic logic unit for performing general arithmetic operations in parallel with matrix operations performed by the NPUs 124, 126, and 128. The activation buffer access unit 120 is configured to access one or more activation buffers in the local memory buffers 140. Similarly, the weight buffer access unit 122 and the output buffer access unit 130 are configured to access one or more weight buffers and one or more output buffers, respectively.).
Regarding claim 12, LAMB teaches the sDNA of claim 1, but it does not teach wherein the sDNA supports different size convolution operations,
On the other hand, AFZAI teaches wherein the sDNA supports different size convolution operations (AFZAI, [col 9, lines 36-61], “FIG. 2 shows an example datapath 200 for operations performed within a compute engine, e.g., the compute engine 116. As shown in FIG. 2, a plurality of NPUs 202, 204, and 206 are communicatively coupled to an activation memory 210 and a weight memory 220. The NPUs 202, 204, and 206 are coupled together to form a processing pipeline, and can correspond to the NPUs 124, 126, and 128, respectively. The activation memory 210 and the weight memory 220 may correspond to the activation buffers and the weight buffers in the local memory buffers 140, respectively. In the example of FIG. 2, the processing performed by each NPU 202, 204, and 206 involves M dot product lanes 225 of N inputs each. Two dot product lanes 225-A and 225-N are shown. In FIG. 2, the activations are 8-bit data values, N activations in total being read out of the activation memory 210 to all M dot product lanes 225. Similarly, the weights are 8-bit data values, N weights per dot product lane 225, for a total of M×N×8 bits of weight data that are read out of the weight memory 220 into a multiplexer 226 for distribution to the dot product lanes 225, with a different set of N weights being supplied to each dot product lane 225. However, in other implementations, the activations and the weights could be represented using a different number of bits. Further, the number of bits used to represent an activation are not necessarily always equal to the number of bits used to represent a weight.” Examiner’s note, the neural network is generating with the different set of N weights and different bit of data ).
LAMB and AFZAI are analogous in arts because they have the same field of endeavor of generating the neural network component.
Accordingly, it would have been obvious to one of the ordinary skills in the art before the effective filing date of the claimed invention to have modified the sDNN, as taught by LAMB, to include the wherein the sDNA supports different size convolution operations, as taught by AFZAI. The modification would have been obvious because one of the ordinary skills in art would be motivated to perform the different set of N weights and different bit of data, ((AFZAI, [col 9, lines 36-61], “FIG. 2 shows an example datapath 200 for operations performed within a compute engine, e.g., the compute engine 116. As shown in FIG. 2, a plurality of NPUs 202, 204, and 206 are communicatively coupled to an activation memory 210 and a weight memory 220. The NPUs 202, 204, and 206 are coupled together to form a processing pipeline, and can correspond to the NPUs 124, 126, and 128, respectively. The activation memory 210 and the weight memory 220 may correspond to the activation buffers and the weight buffers in the local memory buffers 140, respectively. In the example of FIG. 2, the processing performed by each NPU 202, 204, and 206 involves M dot product lanes 225 of N inputs each. Two dot product lanes 225-A and 225-N are shown. In FIG. 2, the activations are 8-bit data values, N activations in total being read out of the activation memory 210 to all M dot product lanes 225. Similarly, the weights are 8-bit data values, N weights per dot product lane 225, for a total of M×N×8 bits of weight data that are read out of the weight memory 220 into a multiplexer 226 for distribution to the dot product lanes 225, with a different set of N weights being supplied to each dot product lane 225. However, in other implementations, the activations and the weights could be represented using a different number of bits. Further, the number of bits used to represent an activation are not necessarily always equal to the number of bits used to represent a weight.”).
Regarding claim 13, LAMB teaches the sDNA of claim 12, but it does not teach wherein the different size convolution operations include two different n*n convolution operations, and wherein n in a first of the convolution operations has a first value that is different than a second value of n in a second of the convolution operations,
On the other hand, AFZAI teaches wherein the different size convolution operations include two different n*n convolution operations, and wherein n in a first of the convolution operations has a first value that is different than a second value of n in a second of the convolution operations (AFZAI, [col 9, lines 36-61], “FIG. 2 shows an example datapath 200 for operations performed within a compute engine, e.g., the compute engine 116. As shown in FIG. 2, a plurality of NPUs 202, 204, and 206 are communicatively coupled to an activation memory 210 and a weight memory 220. The NPUs 202, 204, and 206 are coupled together to form a processing pipeline, and can correspond to the NPUs 124, 126, and 128, respectively. The activation memory 210 and the weight memory 220 may correspond to the activation buffers and the weight buffers in the local memory buffers 140, respectively. In the example of FIG. 2, the processing performed by each NPU 202, 204, and 206 involves M dot product lanes 225 of N inputs each. Two dot product lanes 225-A and 225-N are shown. In FIG. 2, the activations are 8-bit data values, N activations in total being read out of the activation memory 210 to all M dot product lanes 225. Similarly, the weights are 8-bit data values, N weights per dot product lane 225, for a total of M×N×8 bits of weight data that are read out of the weight memory 220 into a multiplexer 226 for distribution to the dot product lanes 225, with a different set of N weights being supplied to each dot product lane 225. However, in other implementations, the activations and the weights could be represented using a different number of bits. Further, the number of bits used to represent an activation are not necessarily always equal to the number of bits used to represent a weight.”). .
LAMB and AFZAI are analogous in arts because they have the same field of endeavor of generating the neural network component.
Accordingly, it would have been obvious to one of the ordinary skills in the art before the effective filing date of the claimed invention to have modified the sDNN, as taught by LAMB, to include the wherein the different size convolution operations include two different n*n convolution operations, and wherein n in a first of the convolution operations has a first value that is different than a second value of n in a second of the convolution operations, as taught by AFZAI. The modification would have been obvious because one of the ordinary skills in art would be motivated to perform the different set of N weights and different bit of data, AFZAI, [col 9, lines 36-61], “FIG. 2 shows an example datapath 200 for operations performed within a compute engine, e.g., the compute engine 116. As shown in FIG. 2, a plurality of NPUs 202, 204, and 206 are communicatively coupled to an activation memory 210 and a weight memory 220. The NPUs 202, 204, and 206 are coupled together to form a processing pipeline, and can correspond to the NPUs 124, 126, and 128, respectively. The activation memory 210 and the weight memory 220 may correspond to the activation buffers and the weight buffers in the local memory buffers 140, respectively. In the example of FIG. 2, the processing performed by each NPU 202, 204, and 206 involves M dot product lanes 225 of N inputs each. Two dot product lanes 225-A and 225-N are shown. In FIG. 2, the activations are 8-bit data values, N activations in total being read out of the activation memory 210 to all M dot product lanes 225. Similarly, the weights are 8-bit data values, N weights per dot product lane 225, for a total of M×N×8 bits of weight data that are read out of the weight memory 220 into a multiplexer 226 for distribution to the dot product lanes 225, with a different set of N weights being supplied to each dot product lane 225. However, in other implementations, the activations and the weights could be represented using a different number of bits. Further, the number of bits used to represent an activation are not necessarily always equal to the number of bits used to represent a weight.”).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to EM N TRIEU whose telephone number is (571)272-5747. The examiner can normally be reached on Mon-Fri from 9:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Omar Fernandez Rivas can be reached on (571) 272-2589. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/E.T./Examiner, Art Unit 2128
/OMAR F FERNANDEZ RIVAS/Supervisory Patent Examiner, Art Unit 2128
Prosecution Insights