DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendments
Applicant’s amendment filed 10/10/2025 has been entered. In the amendment, claims 1, 4, 14-16, 18, and 23-24 were amended, and no claims were cancelled or added. As such, claims 1-25 are pending.
The interpretations of the claim 1, 4, 5, 8, 11, 13, and 14 under U.S.C. 112(f) is maintained, as there are no amendments with respect to the “configured to” language recited in these claims.
The objections to the drawings and specification set forth in the previous office action are withdrawn in view of the amendments to the specification and drawings.
The objections to the claims set forth in the previous office action are withdrawn in view of the amendments to the claims.
The rejection of claim 18 under U.S.C. 112(b), set forth in the previous office action is withdrawn in view of the amendments to the claim.
Response to Arguments
Applicant’s remarks filed 10/10/2025 with respect to the rejection of claims 1-25 under U.S.C 101 in the previous office action have been fully considered, and in light of the amendments to independent claims 1 and 15, are persuasive.
Applicant’s remarks filed 10/10/2025 with respect to the rejections of claims 1-25 under U.S.C 102 and U.S.C 103, in the previous office action have been fully considered but are unpersuasive. Applicant asserts “The Examiners indicated in the interview that such amendments to claim 1 would help overcome the cited references” (Applicant’s Remarks p. 17). However, contrary to the applicant’s assertion, the Examiners made no such statement with respect to the 102 and 103 rejections. Rather, as indicated in the Examiner Interview Summary, the examiners indicated that “the discussed proposed amendments would require further examination and search”, (See, Examiner Interview Summary Record, mailed 10/08/2025).
Applicant’s argument that amended claim 1 is patentably distinguishable from Ovisannikov in light of the amendments of that independent claim (Applicant’s Remarks p. 17), is unpersuasive. As discussed below, despite the amendments to the claims, Ovsiannikov fully discloses all of the limitations recited in amended independent claims 1 and 16. The scope of amended dependent claims 4, 14-15, 18, 23 and 24, was not significantly changed, and as discussed below, the rejections of the dependent claims are maintained using the same references applied to these claims in the previous Non-Final Office Action mailed 7/10/2025.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are:
input register file configured to store an input operand in claim 1.
a weight register file configured to store a weight operand in claim 1.
the plurality of multipliers configured to perform multiplication operations in claim 1.
an adder assembly configured to perform accumulation operations in claim 1.
an output register file configured to store in claim 1.
an adder in the first group is configured to accumulate in claims 4 and 8.
an adder in the second group is configured to accumulate in claims 4 and 8.
the plurality of processing elements configured to generate in claim 5.
additional adder assembly configured to perform accumulation in claim 5.
first multiplier is configured to perform multiplication in claims 9 and 10.
second multiplier is configured to perform multiplication in claim 9.
the first input register file is configured to store in claim 11.
input register files of one or more other processing elements in the array are configured to store in claim 13.
weight register files of one or more other processing elements in the array are configured to store in claim 14.
weight register files is configured to bypass storing weight operand in claim 15.
another weight register file of the plurality of weight register files is configured to store in claim 15.
Regarding claim 1 and the above-noted three-prong test, the recited input register file is a generic placeholder, to store… is functional language, and there is no recitation of sufficient structure in claim 1 to perform the storing. The recited weight register file is a generic placeholder, to store… is functional language, and there is no recitation of sufficient structure in claim 1 to perform the storing. The recited plurality of multipliers is a generic placeholder, perform multiplication operations… is functional language, and there is no recitation of sufficient structure in claim 1 to perform the multiplication. The recited adder is a generic placeholder, perform accumulation operations… is functional language, and there is no recitation of sufficient structure in claim 1 to perform the accumulation. The recited output register file is a generic placeholder, to store… is functional language, and there is no recitation of sufficient structure in claim 1 to perform the storing.
Regarding claims 4 and 8 and the above-noted three-prong test, the recited adder is a generic placeholder, to accumulate… is functional language, and there is no recitation of sufficient structure in claims 4 and 8 to perform the accumulating.
Regarding claim 5 and the above-noted three-prong test, the recited plurality of processing elements is a generic placeholder, to generate… is functional language, and there is no recitation of sufficient structure in claim 5 to perform the generating. The recited additional adder assembly is a generic placeholder, perform accumulation operations… is functional language, and there is no recitation of sufficient structure in claims 5 to perform the accumulation operations.
Regarding claims 9 and 10 and the above-noted three-prong test, the recited first multiplier is a generic placeholder, perform multiplication operations… is functional language, and there is no recitation of sufficient structure in claim 9 to perform the multiplication operations. The recited second multiplier is a generic placeholder, perform multiplication operations… is functional language, and there is no recitation of sufficient structure in claims 9 and 10 to perform the multiplication operations.
Regarding claim 11 and the above-noted three-prong test, the recited first input register is a generic placeholder, to store… is functional language, and there is no recitation of sufficient structure in claim 11 to perform the storing.
Regarding claim 12 and the above-noted three-prong test, the recited third multiplier is a generic placeholder, perform multiplication operations… is functional language, and there is no recitation of sufficient structure in claim 12 to perform the multiplication operations.
Regarding claim 13 and the above-noted three-prong test, the recited input register files is a generic placeholder, to store… is functional language, and there is no recitation of sufficient structure in claim 13 to perform the storing.
Regarding claim 14 and the above-noted three-prong test, the recited weight register files is a generic placeholder, to store… is functional language, and there is no recitation of sufficient structure in claim 14 to perform the storing.
Regarding claim 15 and the above-noted three-prong test, the recited weight register files is a generic placeholder, to bypass storing / to store… is functional language, and there is no recitation of sufficient structure in claim 15 to perform the bypass storing / storing.
Because these claim limitations are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
A review of the specification shows that the corresponding structure is described in the specification for the recited input register files, weight register files, multipliers, adder assemblies, output register files, adders, and processing elements, recited in claims 1, 4, 5, 8, 11, 13, and 14. Specifically, claim 1 states “the apparatus comprising a processing element that includes…”, meaning the apparatus in claim 1 contains a processing element that includes the aforementioned components being interpreted under 112(f).
Paragraph [0184] discloses “FIG. 18 illustrates an example channel-separable elementwise add operation in a PE 1800, in accordance with various embodiments. The channel-separable elementwise add operation involves scale values. The size of the scale value is 8 bits, i.e., a byte, which is the same as the size of an input element. The PE 1800 has the same or similar components as a PE that can be used for depthwise convolution, e.g., the PE 500 or 1000”. The bit-width or byte-size of data used in the depiction of PE 1800 implies the physical storage and processing of data, which is interpreted to provide sufficient physical structure for the PE and its internal components described in the claims.
Paragraph [0225-0226] discloses the depicted apparatus as computing device 2400: “FIG. 24 is a block diagram of an example computing device 2400, in accordance with various embodiments…In some embodiments, some or all of the components included in the computing device 2400 may be attached to one or more motherboards. In some embodiments, some or all of these components are fabricated onto a single system on a chip (SoC) die… The computing device 2400 may include a processing device 2402 (e.g., one or more processing devices). As used herein, the term “processing device” or “processor” may refer to any device or portion of a device that processes electronic data from registers and/or memory to transform that electronic data into other electronic data that may be stored in registers and/or memory. The processing device 2402 may include one or more digital signal processors (DSPs), application-specific (ASICs), central processing units (CPUs), graphics processing units (GPUs), cryptoprocessors (specialized processors that execute cryptographic algorithms within hardware), server processors, or any other suitable processing devices. The computing device 2400 may include a memory 2404, which may itself include one or more memory devices such as volatile memory (e.g., DRAM), nonvolatile memory (e.g., read-only memory (ROM)), flash memory, solid state memory, and/or a hard drive. In some embodiments, the memory 2404 may include memory that shares a die with the processing device 2402”. The computing device 2400 is interpreted to encompass the components of the aforementioned claims as physical components containing circuitry and having sufficient structure. These disclosures indicate that specified hardware or software are used for the specific components and units described in the claims.
If applicant wishes to provide further explanation or dispute the examiner's interpretation of the corresponding structure, applicant must identify the corresponding structure with reference to the specification by page and line number, and to the drawing, if any, by reference characters in response to this Office action.
If applicant does not intend to have these limitations interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitations to avoid them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Objections
Claim 23 is objected to because of the following informalities:
Claim 23 recites “ Transferring” which should be changed to “transferring”.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claim 18 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 18 recites the limitation "the number of weight operands " in line 3. There is insufficient antecedent basis for this limitation in the claim. It should be change to "a".
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 2, 4-6, 13, 14, 16-18 and 23-25 are rejected under 35 U.S.C. 102(a)(1) and 102(a)(2) as being anticipated by Ovsiannikov (US 20200349420 A1; hereinafter Ovsiannikov). Ovsiannikov was published before the effective earliest filing date of this application, i.e., 04/29/2022. Therefore, Ovsiannikov constitutes as prior art under 35 U.S.C 102(a)(1) and 35 U.S.C. 102(a)(2).
Regarding Independent Claim 1, Ovsiannikov discloses An apparatus for deep learning, the apparatus comprising a processing element that includes (see, e.g., paragraphs [0005]: “According to some embodiments of the present disclosure, there is provided a processor, including: a first tile, a second tile…the first tile being configured: to receive a tensor of activations… and to perform a convolution of a kernel with one of the two-dimensional arrays" and [0153-0155]: “FIG. 1A shows a high-level block diagram of a neural processor 100, according to some embodiments. The neural processor 100…The detailed description set forth below in connection with the appended drawings is intended as a description of exemplary embodiments of a mixed-precision NPU tile with depth-wise convolution provided in accordance with the present disclosure and is not intended to represent the only forms in which the present disclosure may be constructed or utilized"):
a plurality of input register files (see, e.g., paragraph [0159]: “Inside each MR tile 102, a compact activations cache, or ‘IFM cache’ 139 stores activations delivered from SRAM 109 over IFM delivery fabric 104” [i.e., MR tiles each contain an IFM cache (input register file)]),
an input register file1 configured to store an input operand that includes a sequence of input elements from an input feature map (see, e.g., paragraph [0159]: “Inside each MR tile 102, a compact activations cache, or “IFM cache” 139 stores activations delivered from SRAM 109 over IFM delivery fabric 104…Therefore, caching activations locally in each MR tile may allow computation to proceed without the need to fetch same IFM element from SRAM repetitively” [i.e., IFM caches are input register files that store activations (input operands)] and paragraph [0162]: “Note that ABU 141 contains an IFM buffer 124 and type converter 135. While the type converter 135 performs numeric format conversions as mentioned above, the IFM buffer 124 buffers two or more activations (per lane) fetched from IFM cache 139” [i.e., multiple activations per lane (sequence of input elements) from IFM cache]),
the input feature map comprising a plurality of channels (see, e.g., paragraph [0036]: “FIG. 2A is a 3D IFM tensor with a planar size of 5 rows by 12 columns and a depth of 16 channels” and paragraph [0155]: “The neural processor 100 may be configured to efficiently calculate a convolution or a tensor product of an input feature map (IFM)”),
and each input element corresponding to a different channel of the plurality of channels (see, e.g., paragraphs [0184-185]: "Depth-wise-separable convolution, also known as depth-wise convolution, is a special case of general direct convolution where output values in depth channel C are computed using same input depth channel C, see FIG. 2E. In other words, information stays within same channel, does not cross channels… an IFM slice is an activations vector containing values for 16 depth channels all of which belong to the same planar coordinate (row, col), e.g. I.sub.row,col,0 . . . 15, where 0 . . . 15 corresponds to depth channels" and paragraph [0205]: “In general, the input feature map can be traversed in an optimal sequence… since depth-wise convolution operates on each depth channel independently, the depth-wise convolution's output depth channel is same as the depth-wise convolution's input channel, here referred to as just ‘depth-wise convolution's channel’” [i.e., per channel operations are performed on depth-wise input channels]);
a plurality of weight register files (see, e.g., paragraph [0161]: “Similar to ABU's numeric format conversion, weight decompression unit (WDU) converts signed and multi-byte weights into one or more sign-and-magnitude values as it loads weights from SRAM to weight register files 127”),
a weight register file2 configured to store a weight operand that includes a sequence of weights from a filter (see, e.g., paragraphs [0161-163]: “As used herein, a ‘weight register file’ is an array of weight registers, each weight register being a register configured to store a weight… Weight register index “seq” corresponds to the weight's sequence order in the weight kernel. For example, a 3×3 weight kernel has 9 weights, each weight associated with its respective activation”);
a plurality of multipliers (see, e.g., paragraph [0007]: “In some embodiments, the first tile includes a plurality of multipliers including the first multiplier and the second multiplier, arranged in a plurality of columns and a plurality of lanes”),
the plurality of multipliers3 configured to perform multiplication operations in parallel, each multiplier to perform a group of multiplication operations (see, e.g., paragraph [0239-0240]: “Multiplying a 4-bit activation by an 8-bit weight also takes two clock cycles. In this case, ABU 141 keeps the activation unchanged during the two clock cycles… Multiplying an 8-bit activation by an 8-bit weight takes four clock cycles. ABU 141 broadcasts activation LSN during the first and second clock cycles and broadcasts activation MSN during the third and fourth clock cycles” [i.e., multiplication operations are performed in different cycles (group of operations)], paragraphs [0242-0243]: “FIGS. 1C through 1F, 8-bit IFM slices 110 arrive from SRAM 109 into an activations buffer, or “IFM buffer” 124 (which operates as a plurality of parallel queues)… As 4-by-8-bit multipliers 126 compute activation-and-weight products, adder trees 128 sum up these products, into dot products ΣA,c = 0x2*w0, c, 0+0x4*w1, c, 0+0x8*w2, c, 0+0xe*w3, c, 0 column-wise, for OFM column index c ranging from 0 to 7” [i.e., a plurality of multiplier units operate in parallel across the array to process different parts of the feature map (IFM slices)] and paragraph [0215]: “As the four convolutions at locations shown in FIG. 3E proceed in parallel” [i.e., convolutions involve dot-product multiplication performed in parallel]),
on a respective input operand from a respective input register file and a respective weight operand from a respective weight register file (see, e.g., paragraphs [0232-0233]: “…multiplying a uint8 weight magnitude mult_in_a[7:0] by a uint8 activation magnitude to obtain an uint16 result multi_out_abs[15:0] as the product's magnitude…More specifically, the weight register file 127 stores 18 weights {sw0,w_abs0[7:0]} through {sw17,w_abs17[7:0]}… Both weight and activation arrive to multiplier 126 in the sign-and-magnitude format" and paragraphs [0236-0237]: “Note that is_msn gets registered along with the activation sign s_in_b and magnitude mult_in_b[4:0] by registers 417" [i.e., activation values from input registers 417 are multiplied with weight values from weight register file 127]),
wherein each multiplication operation in the group includes a multiplication of a different input element in the respective input operand and a different weight in the respective weight operand (see, e.g., paragraph [0163]: “With activation values 0xa0 through 0xa3 being broadcast, each multiplier unit 103 selects a weight w.sub.row,col,seq associated with the activation the particular multiplier unit is receiving…For example, a 3×3 weight kernel has 9 weights, each weight associated with its respective activation, that can be labeled seq∈{a, b . . . h} in the order of the activations' arrival to the multiplication unit 103” and paragraph [0243]: “As 4-by-8-bit multipliers 126 compute activation-and-weight products, adder trees 128 sum up these products, into dot products ΣA,c = 0x2*w0, c, 0+0x4*w1, c, 0+0x8*w2, c, 0+0xe*w3, c, 0 column-wise, for OFM column index c ranging from 0 to 7” [i.e., the dot product calculation is a summation of a group of multiplication operations occurring over different cycles]);
an adder assembly4 configured to perform accumulation operations on products generated by the plurality of multipliers in parallel and to generate an output operand (see, e.g., paragraph [0158]: “…each MR tile 102 contains an 8×16 array of multiplier units 126. The array of multiply-reduce units is organized into 8 multiply-reduce columns 133” and paragraph [0175]: “FIG. 1J illustrates the accumulate-and-return unit (ARU) 167…As the adder tree 128A output arrives to accumulation module 150 in AA 168, the adder tree's output is, optionally, scaled by a signed up-shifter 175 and, optionally, added to the accumulator 130A contents using adder 181…Subsequently, 9 IFM slices corresponding to one 3×3×16 activations tensor may be supplied to the MR tile, e.g. over 9 clocks, to compute 9 dot products. Completing the convolution requires adding these 9 dot products in accumulator 130A (and/or 130B)” and paragraph [0180]: “The eight MR columns 133 operate in parallel, producing a stream of OFM vectors (slices), where each OFM slice has, correspondingly, eight elements” [i.e. multiply-reduce columns 133 perform operations in parallel]);
and an output register file5 configured to store the output operand (see, e.g., paragraph [0156]: “Tiles 102 also connect to the SRAM bank sets 109 via an output feature map (OFM) delivery fabric 106 that transmits computed results from tiles 102 to SRAM bank sets 109 for storage” [i.e., SRAM bank sets serve as output register files storing output result operands] and paragraph [0158]: “Subsequently, the accumulate-and-return (ARU) unit 167 in MR column 133 may perform additional summation between tiles using the reduction fabric 111 and/or accumulation in order to support large weight kernel sizes. ARU 167 in each MR column 133 also typically applies an activation function to the sum and sends the resulting activation back to SRAM 109 via OFM delivery fabric 106”),
wherein the output operand includes a sequence of output elements (see, e.g., paragraph [0165]: “Following the example in FIG. 1D, in each column 133 “col”, adder tree 128A computes the desired dot product, stores it in the accumulator 130A Σ.sub.A,col=0xa0*w.sub.0,col,a+0xa1*w.sub.1,col,a+0xa2*w.sub.2,col,a+0xa3*w.sub.3,col,a for col=[0 . . . 3]. The computation involving IFM slice “a” has now been completed and the calculated dot product, in each column, is passed on to the activation function module 151”
[i.e., a dot product (output operand) is produced as a sequence of elements generated in multiple channels]),
and each output element corresponds to a different channel of the plurality of channels (see, e.g., paragraph [0163]: “IFM has 4 channels, OFM has 8 channels and NPU has only one MR tile, multiplier unit located at row “row”, column “col” within MR tile 102 receives activation broadcast lane “row” and computes partial product for MR column “col” that, respectively, contributes to OFM channel O.sub.col.” [i.e., output feature map has multiple channels in which output operands are distributed amongst respectively per column]).
Regarding claim 2, as discussed above Ovsiannikov discloses the apparatus of claim 1. Ovsiannikov further discloses wherein a position of the different input element in the respective input operand matches a position of the different weight in the respective weight operand (see, e.g., paragraphs [0165-0167]: “Following the example in FIG. 1D, in each column 133 “col”, adder tree 128A computes the desired dot product, stores it in the accumulator 130A Σ.sub.A,col=0xa0*w.sub.0,col,a+0xa1*w.sub.1,col,a+0xa2*w.sub.2,col,a+0xa3*w.sub.3,col,a for col=[0 . . . 3]… At the same time, in each column 133, the second adder tree 128B computes a portion of the desired dot product for IFM slice “c” and stores it in the accumulator 130B: Σ=.sub.B,col=0xc1*w.sub.1,col,c for col=[0 . . . 3]” [i.e., the notation of the dot product describes an input element 0x0an and a weight w.sub.n in alignment of position]),
Regarding claim 4, as discussed above Ovsiannikov discloses the apparatus of claim 1. Ovsiannikov further discloses wherein the adder assembly comprises a first group of adders and a second group of one or more adders (see, e.g., paragraph [0175]: “FIG. 1J illustrates the accumulate-and-return unit (ARU) 167 as described in the '610 application. ARU 167 contains two accumulation-and-activation (AA) channels 168, one per adder tree 128A and 128B…Completing the convolution requires adding these 9 dot products in accumulator 130A (and/or 130B)" and [0193]: “Referring to FIG. 2F, each quad has its own pair of adder trees 213 and 214 with corresponding accumulator pairs in ARU"),
an adder6 in the first group is configured to accumulate products generated by at least two multipliers of the plurality of multipliers (see, e.g., paragraph [0164]: “With 0xa0 . . . 0ax3 activations being broadcast and w.sub.row,col,a weights selected by MR units, multiplier 126 in each MR unit 103 proceeds to compute the product of activation act.sub.ow,a with W.sub.row,col,a: p.sub.row,col=w.sub.row,col,a*act.sub.row,a. Subsequently, adder trees 128A and/or 128B proceed to sum up values p.sub.row,col,a for col=[0 . . . 3]: S.sub.col=Σp.sub.row,col. as accumulators 130A and/or 130B in each ARU 167 work to accumulate S.sub.col over time"),
and an adder in the second group is configured to accumulate sums generated by at least two adders in the first group (see, e.g., paragraph [0175]: “As the adder tree 128A output arrives to accumulation module 150 in AA 168, the adder tree's output is…added to the accumulator 130A contents using adder 181…Completing the convolution requires adding these 9 dot products in accumulator 130A (and/or 130B)” [i.e., an accumulator (second group of adders) accumulates the sums of the adder trees (first group of adders)] and paragraph [0179]: “As the adder tree 128A output arrives to accumulation module 150 in AA 168, the adder tree's output is…added to the accumulator 130A contents using adder 181”).
Regarding claim 5, as discussed above Ovsiannikov discloses the apparatus of claim 1. Ovsiannikov further discloses a plurality of processing elements including the processing element (see, e.g., paragraph [0156]: “a plurality of SRAM bank sets 109 (each including several, e.g., four SRAM banks) may be connected to Multiply-and-Reduce tiles 102 (or “MR tiles”) through an input feature map (IFM) delivery fabric 104” [i.e., Multiply-and-Reduce tiles function as processing elements] and paragraph [0177]: “Since the computation throughput and delay may vary slightly between MR tiles 102 due to fluctuations in IFM sparsity as different MR tiles simultaneously receive and process different portions of the IFM tensor”),
the plurality of processing elements7 configured to generate a number of output operands, each of which is generated by a different one of the plurality of processing elements (see, e.g., paragraph [0180]: “The eight MR columns 133 operate in parallel, producing a stream of OFM vectors (slices), where each OFM slice has, correspondingly, eight elements. Since sparse activation processing may result in each MR tile 102 computing results with slightly different throughput and delays, FIFOs 198 are responsible for re-aligning OFM outputs between MR tiles for streaming OFM slices over OFM delivery fabric 106 from various MR tiles 102 to SRAMs 109”);
and an additional adder assembly configured to perform accumulation operations on the number of output operands and to generate a new output operand (see, e.g., paragraph [0177]: “In this case each MR tile calculates a portion of the final dot product and these partial dot product results must be further reduced (added up) using reduction fabric 111 that spans all MR tiles” [i.e., the reduction fabric 111 acts as an additional adder assembly that accumulates the partial dot product results to produce another output operand]),
wherein each accumulation operation includes an accumulation of the number of output elements (see, e.g., paragraph [0177]: “In this case each MR tile calculates a portion of the final dot product and these partial dot product results must be further reduced (added up) using reduction fabric 111 that spans all MR tiles” [i.e., the reduction fabric 111 accumulates partial dot products (a number of output elements) of the output operand (the final dot product)]),
each of the number of output elements is from a different one of the number of output operands (see, e.g., paragraph [0177]: “In this case each MR tile calculates a portion of the final dot product and these partial dot product results must be further reduced (added up) using reduction fabric 111 that spans all MR tiles” and paragraph [0180]: “The eight MR columns 133 operate in parallel, producing a stream of OFM vectors (slices), where each OFM slice has, correspondingly, eight elements” [i.e., each MR tiles/columns produce a number of output operands in which partial dot product portions originate]),
and the number of output elements correspond to a same channel of the plurality of channels (see, e.g., paragraph [0158]: “Using these received activations each multiply-reduce column calculates one OFM channel. Specifically, each multiply-reduce column calculates a dot product of 16 incoming activations with corresponding 16 weights…Subsequently, the accumulate-and-return (ARU) unit 167 in MR column 133 may perform additional summation between tiles using the reduction fabric 111” [i.e., the accumulation performed by the reduction fabric 111 for a same OFM channel]).
Regarding claim 6, as discussed above Ovsiannikov discloses the apparatus of claim 5. Ovsiannikov further discloses wherein the plurality of processing elements is arranged in a column (see, e.g., paragraph [0180]: “The eight MR columns 133 operate in parallel, producing a stream of OFM vectors (slices), where each OFM slice has, correspondingly, eight elements”),
and the additional adder assembly is external to the column (see, e.g., paragraph [0177]: “partial dot product results must be further reduced (added up) using reduction fabric 111 that spans all MR tiles" [i.e., the reduction fabric as shown in FIG. 1A is a separate component that spans all MR tiles (and by extension MR columns)]).
Regarding claim 13, as discussed above Ovsiannikov discloses the apparatus of claim 1. Ovsiannikov further discloses further comprising an array of processing elements that includes the processing element (see, e.g., paragraph [0158]: “each MR tile 102 contains an 8×16 array of multiplier units 126. The array of multiply-reduce units is organized into 8 multiply-reduce columns 133”),
wherein input register files of one or more other processing elements in the array are configured to store same input operands as the plurality of input register files of the processing element (see, e.g., paragraph [0159]: “Inside each MR tile 102, a compact activations cache, or “IFM cache” 139 stores activations delivered from SRAM 109 over IFM delivery fabric 104…an element in the IFM tensor may be needed several times to calculate convolution at adjacent locations within the OFM tensor. Therefore, caching activations locally in each MR tile may allow computation to proceed without the need to fetch same IFM element from SRAM repetitively” [i.e., local activation cache in each MR tile (processing element) and elements in the IFM tensor is used at adjacent locations (same input operands are stored in multiple MR tiles)] and paragraph [0160]: “Activations from IFM cache 139 pass through the activation broadcast unit (ABU) 141… Subsequently, activations in sign-and-magnitude numeric format are broadcast over 16 activation lanes 137 to all eight MR columns 133 simultaneously”
[i.e., Multiplier units of each MR column receive the same activations (input operands) from the IFM cache (input register files)]).
Regarding claim 14, as discussed above Ovsiannikov discloses the apparatus of claim 13. Ovsiannikov further discloses wherein weight register files of one or more other processing elements in the array are configured to store the same weight operands as the plurality of weight register files of the processing element (see, e.g., paragraphs [0161-0163]: “Multiplier units 126 store kernel weights locally in weight register file 127, see FIG. 1C… With activation values 0xa0 through 0xa3 being broadcast, each multiplier unit 103 selects a weight w.sub.row,col,seq associated with the activation the particular multiplier unit is receiving ").
Regarding Independent Claim 16, Ovsiannikov discloses A method for deep learning, comprising: determining a number of input register files in a processing element (PE) of a plurality of PEs (see, e.g., paragraph [0159]: “Inside each MR tile 102, a compact activations cache, or “IFM cache” 139 stores activations delivered from SRAM 109 over IFM delivery fabric 104” [i.e., MR tiles (PEs) each contain an IFM cache (input register file)] and paragraphs [0177-0179]: “However, weights from large weight kernels have to be distributed over multiple MR tiles, as described in the '610 application. In other words, the dot product calculation now takes place over several MR tiles, as opposed to just one MR tile…Since the computation throughput and delay may vary slightly between MR tiles 102 due to fluctuations in IFM sparsity as different MR tiles simultaneously receive and process different portions of the IFM tensor… In cases when the weight kernel is too large to fit into all MR tiles 102 available in NPU 100, the dot product computation has to be done by processing the IFM tensor two or more times while saving partial results in SRAM, as explained in disclosure '601” [i.e., a determination of the number of MR tiles (each containing input register files) is made based on the weight kernel size]),
the PE comprising a plurality of multipliers to perform multiplication operations in parallel (see, e.g., paragraph [0158]: “…each MR tile 102 contains an 8×16 array of multiplier units 126. The array of multiply-reduce units is organized into 8 multiply-reduce columns 133” and paragraph [0175]: “FIG. 1J illustrates the accumulate-and-return unit (ARU) 167…As the adder tree 128A output arrives to accumulation module 150 in AA 168, the adder tree's output is, optionally, scaled by a signed up-shifter 175 and, optionally, added to the accumulator 130A contents using adder 181…Subsequently, 9 IFM slices corresponding to one 3×3×16 activations tensor may be supplied to the MR tile, e.g. over 9 clocks, to compute 9 dot products. Completing the convolution requires adding these 9 dot products in accumulator 130A (and/or 130B)” and paragraph [0180]: “The eight MR columns 133 operate in parallel, producing a stream of OFM vectors (slices), where each OFM slice has, correspondingly, eight elements” [i.e. multiply-reduce columns 133 (containing a plurality of multipliers) perform multiplication operations in parallel]),
the plurality of PEs configured to perform multiply-accumulation operations on a filter and an input feature map that includes a plurality of channels (see, e.g., paragraph [0158]: “each MR tile 102 contains an 8×16 array of multiplier units 126. The array of multiply-reduce units is organized into 8 multiply-reduce columns 133. All 8 multiply-reduce columns receive same IFM data, which consists of 16 IFM channels” and paragraph [0175]: “For example, if the weight kernel size is 3×3×16×8, where 3×3 is the planar width and height and 16 is the number of channels, a single MR tile can perform the associated convolution by storing 3×3=9 weights in each of 16×8 multiplier units in the tile”);
forming the number of input operands from the input feature map (see, e.g., paragraph [0163]: “FIG. 1D shows the first two IFM vectors 110 (IFM slices) arrive to IFM buffer 124, with IFM cache 139 and type converter 135 omitted for clarity. In this clock cycle, activations in lanes 0, 1, 2 and 3 have values 0xa0, 0xa1, 0xa2 and 0xa3 respectively. In this example, since none of the activation values at the front of the IFM buffer are zero, all four non-zero activations 0xa0, 0xa1, 0xa2 and 0xa3 get broadcast to MR columns O.sub.0 through O.sub.7 via activation lanes 137” and paragraph [0173]: “FIGS. 1H and 1I illustrate IFM buffer 124, within ABU 141… where values from IFM cache 139 arrive from the left and gradually propagate to the right (Col0) for broadcast to MR array 122, while being reordered to eliminate zero-valued elements”),
each input operand including a sequence of input elements from the input feature map (see, e.g., paragraph [0210]: “Each IFM slice 201 carries four ARGB pixels. The example in FIGS. 3C through 3J illustrates a 3×3 convolution window scanning the input image, i.e. IFM tensor 201, in a planar fashion… In particular, the very first IFM slice 110 that arrives to IFM cache 139 contains ARGB pixels at planar locations (0, 0), (0, 1), (0, 2) and (0, 3)”),
each input element corresponding to a different channel of the plurality of channels (see, e.g., paragraph [0208]: “As shown in FIG. 3A, the 3D IFM tensor 200 has planar dimensions of 5 rows and 12 columns and contains four depth channels, corresponding to the alpha and three color channels comprising the color image. Each planar location at image row “r” and column “c” contains a four-element vector {A.sub.r,c, R.sub.r,c, G.sub.r,c, B.sub.r,c}”);
transferring each of the number of input operands to a different one of the input register files (see, e.g., paragraph [0159]: “Inside each MR tile 102, a compact activations cache, or “IFM cache” 139 stores activations delivered from SRAM 109 over IFM delivery fabric 104” and paragraph [0210]: “In particular, the very first IFM slice 110 that arrives to IFM cache 139 contains ARGB pixels at planar locations (0, 0), (0, 1), (0, 2) and (0, 3), where we use the (row, column) notation to indicate planar coordinates. Each of these four pixels respectively come to IFM cache 139 lanes 0 . . . 3, 4 . . . 7, 8 . . . 11 and 12 . . . 15, in turn corresponding to quads 0, 1, 2 and 3”),
different ones of the plurality of multipliers to receive input operands from different ones of the input register files (see, e.g., paragraph [0163]: “each multiplier unit 103 selects a weight wrow,col,seq associated with the activation the particular multiplier unit is receiving. Assuming, for simplicity, that IFM has 4 channels, OFM has 8 channels and NPU has only one MR tile, multiplier unit located at row ‘row’, column ‘col’ within MR tile 102 receives activation broadcast lane ‘row’” [i.e., different multipliers (at different rows) receive different input operands (activations) from different lanes of the IFM buffer from the IFM cache (input register files)] and paragraph [0189]: “Note that the 16 depth channels have been sub-divided into four sets, each set corresponding to a quad 215. More specifically, depth channels 0, 1, 2 and 3 correspond to the first quad 215 comprised of activation lanes act0, act1, act2 and act3” [i.e., different groups of multipliers (quads) receive different input operands (distinct depth channels) from different specific hardware lanes/registers (act0-3)]);
determining a size of the filter (see, e.g., paragraph [0179]: “In cases when the weight kernel is too large to fit into all MR tiles 102 available in NPU 100, the dot product computation has to be done by processing the IFM tensor two or more times while saving partial results in SRAM”);
identifying, based on the size of the filter, a tier in an adder assembly coupled to the plurality of PEs (see, e.g., paragraphs [0175-178] “FIG. 1J illustrates the accumulate-and-return unit (ARU) 167 as described in the '610 application. ARU 167 contains two accumulation-and-activation (AA) channels 168, one per adder tree 128A and 128B, and a return module 152… When weight kernel size is too large to fit into a single MR tile, reduction fabric 111 effectively acts as extra upper levels for adder trees 128A/B to add partial dot products generated by different MR tiles and forwards the finished sum back to ARU 167 via multiplexer 174 to apply the activation function” [i.e., the reduction fabric acts as an upper tier for adder trees within the ARU (adder assembly) and is used based on kernel (filter) size] and paragraph [0202]: “Note that ColAdderStage outputs of accumulator units 150 are shown to be disconnected when computing depth-wise convolution since there is no need to use the reduction fabric 111. More specifically, the reduction (addition using adder trees as well as accumulation) during depth-wise convolution is accomplished within each MR tile”),
the adder assembly including adders arranged in a sequence of tiers (see, e.g., paragraph [0164]: “reduction fabric 111 further may add up accumulator 130A (B) or adder trees to compute the dot product over weight kernels that are too large to fit into a single MR tile and thus end up being distributed over multiple NPU MR tiles” [i.e., reduction fabric functions as an upper tier of the ARU], paragraphs [0166-0167]: “in each column 133, adder tree 128A computes the desired dot product for IFM slice “b” and stores it in the accumulator 130A… At the same time, in each column 133, the second adder tree 128B computes a portion of the desired dot product for IFM slice “c” and stores it in the accumulator 130B” [i.e., adders and accumulators work within a tier of the ARU], paragraph [0178]: “When weight kernel size is too large to fit into a single MR tile, reduction fabric 111 effectively acts as extra upper levels for adder trees 128A/B to add partial dot products “ [i.e., the reduction fabric functions as an upper tier adder of the ARU] and paragraph [0186]: “Note that large adder trees 128A/B become split into sub-trees 213, 214 and send their outputs into ARUs 210 that now each have 4 pairs of accumulators in ARU”),
each tier including one or more adders (see, e.g., paragraph [0164]: “reduction fabric 111 further may add up accumulator 130A (B) or adder trees to compute the dot product” [i.e., multiple adders are added in the upper tier] and paragraph [0175]: “ARU 167 contains two accumulation-and-activation (AA) channels 168, one per adder tree 128A and 128B, and a return module 152” [i.e., multiple adders in the lower tier]);
and obtaining an output operand from each adder in the tier (see, e.g., paragraph [0165]: “Following the example in FIG. 1D, in each column 133 “col”, adder tree 128A computes the desired dot product, stores it in the accumulator 130A… The computation involving IFM slice “a” has now been completed and the calculated dot product, in each column, is passed on to the activation function module 151” and paragraphs [0175-0178] “As the adder tree 128A output arrives to accumulation module 150 in AA 168, the adder tree's output is, optionally, scaled by a signed up-shifter 175 and, optionally, added to the accumulator 130A contents using adder 181… When weight kernel size is too large to fit into a single MR tile, reduction fabric 111 effectively acts as extra upper levels for adder trees 128A/B to add partial dot products generated by different MR tiles and forwards the finished sum back to ARU 167 via multiplexer 174 to apply the activation function” [i.e., upper tier output operand (finished sum of partial dot products) is forwarded to ARU]),
wherein the output operand includes a sequence of output elements (see, e.g., paragraph [0165]: “Following the example in FIG. 1D, in each column 133 “col”, adder tree 128A computes the desired dot product, stores it in the accumulator 130A Σ.sub.A,col=0xa0*w.sub.0,col,a+0xa1*w.sub.1,col,a+0xa2*w.sub.2,col,a+0xa3*w.sub.3,col,a for col=[0 . . . 3]” and paragraph [0202]: “The return unit 152 collects final outputs from one or more activation units 151 and passes the resulting OFM slices (vectors) on to OFM delivery fabric 104 for storage in SRAM 109” [i.e., OFM slices (output operands) contain sequences of output elements]),
and each output element corresponds to a different channel of the plurality of channels (see, e.g., paragraph [0205]: “For example, OFM slice from a depth-wise convolution computed at location row “r and column “c” should be a vector of the shape {O.sub.r,c,d, O.sub.r,c,d+1, O.sub.r,c,d+2, O.sub.r,c,d+3, O.sub.r,c,d+4, O.sub.r,c,d+5, O.sub.r,c,d+6, O.sub.r,c,d+7}, where index d corresponds to the depth-wise convolution's depth channel. More specifically, since depth-wise convolution operates on each depth channel independently, the depth-wise convolution's output depth channel is same as the depth-wise convolution's input channel, here referred to as just ‘depth-wise convolution's channel’”).
Regarding claim 17, as discussed above Ovsiannikov discloses the method of claim 16. Ovsiannikov further discloses wherein the filter includes weights arranged in rows and columns (see, e.g., paragraph [0163]: “OFM has 8 channels and NPU has only one MR tile, multiplier unit located at row “row”, column “col” within MR tile 102 receives activation broadcast lane “row” and computes partial product for MR column “col” that, respectively, contributes to OFM channel O.sub.col. Weight register index “seq” corresponds to the weight's sequence order in the weight kernel. For example, a 3×3 weight kernel has 9 weights, each weight associated with its respective activation”),
and the size of the filter is a number of the rows or a number of the columns (see, e.g., paragraph [0175]: “For example, if the weight kernel size is 3×3×16×8, where 3×3 is the planar width and height and 16 is the number of channels, a single MR tile can perform the associated convolution by storing 3×3=9 weights in each of 16×8 multiplier units in the tile”).
Regarding claim 18, as discussed above Ovsiannikov discloses the method of claim 16. Ovsiannikov further discloses further comprising: determining a number of weight register files in the PE (see, e.g., paragraph [0232]: “To support look-aside operation with the distance of one, multiplier 126 can receive its weight—via multiplexer 414—from three sources: the weight register file 127 co-located with the 126 multiplier in the same multiplier unit 103 via bus {swt_self, wt_abs_self[7:0]}, the weight register file 127 located in the multiplier unit 103 one lane above via bus {swt_up1_in, wt_abs_up1_in[7:0]} and the weight register file 127 located in the multiplier unit 103 one lane below via bus {swt_dn1_in, wt_abs_dn1_in[7:0]} " [i.e., the multiplier fetches weights from a number or weight register files functions as determining a number of weight register files to access]);
forming the number of weight operands from the filter (see, e.g., paragraph [0161]: “Multiplier units 126 store kernel weights locally in weight register file 127, see FIG. 1C. Similar to activations, the weights are also stored in the sign-and-magnitude format to support calculation using signed, unsigned, 8-bit, 16-bit and 8×N-bit weights. Similar to ABU's numeric format conversion, weight decompression unit (WDU) converts signed and multi-byte weights into one or more sign-and-magnitude values as it loads weights from SRAM to weight register files 127” [i.e., conversions from the WDU form the kernel weight operands from the kernel]),
each weight operand including a sequence of weight from the filter (see, e.g., paragraph [0163]: “For example, a 3×3 weight kernel has 9 weights, each weight associated with its respective activation, that can be labeled seq ∈ {a, b . . . h} in the order of the activations' arrival to the multiplication unit 103"),
each weight corresponding to a different channel of the plurality of channels (see, e.g., paragraph [0175]: “For example, if the weight kernel size is 3×3×16×8, where 3×3 is the planar width and height and 16 is the number of channels, a single MR tile can perform the associated convolution by storing 3×3=9 weights in each of 16×8 multiplier units in the tile” and paragraph [0189]: “More specifically, as illustrated in FIG. 2E, each depth channel 202—independently from all other depth channels—is 2D-convolved with a planar 3×3 patch of weights 207 (associated with that channel) to generate output elements 208 that make up the output (OFM) tensor 209)”);
and transferring each of the number of weight operands to a different one of the weight register files (see, e.g., paragraph [0161]: “Similar to ABU's numeric format conversion, weight decompression unit (WDU) converts signed and multi-byte weights into one or more sign-and-magnitude values as it loads weights from SRAM to weight register files 127”, and paragraphs [0232-235] the weight register file 127 co-located with the 126 multiplier in the same multiplier unit 103 via bus {swt_self, wt_abs_self[7:0]}, the weight register file 127 located in the multiplier unit 103 one lane above via bus {swt_up1_in, wt_abs_up1_in[7:0]} and the weight register file 127 located in the multiplier unit 103 one lane below via bus {swt_dn1_in, wt_abs_dn1_in[7:0]}… Before computation starts, weight decompression unit (WDM) 138 loads weights into weight registers 127 over a vertical bus {swt_in[C], wt_abs_Id_in[7:0][C]}, where C is the MR column index 0 . . . 7. To load one weight into each of the eight multiplier units in lane with index L 0 . . . 15, WDM asserts wt_Id_en_lane[L] causing de-multiplexer 410 to un-gate the clock input of weight register at index wt_Id_idx[4:0], where wt_Id_idx ranges from 0 to 17”).
Regarding claim 23, as discussed above Ovsiannikov discloses the method of claim 16. Ovsiannikov further discloses further comprising: transferring the same input operands to the input register files in the PE and to input register files in another PE of the plurality of PEs (see, e.g. paragraph [0159]: “Inside each MR tile 102, a compact activations cache, or “IFM cache” 139 stores activations delivered from SRAM 109 over IFM delivery fabric 104…Therefore, caching activations locally in each MR tile may allow computation to proceed without the need to fetch same IFM element from SRAM repetitively” [i.e., local activation cache in each MR tile (processing element)] and paragraph [0160]: “Activations from IFM cache 139 pass through the activation broadcast unit (ABU) 141… Subsequently, activations in sign-and-magnitude numeric format are broadcast over 16 activation lanes 137 to all eight MR columns 133 simultaneously” [i.e., Multiplier units of each MR column receive the same activations (input operands) from the IFM cache (input register files)]).
Regarding claim 24, as discussed above Ovsiannikov discloses the method of claim 23. Ovsiannikov further discloses further comprising: transferring the same weight operands to weight register files in the PE and to weight register files in another PE of the plurality of PEs (see, e.g., paragraph [0212]: “convolution weight kernels must be pre-loaded into weight register files of multiply-reduce units before layer convolution can commence. In the case of ARGB convolution kernels loaded into MR.sub.0 . . . 3, c, where column c=0 . . . 7, may have identical values since MR.sub.0 . . . 3, c perform same convolution, i.e. with same weight kernel, at different planar locations” [i.e., identical weight kernels are loaded into several MR units in each MR tile (PEs)] and paragraph [0220]: “NPU can be configured to increase the number of OFM channels, e.g. beyond the 8 values contained in a single OFM slice, by feeding same ARGB data into multiple MR tiles, in parallel, to apply a different convolution weight kernel”),
wherein each weight operand includes a sequence of weight from the filter (see, e.g., paragraph [0163]: “For example, a 3×3 weight kernel has 9 weights, each weight associated with its respective activation, that can be labeled seq∈{a, b . . . h} in the order of the activations' arrival to the multiplication unit 103”),
and each weight corresponds to a different channel of the plurality of channels (see, e.g., paragraph [0158]: “All 8 multiply-reduce columns receive same IFM data, which consists of 16 IFM channels. Using these received activations each multiply-reduce column calculates one OFM channel. Specifically, each multiply-reduce column calculates a dot product of 16 incoming activations with corresponding 16 weights” [i.e., 16 weights corresponding to one of the 16 IFM channels]).
Regarding claim 25, as discussed above Ovsiannikov discloses the method of claim 16. Ovsiannikov further discloses further comprising: selecting, based on the size of the filter, a subset of PEs from the plurality of PEs (see, e.g., paragraph [0177]: “The value of accumulator 130A/B at this point may be the final result when weight kernel is small enough to fit into a single MR tile. In this case, the final result passes to the activation module 151… However, weights from large weight kernels have to be distributed over multiple MR tiles, as described in the '610 application. In other words, the dot product calculation now takes place over several MR tiles, as opposed to just one MR tile" [i.e., dot product calculations occur over a selection of MR tiles based on the size of the weight kernel (corresponding to filter size)]);
and transferring input operands to input registers files of the PEs in the subset (see, e.g., paragraph [0159]: “Inside each MR tile 102, a compact activations cache, or ‘IFM cache’ 139 stores activations delivered from SRAM 109 over IFM delivery fabric 104" [i.e., input operands (activations) are transferred (delivered) and stored in input register files (IFM cache 139) within each PE (MR tile)]).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 3 and 7 are rejected under 35 U.S.C. 103 as being unpatentable over Ovsiannikov in view of Kwon (US 20220164164 A1; hereinafter Kwon).
Regarding claim 3, as discussed above, Ovsiannikov discloses the apparatus of claim 1. Ovsiannikov further discloses wherein the filter includes a plurality of weights arranged in rows and columns (see, e.g., paragraph [0163]: “each multiplier unit 103 selects a weight w.sub.row,col,seq associated with the activation the particular multiplier unit is receiving…MR tile 102 receives activation broadcast lane “row” and computes partial product for MR column “col” that, respectively, contributes to OFM channel O.sub.col. Weight register index “seq” corresponds to the weight's sequence order in the weight kernel. For example, a 3×3 weight kernel has 9 weights, each weight associated with its respective activation”),
an output element in the output operand is a sum of products of input elements from different input operands and weights from different weight operands, (seem e.g., paragraph [0165]: “Following the example in FIG. 1D, in each column 133 “col”, adder tree 128A computes the desired dot product, stores it in the accumulator 130A Σ.sub.A,col=0xa0*w.sub.0,col,a+0xa1*w.sub.1,col,a+0xa2*w.sub.2,col,a+0xa3*w.sub.3,col,a for col=[0 . . . 3]” and paragraph [0175]: “accumulation of adder tree output over several clock cycles may be necessary to convolve an entire weight kernel. For example… a single MR tile can perform the associated convolution by storing 3×3=9 weights in each of 16×8 multiplier units in the tile. Subsequently, 9 IFM slices corresponding to one 3×3×16 activations tensor may be supplied to the MR tile, e.g. over 9 clocks, to compute 9 dot products. Completing the convolution requires adding these 9 dot products in accumulator 130A (and/or 130B), which is done simultaneously with the 9 IFM slices being supplied to the MR tile” [i.e., the accumulated result of adder tree output includes an output element of a dot product (sum of products of input elements) with different inputs and weights from IFM slices]).
Although Ovsiannikov substantially discloses the claimed invention, Ovsiannikov fails to explicitly teach the limitation and the weights for the output element are in a same row of the filter.
In the same field, analogous art Kwon teaches and the weights for the output element are in a same row of the filter (see, e.g., Kwon paragraph [0055]: “the adder tree structure may perform a convolution operation… the input feature map 140 and weights 0, 1, 2, 3, 4,… of a first filter 151 of the filters 150” and paragraphs [0088-0091]: “Referring to FIG. 5, in a depthwise convolution operation, filter data (e.g., weights) may be disposed as shown in examples 510 to 560 to reduce memory read power…the filter data are stored in rows… Further, when processing a large filter such as a 9×9 filter, by storing data in rows of adder trees as in the example 560, the filter may be processed without changing the value of a register” [i.e., the rows of filter data (output elements from adder trees) are mapped to the rows of adder trees] and paragraph [0105]: “For example, when a 2×2 convolution operation is to be performed, by storing four pieces of filter data in input registers in respective rows and inserting activations into respective corresponding rows, it is possible to perform the operation without a partial sum”).
Ovsiannikov and Kwon are analogous art because they are both directed to hardware implementations of neural networks (see, e.g., Ovsiannikov paragraph [0203], Kwon paragraph [0113]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ovsiannikov to incorporate the teachings of Kwon to configure the weights of output elements to be in the same row as a filter. Doing so would have allowed Ovsiannikov to use Kwon's method in order to “reduce the power consumption” of “reading from/writing to the memory” in “the deep learning operation”, as suggested by Kwon (see, e.g., Kwon, paragraph [0106]).
Regarding claim 7, as discussed above, Ovsiannikov discloses the apparatus of claim 5. Ovsiannikov further discloses wherein the filter includes a plurality of weights arranged in rows and columns, (see, e.g., paragraph [0163]: “each multiplier unit 103 selects a weight w.sub.row,col,seq associated with the activation the particular multiplier unit is receiving…MR tile 102 receives activation broadcast lane “row” and computes partial product for MR column “col” that, respectively, contributes to OFM channel O.sub.col. Weight register index “seq” corresponds to the weight's sequence order in the weight kernel. For example, a 3×3 weight kernel has 9 weights, each weight associated with its respective activation”),
each of the number of output elements is a sum of products of input elements from different input operands and weights from different weight operands, (see, e.g., paragraph [0165]: “Following the example in FIG. 1D, in each column 133 “col”, adder tree 128A computes the desired dot product, stores it in the accumulator 130A Σ.sub.A,col=0xa0*w.sub.0,col,a+0xa1*w.sub.1,col,a+0xa2*w.sub.2,col,a+0xa3*w.sub.3,col,a for col=[0 . . . 3]” and paragraph [0175]: “accumulation of adder tree output over several clock cycles may be necessary to convolve an entire weight kernel. For example… a single MR tile can perform the associated convolution by storing 3×3=9 weights in each of 16×8 multiplier units in the tile. Subsequently, 9 IFM slices corresponding to one 3×3×16 activations tensor may be supplied to the MR tile, e.g. over 9 clocks, to compute 9 dot products. Completing the convolution requires adding these 9 dot products in accumulator 130A (and/or 130B), which is done simultaneously with the 9 IFM slices being supplied to the MR tile” [i.e., the accumulated result of adder tree output includes output elements of a dot product (sum of products of input elements) with different inputs and weights from IFM slices]).
Although Ovsiannikov substantially discloses the claimed invention, Ovsiannikov fails to explicitly teach the limitation the weights for an output element of the number of output elements are in a same row of the filter and are in a different row of the filter from another output element of the number of output elements.
In the same field, analogous art Kwon teaches the weights for an output element of the number of output elements are in a same row of the filter and are in a different row of the filter from another output element of the number of output elements (see, e.g., paragraphs paragraph [0055]: “the adder tree structure may perform a convolution operation… the input feature map 140 and weights 0, 1, 2, 3, 4,… of a first filter 151 of the filters 150” and paragraphs [0088-0091]: “Referring to FIG. 5, in a depthwise convolution operation, filter data (e.g., weights) may be disposed as shown in examples 510 to 560 to reduce memory read power… the filter data are stored in rows… Further, when processing a large filter such as a 9×9 filter, by storing data in rows of adder trees as in the example 560, the filter may be processed without changing the value of a register” [i.e., the rows of filter data (output elements from adder trees) are mapped to the rows of adder trees], paragraph [0104]: “Referring to FIG. 8, a deep learning operation apparatus may include an output register 830 configured to store the sum of output data of adder trees included in a first row 810, and an output multiplexer 840 configured to determine whether to transfer data stored in the output register 830 to a second row 820” [i.e., output sums (weights for an output element) from distinct hardware rows that process data from respective filter rows] and paragraph [0105]: “For example, when a 2×2 convolution operation is to be performed, by storing four pieces of filter data in input registers in respective rows and inserting activations into respective corresponding rows, it is possible to perform the operation without a partial sum”).
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Ovsiannikov in view of Badaroglu (US 12340304 B2; hereinafter Badaroglu).
Regarding claim 8, as discussed above, Ovsiannikov discloses the apparatus of claim 5. However, Ovsiannikov fails to teach wherein the additional adder assembly comprises a first group of adders and a second group of one or more adders,
an adder in the first group is configured to accumulate outputs from at least two processing elements of the plurality of processing elements,
and an adder in the second group is configured to accumulate outputs from at least two adders in the first group
Nevertheless, in the same field, analogous art Badaroglu teaches wherein the additional adder assembly comprises a first group of adders and a second group of one or more adders (see, e.g., Cols. 13-14 lines 54-67, 1-2: “The systolic flow architecture 700 may include a cascaded series 701 of PE circuits 702.sub.1 to 702.sub.8…each of the PE circuits 702 includes a multiply-and-accumulate (MAC) adder tree 704 and a local accumulator 706)” [i.e., each PE circuit series of MAC adder trees (adder assembly) includes groups of adder trees (first and second groups of adders) in succession]),
an adder in the first group is configured to accumulate outputs from at least two processing elements of the plurality of processing elements (see, e.g., Col. 14 lines 21-25: “The PE circuits 702 may be systolically connected such that the output of a local accumulator 706 from one PE circuit (e.g., PE circuit 702.sub.1) is input as a partial accumulation result to the MAC adder tree 704 of a subsequent PE circuit” and Col. 16 lines 2-4: The large accumulator 711 may accumulate the selection (A1 or A2) with (B) the accumulation result from the previous PE circuit 702” [i.e., the accumulation result from the previous PE circuit is from a prior group of adders in the PE circuit series]),
and an adder in the second group is configured to accumulate outputs from at least two adders in the first group (see, e.g., Col. 14 lines 21-25: “The PE circuits 702 may be systolically connected such that the output of a local accumulator 706 from one PE circuit (e.g., PE circuit 702.sub.1) is input as a partial accumulation result to the MAC adder tree 704 of a subsequent PE circuit” and Col. 16 lines 2-4: “The large accumulator 711 may accumulate the selection (A1 or A2) with (B) the accumulation result from the previous PE circuit 702”
[i.e., the accumulation result from the previous PE circuit is from a prior group of adders in the PE circuit series]).
Ovsiannikov and Badaroglu are analogous art because they are both directed to hardware implementations of neural networks (see, e.g., Ovsiannikov paragraph [0203], Badaroglu Col. 2 lines 1-9).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ovsiannikov to incorporate the teachings of Badaroglu to utilize an adder assembly in a hierarchical manner, in which a first group of adders sums results from processing elements, and a second group of adders accumulates the results from the first group. Doing so would have allowed Ovsiannikov to use Badaroglu's method for “improved handling of partial accumulation results in weight-stationary operations, such as operations occurring in compute-in-memory (CIM) processing elements (PEs)”, as suggested by Badaroglu (see, e.g., Badaroglu, Col. 4 lines 15-18).
Claims 9, 10, 11, 12, 19, 20, 21 and 22 are rejected under 35 U.S.C. 103 as being unpatentable over Ovsiannikov in view of Liu (US 20210124560 A1; hereinafter Liu).
Regarding claim 9, as discussed above, Ovsiannikov discloses the apparatus of claim 1. Ovsiannikov further discloses the plurality of multipliers comprises a first multiplier and a second multiplier, (see, e.g., paragraph [0158]: “each MR tile 102 contains an 8×16 array of multiplier units 126”),
the first multiplier is configured to perform multiplication operations on a first input operand from a first input register file and a first weight operand from a first weight register file at a first time, (see, e.g., paragraph [0164]: “With 0xa0 . . . 0ax3 activations being broadcast and w.sub.row,col,a weights selected by MR units, multiplier 126 in each MR unit 103 proceeds to compute the product of activation act.sub.ow,a with W.sub.row,col,a: p.sub.row,col=w.sub.row,col,a*act.sub.row,a”).
Although Ovsiannikov substantially discloses the claimed invention Ovsiannikov is not relied on for explicitly disclosing the second multiplier is configured to perform multiplication operations on the first input operand from the first input register file and a second weight operand from a second weight register file
at a second time, the second time different from the first time
In the same field, analogous art Liu teaches the second multiplier is configured to perform multiplication operations on the first input operand from the first input register file and a second weight operand from a second weight register file (see, e.g., Liu paragraph [0062] “With respect to MAC Unit.sub.2, register 91 receives element a.sub.1,1 from MAC Unit.sub.1, register 92 receives element w.sub.1,2 from the WO register, multiplier circuit 93 multiplies element a.sub.1,1 and element w.sub.1,2” [i.e., the second multiplier MAC unit 2 performs multiplication operations from input from element received from register 91 (first input register file) and a weight element from the WO register 92 (second weight operand from second weight register file]),
at a second time, the second time different from the first time (see, e.g., Liu paragraph [0060]: “The 2.sup.nd cycle includes MAC Unit.sub.1, MAC Unit.sub.2, and MAC Unit.sub.5”).
Ovsiannikov and Liu are analogous art because they are both directed to hardware implementations of neural networks (see, e.g., Ovsiannikov paragraph [0203], Liu paragraph [0038]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ovsiannikov to incorporate the teachings of Liu to utilize a second multiplier to perform multiplication operations on the first input operand and a second weight operand at a distinct second time. Doing so would have allowed Ovsiannikov to use Liu's method for “efficiently multiplying matrices by performing vector multiply and accumulate (VMAC) operations within each calculation cycle”, as suggested by Liu (see, e.g., Liu, paragraph [0022]).
Regarding claim 10, as discussed above, Ovsiannikov in view of Liu teaches the apparatus of claim 9. Although Ovsiannikov substantially teaches the claimed invention, Ovsiannikov fails to explicitly teach wherein the second multiplier is configured to perform multiplication operations on a second input operand and the second weight operand at the first time.
Nevertheless, in the same field, analogous art Liu teaches wherein the second multiplier is configured to perform multiplication operations on a second input operand and the second weight operand at the first time (see, e.g., paragraphs [0141-0143]: “The second multiplier circuit 253 multiplies the data value, m.sub.a2, provided by the second AO row vector signal line, and the data value, m.sub.w2, provided by second WO column vector signal line, and outputs the resulting data value or intermediate product, ip.sub.2, to accumulator circuit 254… In this embodiment, VMAC.sup.4 unit 250 advantageously performs 4 MAC operations in a single VMAC processing cycle, which reduces hardware power consumption, hardware costs, processing latency, etc.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ovsiannikov to incorporate the teachings of Liu to include a second multiplier configured to perform multiplication operations on a second input operand and the second weight operand a first time. Doing so would have allowed Ovsiannikov to use Liu’s method for “efficiently multiplying matrices by performing vector multiply and accumulate (VMAC) operations within each calculation cycle”, as suggested by Liu (see, e.g., Liu, paragraph [0022]).
Regarding claim 11, as discussed above, Ovsiannikov in view of Liu teaches the apparatus of claim 9. Although Ovsiannikov substantially teaches the claimed invention, Ovsiannikov fails to explicitly teach wherein the first input register file is configured to store a new input operand at the second time.
Nevertheless, in the same field, analogous art Liu teaches wherein the first input register file is configured to store a new input operand at the second time (see, e.g., Liu paragraph [0060]: “The 2.sup.nd cycle includes MAC Unit.sub.1, MAC Unit.sub.2, and MAC Unit.sub.5” and paragraph [0063]: “With respect to MAC Unit.sub.5, register 91 receives element a.sub.2,1 from the AO register” [i.e., the register 91 (first input register file) of MAC Unit 5 receives a new input operand in the second cycle (at the second time)]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ovsiannikov to incorporate the teachings of Liu to store a new input operand in a first input register file at a second time. Doing so would have allowed Ovsiannikov to use Liu’s method for “efficiently multiplying matrices by performing vector multiply and accumulate (VMAC) operations within each calculation cycle”, as suggested by Liu (see, e.g., Liu, paragraph [0022]).
Regarding claim 12, as discussed above, Ovsiannikov teaches the apparatus of claim 11. Although Ovsiannikov substantially discloses the claimed invention, Ovsiannikov fails to explicitly teach wherein the plurality of multipliers further comprises a third multiplier,
and the third multiplier is configured to perform multiplication operations on the new input operand from the first input register file
and a third weight operand from a third weight register file at a third time,
and the third time is after the second time
Nevertheless, in the same field, analogous art Liu teaches wherein the plurality of multipliers further comprises a third multiplier (see, e.g., Liu paragraph [0056] “Referring back to FIG. 3, the elements from matrices 20 and 30 enter MAC array 8 in a staggered fashion over several calculation cycles in order to properly align the elements as they flow through MAC array 8” and paragraph [0060]: “The 2.sup.nd cycle includes MAC Unit.sub.1, MAC Unit.sub.2, and MAC Unit.sub.5.”),
and the third multiplier is configured to perform multiplication operations on the new input operand from the first input register file (see, e.g., Liu paragraphs [0052-0054]: “and the direction of WO flow through MAC units 9 of MAC array 8 is from top to bottom. In other words, each MAC unit 9 receives AOs… and transmits WOs to the MAC unit 9 located below, and outputs the value of its dot product… MAC unit 9 includes register 91, register 92, multiplier circuit 93 coupled to registers 91 and 92, and accumulator circuit 94 coupled to multiplier circuit 93…Multiplier circuit 93 multiplies the data value, m.sub.a, provided by register 91 and the data value, m.sub.w, provided by register 92, and outputs the resulting data value or intermediate product, ip, to accumulator circuit 9” [i.e., a multiplier in MAC unit 9 functions as subsequent (third) multiplier in a series of multipliers] and paragraphs [0059-0061]: “With respect to MAC Unit.sub.1, element a.sub.1,1 from row 21 of matrix 20 is transmitted from the AO register (not shown for clarity) to register 91… Element a.sub.1,2 from row 21 of matrix 20 is transmitted from the AO register to register 91, element w.sub.2,1 from column 31 of matrix 30 is transmitted from the WO register to register 92, multiplier circuit 93 multiplies element a.sub.1,2 and element w.sub.2,1” [i.e., register 91 (input register file) within each MAC unit functions as a first input register file in the plurality of MAC units]),
and a third weight operand from a third weight register file at a third time, (see, e.g., Liu paragraph [0052]: “and the direction of WO flow through MAC units 9 of MAC array 8 is from top to bottom. In other words, each MAC unit 9 receives AOs from the MAC unit 9 to its left and transmits AOs to the MAC unit 9 to its right, receives WOs from the MAC unit 9 located above and transmits WOs to the MAC unit 9 located below, and outputs the value of its dot product”, paragraphs [0059-0061]: “With respect to MAC Unit.sub.1, element a.sub.1,1 from row 21 of matrix 20 is transmitted from the AO register (not shown for clarity) to register 91… Element a.sub.1,2 from row 21 of matrix 20 is transmitted from the AO register to register 91, element w.sub.2,1 from column 31 of matrix 30 is transmitted from the WO register to register 92, multiplier circuit 93 multiplies element a.sub.1,2 and element w.sub.2,1” [i.e., register 91 (input register file) within each MAC unit functions as a first input register file in the plurality of MAC units], and paragraph [0064]: “The 3.sup.rd cycle includes MAC Unit.sub.1, MAC Unit.sub.2, MAC Unit.sub.3, MAC Unit.sub.5, MAC Unit.sub.6, and MAC Unit.sub.9” [i.e., the third cycle for a given MAC unit functions as a third time the operand is multiplied]),
and the third time is after the second time (see, e.g., Liu paragraph [0056]: “Referring back to FIG. 3, the elements from matrices 20 and 30 enter MAC array 8 in a staggered fashion over several calculation cycles in order to properly align the elements as they flow through MAC array 8”, paragraph [0061]: “The 2.sup.nd cycle includes MAC Unit.sub.1, MAC Unit.sub.2, and MAC Unit.sub.5” and paragraph [0064]: “The 3.sup.rd cycle includes MAC Unit.sub.1, MAC Unit.sub.2, MAC Unit.sub.3, MAC Unit.sub.5, MAC Unit.sub.6, and MAC Unit.sub.9” [i.e., the third cycle for a given MAC unit functions as a third time the operand is multiplied after the 2nd cycle in paragraph [0060]]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ovsiannikov to incorporate the teachings of Liu to include a third multiplier configured to perform multiplication operations on a new input operand and a third weight operand at a third time, occurring after the second time. Doing so would have allowed Ovsiannikov to use Liu’s method for “efficiently multiplying matrices by performing vector multiply and accumulate (VMAC) operations within each calculation cycle”, as suggested by Liu (see, e.g., Liu, paragraph [0022]).
Regarding claim 19, as discussed above, Ovsiannikov discloses the method of claim 16.
Ovsiannikov further teaches further comprising: instructing, at a first time, a first multiplier in the PE to perform multiplication operations on a first input operand from a first input register file and a first weight operand from a first weight register file; (see, e.g., paragraph [0164]: “With 0xa0 . . . 0ax3 activations being broadcast and w.sub.row,col,a weights selected by MR units, multiplier 126 in each MR unit 103 proceeds to compute the product of activation act.sub.ow,a with W.sub.row,col,a: p.sub.row,col=w.sub.row,col,a*act.sub.row,a”).
Although Ovsiannikov substantially discloses the claimed invention, Ovsiannikov fails to explicitly teach the limitation and instructing, at a second time that is different from the first time, a second multiplier of the PE to perform multiplication operations on the first input operand from the first input register file and a second weight operand from a second weight register file.
In the same field, analogous art Liu teaches and instructing, at a second time that is different from the first time, a second multiplier of the PE to perform multiplication operations on the first input operand from the first input register file and a second weight operand from a second weight register file (see, e.g., Liu paragraph [0060-0062]: “The 2.sup.nd cycle includes MAC Unit.sub.1, MAC Unit.sub.2, and MAC Unit.sub.5… With respect to MAC Unit.sub.2, register 91 receives element a.sub.1,1 from MAC Unit.sub.1, register 92 receives element w.sub.1,2 from the WO register, multiplier circuit 93 multiplies element a.sub.1,1 and element w.sub.1,2” [i.e., the second multiplier MAC unit 2 performs multiplication operations from input from element received from register 91 (first input register file) and a weight element from the WO register 92 (second weight operand from second weight register file]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ovsiannikov to incorporate the teachings of Liu to sequentially instruct a first multiplier in a PE to multiply a first weight at a first time, then a second multiplier in the same PE to multiply the same first input operand with a second weight at an alternate second time. Doing so would have allowed Ovsiannikov to use Liu’s method for “efficiently multiplying matrices by performing vector multiply and accumulate (VMAC) operations within each calculation cycle”, as suggested by Liu (see, e.g., Liu, paragraph [0022]).
Regarding claim 20, as discussed above, Ovsiannikov in view of Liu teaches the method of claim 19. Although Ovsiannikov substantially discloses the claimed invention, Ovsiannikov fails to teach further comprising: instructing, at the first time, the second multiplier of the PE to perform multiplication operations on a second input operand and the second weight operand at the first time.
Nevertheless, in the same field, analogous art Liu teaches further comprising: instructing, at the first time, the second multiplier of the PE to perform multiplication operations on a second input operand and the second weight operand at the first time (see, e.g., Liu paragraphs [0141-0143]: “The second multiplier circuit 253 multiplies the data value, m.sub.a2, provided by the second AO row vector signal line, and the data value, m.sub.w2, provided by second WO column vector signal line, and outputs the resulting data value or intermediate product, ip.sub.2, to accumulator circuit 254… In this embodiment, VMAC.sup.4 unit 250 advantageously performs 4 MAC operations in a single VMAC processing cycle, which reduces hardware power consumption, hardware costs, processing latency, etc.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ovsiannikov to incorporate the teachings of Liu to include instructions to have a second multiplier of a PE to perform multiplication operations on a second input operand and second weight operand at a first time. Doing so would have allowed Ovsiannikov to use Liu’s method for “efficiently multiplying matrices by performing vector multiply and accumulate (VMAC) operations within each calculation cycle”, as suggested by Liu (see, e.g., Liu, paragraph [0022]).
Regarding claim 21, as discussed above, Ovsiannikov in view of Liu teaches the method of claim 19. Although Ovsiannikov substantially discloses the claimed invention, Ovsiannikov fails to teach further comprising: transferring, at the second time, a new input operand to the first input register file.
Nevertheless, in the same field, analogous art Liu teaches further comprising: transferring, at the second time, a new input operand to the first input register file (see, e.g., Liu paragraph [0060]: “The 2.sup.nd cycle includes MAC Unit.sub.1, MAC Unit.sub.2, and MAC Unit.sub.5” and paragraph [0063]: “With respect to MAC Unit.sub.5, register 91 receives element a.sub.2,1 from the AO register” [i.e., the register 91 (first input register file) of MAC Unit 5 receives a new input operand in the second cycle (at the second time)]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ovsiannikov to incorporate the teachings of Liu to transfer a new input operand to the first input register file at a second time. Doing so would have allowed Ovsiannikov to use Liu’s method for “efficiently multiplying matrices by performing vector multiply and accumulate (VMAC) operations within each calculation cycle”, as suggested by Liu (see, e.g., Liu, paragraph [0022]).
Regarding claim 22, as discussed above, Ovsiannikov in view of Liu teaches the method of claim 19. Although Ovsiannikov substantially discloses the claimed invention, Ovsiannikov fails to teach further comprising: instructing, at a third time that is after the second time,
a third multiplier to perform multiplication operations on the new input operand from the first input register file and a third weight operand from a third weight register file
Nevertheless, in the same field, analogous art Liu teaches further comprising: instructing, at a third time that is after the second time, (see, e.g., Liu paragraph [0064] “The 3.sup.rd cycle includes MAC Unit.sub.1, MAC Unit.sub.2, MAC Unit.sub.3, MAC Unit.sub.5, MAC Unit.sub.6, and MAC Unit.sub.9” [i.e., the third cycle for a given MAC unit functions as a third time the operand is multiplied]),
a third multiplier to perform multiplication operations on the new input operand from the first input register file and a third weight operand from a third weight register file (see, e.g., Liu paragraphs [0052-0054]: “and the direction of WO flow through MAC units 9 of MAC array 8 is from top to bottom. In other words, each MAC unit 9 receives AOs from the MAC unit 9 to its left and transmits AOs to the MAC unit 9 to its right, receives WOs from the MAC unit 9 located above and transmits WOs to the MAC unit 9 located below, and outputs the value of its dot product…MAC unit 9 includes register 91, register 92, multiplier circuit 93 coupled to registers 91 and 92, and accumulator circuit 94 coupled to multiplier circuit 93…Multiplier circuit 93 multiplies the data value, m.sub.a, provided by register 91 and the data value, m.sub.w, provided by register 92, and outputs the resulting data value or intermediate product, ip, to accumulator circuit 9” [i.e., a multiplier in MAC unit 9 functions as a third or subsequent multiplier in a series of multipliers] and paragraphs [0059-0061]: “element w.sub.1,1 from column 31 of matrix 30 is transmitted from the WO register (not shown for clarity) to register 92… element w.sub.2,1 from column 31 of matrix 30 is transmitted from the WO register to register 92, multiplier circuit 93 multiplies element a.sub.1,2 and element w.sub.2,1” [i.e., register 92 (weight register file) within each MAC unit functions as a third weight register file and register 91 (input register file) within each MAC unit functions as a first input register file in the plurality of MAC units]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ovsiannikov to incorporate the teachings of Liu to include instructions for a third multiplier to perform multiplication operations on a new input operand and third weight operand at a distinct third time. Doing so would have allowed Ovsiannikov to use Liu’s method for “efficiently multiplying matrices by performing vector multiply and accumulate (VMAC) operations within each calculation cycle”, as suggested by Liu (see, e.g., Liu, paragraph [0022]).
Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Ovsiannikov in view of Vantrease (US 11275997 B1; hereinafter Vantrease).
Regarding claim 15, as discussed above, Ovsiannikov discloses the apparatus of claim 1. Although Ovsiannikov substantially discloses the claimed invention, Ovsiannikov fails to explicitly teach wherein a weight register file of the plurality of weight register files is configured to bypass storing any weight operand at a time,
and another weight register file of the plurality of weight register files is configured to store a weight operand at the time.
Nevertheless, in the same field, analogous art Vantrease teaches wherein a weight register file of the plurality of weight register files is configured to bypass storing any weight operand at a time, (see, e.g., Vantrease Col. 15 lines 4-20: "At time T2…the assertion of weight load signal may also cause multiplexor 472 to provide a flush value (e.g., a zero value) at the inputs of weight pipeline registers 442a, 442b, 442c, and 442d. Upon each of these registers receiving a second clock edge of a second clock cycle during time T2, weight pipeline registers 442a, 442b, 442c, and 442d may store a zero, and weight load registers 444c and 444d (of PEs 13 and 14) may also latch in a zero from the output of, respectively, weight pipeline register 442b and 442c" [i.e., a weight pipeline register (weight register file) stores flush values bypassing storage of weights at time T2]),
and another weight register file of the plurality of weight register files is configured to store a weight operand at the time (see, e.g., Vantrease Col. 15 lines 4-20: "At time T2, after the first clock edge at T1 (and after weight pipeline register 442a stores weight W1), weight load may be asserted, and weight load register 444a of PE 11 may latch in W0, whereas weight load register 444b of PE 12 may latch in W1 provided by weight pipeline register 442a of PE 11" [i.e., at time T2 a different weight load register 444a (another weight register file) latches in W0 (stores a weight operand)]).
Ovsiannikov and Vantrease are analogous art because they are both directed to hardware implementation of neural networks (see, e.g., Ovsiannikov paragraph [0203], Vantrease Background).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Ovsiannikov to incorporate the teachings of Vantrease to configure certain weight register files to not store weight operands at a given time, and other weight register files to store weight operands at the same time. Doing so would have allowed Ovsiannikov to use Vantrease’s method in order to “shorten the time between pre-fetching of weights for the PEs and can improve the performance of computation engine”, as suggested by Vantrease (see, e.g., Vantrease, Cols. 12-13 lines 67, 1-2).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Kamran Afshar whose telephone number is (571)272-7796. The examiner can normally be reached Mon-Fri 7:30-5:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KAMRAN AFSHAR/Supervisory Patent Examiner, Art Unit 2125
1 As discussed above in the section 112(f) interpretation of this claim, the “input register file” has been
interpreted as the specified combination of software and/or hardware disclosed in the specification for this specific component that is capable of performing the claimed functions
2 As discussed above in the section 112(f) interpretation of this claim, the “weight register file” has been
interpreted as the specified combination of software and/or hardware disclosed in the specification for this specific component that is capable of performing the claimed functions
3 As discussed above in the section 112(f) interpretation of this claim, the “multiplier” has been
interpreted as the specified combination of software and/or hardware disclosed in the specification for this specific component that is capable of performing the claimed functions
4 As discussed above in the section 112(f) interpretation of this claim, the “adder assembly” has been
interpreted as the specified combination of software and/or hardware disclosed in the specification for this specific component that is capable of performing the claimed functions
As discussed above in the section 112(f) interpretation of this claim, the “output register file” has been
interpreted as the specified combination of software and/or hardware disclosed in the specification for this specific component that is capable of performing the claimed functions
6 As discussed above in the section 112(f) interpretation of this claim, the “adder” has been
interpreted as the specified combination of software and/or hardware disclosed in the specification for this specific component that is capable of performing the claimed functions
7 As discussed above in the section 112(f) interpretation of this claim, the “processing elements” has been
interpreted as the specified combination of software and/or hardware disclosed in the specification for this specific component that is capable of performing the claimed functions