DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This Action is Final and is in response to the claims filed 11/13/2025. Claims 1-25 are currently pending, of which claims 1-25 are currently rejected.
Response to Arguments
Applicant’s arguments filed 11/13/2025 have been fully considered.
35 U.S.C. 112 – Applicant’s arguments regarding the 35 U.S.C. 112 (a) and (b) rejections based on the limitation “concatenating module” invoking 35 U.S.C. 112(f) have been fully considered, but they are not persuasive.
Applicant argues in page 10 that the phrase “concatenating module” falls under 112(f) and meets the requirements of 35 U.S.C. 112(a) and (b). Specifically, Applicant argues “During the interview, the examiner indicated that all the § 112 issues related to the phrase "concatenating module" falls under 35 U.S.C. § 112(f) and that the phrase "concatenating module" meets the requirements under 35 U.S.C. § 112(a) and (b). Accordingly, withdrawal of the § 112 rejections is respectfully requested.”
Examiner respectfully disagrees. Regarding the 35 U.S.C. 112(a) and (b) rejections, Examiner disagrees with Applicants’ assertion that the USPTO agreed to withdraw the rejection during the interview. See Examiner’s interview summary mailed on 11/18/2025. Applicant did not provide any reasoning as to why the rejections should be withdrawn. Still, after reconsideration, the rejection is maintained.
35 U.S.C. 103 – Applicant’s arguments regarding the 35 U.S.C. 103 rejection have been considered, but they are not persuasive.
Applicant argues at the bottom of page 10 and in page 11 that amended claim 1 is not taught by the cited references. Applicant specifically argues “The combination of the cited references does not teach or render such a processing element. Therefore, amended claim 1 is patentable in light of the cited references. The other independent claims are amended to recite similar claim limitations and therefore, are also patentable in light of the cited references.”
Examiner respectfully disagrees. Applicant's arguments fail to comply with 37 CFR 1.111(b) because they amount to a general allegation that the claims define a patentable invention without specifically pointing out how the language of the claims patentably distinguishes them from the references. Applicant amended claim 1 to include an adder configured to perform a first sum in a first accumulation cycle, and a second sum and normalization of an exponent in a second accumulation cycle, and does not point out how the cited prior art does not teach amended claim 1. See new grounds of rejection below necessitated by amendments.
Drawings
The drawings are objected to under 37 CFR 1.83(a). The drawings must show every feature of the invention specified in the claims. Therefore, the “adder configured to: compute a first sum from the individual partial sum of the processing element in a first accumulation cycle, compute a second sum in a second accumulation cycle, and normalize an exponent of the first sum in the second accumulation cycle” first recited in claim 1 must be shown or the feature(s) canceled from the claim(s). No new matter should be entered.
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
“a concatenating module”
First recited in Claim 1.
The term “module” has been interpreted as a generic placeholder. See MPEP 2181.I.A.
Furthermore, this module is modified by functional language, not modified by structure or acts for performing the claimed function.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
The “concatenating module” is included in the DNN accelerator and in individual PEs divided into portions as in Figs. 3 and 5A and [0044] [0080], and coupled to storage units and register files, as further disclosed in [0079].
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-25 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim 1 recites the limitation “and an adder configured to: compute a first sum from the individual partial sum of the processing element in a first accumulation cycle, compute a second sum in a second accumulation cycle, and normalize an exponent of the first sum in the second accumulation cycle.” There is no indication in the original disclosure describing how a second sum is computed and the exponent of the first sum is normalized both in a second accumulation cycle. As best understood by Examiner, paragraph 0075 appears to be pertinent to this amendment, however it does not describe how a second sum is computed in the same second accumulation cycle as a normalization of exponent of a first sum.
Claims 2-10 inherit the same deficiency as Claim 1 by reason of dependence.
Claims 11 and 21 recite similar limitations as Claim 1, and are rejected for the same reasons therein.
Claims 12-20 and 22-25 inherit the same deficiency as Claims 11 and 21, respectively, by reason of dependence.
Additionally, Claim 1 recites “concatenation module”. Claims 2-10 recite the same limitation by reason of dependence. This limitation invokes 35 U.S.C 112 (f) or pre-AIA 35 U.S.C. 112, sixth paragraph. However, the written description fails to provide adequate written description of the corresponding structure, material, or acts for performing the entire claimed functions of this limitation. See rejection under 35 U.S.C 112 (b) below for further details as to the requirement for the written description.
Claims 2-10 inherit the same deficiency as Claim 1 by reason of dependence.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 7 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 7 recites the limitations “the first path” and “the second path”. There is insufficient antecedent basis for this limitation in the claim.
Claim limitation “concatenating module” invokes 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. However, the written description fails to disclose the corresponding structure, material, or acts for performing the entire claimed function and to clearly link the structure, material, or acts to the function.
As to the “concatenating module”, this module is discussed at paragraphs [0022], [0023], [0051], [0079], [0080] of the Specification. These paragraphs appear to merely restate the functional limitations set forth in the claims. The cited paragraphs do not describe an algorithm or particular structure. No algorithm or structure could be found in other parts of the specification.
Therefore, the claim is indefinite and is rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph. Claims 2-10 inherit the same deficiency as Claim 1 by reason of dependence.
Applicant may:
(a) Amend the claim so that the claim limitation will no longer be interpreted as a limitation under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph;
(b) Amend the written description of the specification such that it expressly recites what structure, material, or acts perform the entire claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(c) Amend the written description of the specification such that it clearly links the structure, material, or acts disclosed therein to the function recited in the claim, without introducing any new matter (35 U.S.C. 132(a)).
If applicant is of the opinion that the written description of the specification already implicitly or inherently discloses the corresponding structure, material, or acts and clearly links them to the function so that one of ordinary skill in the art would recognize what structure, material, or acts perform the claimed function, applicant should clarify the record by either:
(a) Amending the written description of the specification such that it expressly recites the corresponding structure, material, or acts for performing the claimed function and clearly links or associates the structure, material, or acts to the claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(b) Stating on the record what the corresponding structure, material, or acts, which are implicitly or inherently set forth in the written description of the specification, perform the claimed function. For more information, see 37 CFR 1.75(d) and MPEP §§ 608.01(o) and 2181.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 5-9, 11-13, 15, 17, 21, and 22 are rejected under 35 U.S.C. 103 as being unpatentable over Huang (U.S. Patent Application Publication No.: US 20200410337 A1), hereinafter “Huang”, in view of Raha et al. (U.S. Patent Application Publication No.: US 20210397414 A1), hereinafter “Raha”, further in view of A. Beaumont-Smith in NPL: Reduced Latency IEEE Floating-Point Standard Adder Architectures (https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=762826), hereinafter “Beaumont-Smith”.
With regards to Claim 1, Huang teaches:
An apparatus for a deep neural network (DNN) includes (¶0038):
… a floating-point multiply-accumulate (FPMAC) operation (Figs. 7 and 8, e.g., show PE array; ¶0100, e.g., PEs perform floating point MAC operations (FPMAC)), …
a processing element (Fig. 7, e.g., shows one processing element 711 in a processing element array 710) comprising:
a register file configured to store the input operand by storing the plurality of bits in a sequence (¶0102, e.g., Each PE may store inputs (input operand) in sequential logic registers (register file); Fig. 7, e.g., shows flip-flops (registers) to store input data; ¶0100, e.g., PEs perform floating point operations (plurality of bits)),
a multiplier configured to perform a series of multiplication operations with the plurality of bits based on the sequence (¶0036, e.g., Each processing node receives sequential stream of data (plurality of bits based on a sequence); ¶0101, e.g., Multiplier would handle one input data element at a time (performing series of multiplication operations); Fig. 8, e.g., multiplier 823), wherein each multiplication operation is with one of the plurality of bits (¶0036, e.g., Each processing node receives sequential stream of data (plurality of bits based on a sequence); ¶0101, e.g., Multiplier would handle one input data element (one of the plurality of bits) at a time), and
… perform an accumulation operation (¶0100, e.g., PE can perform multiply and accumulate operations) … and to generate an individual partial sum of the processing element (Fig. 8, e.g., shows output from multiplier (individual partial sum) inputted to the adder).
an adder configured to:
compute a first sum from the individual partial sum of the processing element … (Fig. 8, e.g., Adder 825 adds multiplier output (individual partial sum)),
Huang does not teach:
a concatenating module configured to generate an input operand … , the input operand including a plurality of bits, wherein a first portion of the input operand is stored in a first storage unit of a memory, and a second portion of the input operand is stored in a second storage unit of the memory; and
an accumulator configured to perform an accumulation operation with results of the series of multiplication operations …; and
an adder configured to:
compute a first sum from the individual partial sum of the processing element in a first accumulation cycle,
compute a second sum in a second accumulation cycle, and
normalize an exponent of the first sum in the second accumulation cycle.
However, Raha teaches:
a concatenating module configured to generate an input operand (Fig. 21, e.g., shows concatenating 231 (concatenating module). Inputs (input operand) are fed to multiplier) … , the input operand including a plurality of bits, wherein a first portion of the input operand is stored in a first storage unit of a memory, and a second portion of the input operand is stored in a second storage unit of the memory (Fig. 21, e.g., each portion of input operands includes a plurality of bits and are stored in Subbanks SB0 and SB1 (first and second storage unit));
Therefore, it would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to which said subject matter pertains to combine the concatenating module and register subbanks as taught by Raha with the Processing Element as taught by Huang. One would have been motivated to combine these references because both references disclose sequential FPMAC operations, and Raha enhances the model of Huang by providing "for a DNN processing engine (PE) that can support MAC operations of different precisions, (INT8/4/2/1 to FP16/BF16) while using a common low overhead sparsity acceleration logic." ¶0038
Raha also teaches:
an accumulator configured to perform an accumulation operation with results of the series of multiplication operations (Fig. 21, e.g., shows accumulator doing accumulation of multiplication operations; ¶0076, e.g., Fig. 21 shows floating point execution (including accumulation) of a single PE (individual partial sum)) …
Therefore, It would have also been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to which said subject matter pertains to combine the accumulator as taught by Raha with the Processing Element as taught by Huang. One would have been motivated to combine these references because both references disclose sequential FPMAC operations, and Raha enhances the model of Huang by allowing for sequential multiplication results to be accumulated to form final multiplication result.
Huang in view of Raha do not teach:
an adder configured to:
compute a first sum from the individual partial sum of the processing element in a first accumulation cycle,
compute a second sum in a second accumulation cycle, and
normalize an exponent of the first sum in the second accumulation cycle.
However, in the same field of endeavor, Beaumont-Smith teaches a 3-cycle Floating Point Adder that uses 3 pipelining stages. Beaumont-Smith explains “the first stage is an exponent subtraction, significand swap, unpacking and alignment shift, the second stage performs the significand addition and the third stage performs the post normalisation shift.” See Beaumont-Smith, second page, Section 2. “A 3-cycle Floating Point Adder using a Flagged Prefix Integer Adder”. Since the 3-cycle Floating Point Adder uses a pipelining method, and has a second stage that performs significand addition and a third stage that performs post normalization shift (normalizing), one ordinarily skilled in the art could determine that the result of the second stage (first sum) of a first cycle would be normalized, in a second cycle, in the third pipelining stage while the second stage performs another summation operation (second sum).
Therefore, it would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to which said subject matter pertains to combine the 3-cycle Floating Point Adder as taught by Beaumont-Smith with the adder 825 as taught by Huang in view of Raha. One would have been motivated to combine these references because both references disclose floating point accumulation operations, and Beaumont-Smith enhances the model of Huang in view of Raha because the adder would use “minimal hardware for use in a parallel processor with multiple FPUs” (Beaumont-Smith: 1. Introduction), and would allow for normalization of floating point values after a summation operation.
With regards to Claim 2, Huang in view of Raha in view of Beaumont-Smith teach:
The apparatus of claim 1, wherein the processing element further comprises an adder that is configured to compute the first sum by performing an accumulation operation with the individual partial sum of the processing element and an output of an additional processing element (Huang: Fig.8, e.g., Added 825 (Adder); ¶0101, e.g., Added 825 may add (accumulation operation) weighted input (individual partial sum) to the partial weighted sum (output of an additional processing element) and generates updated partial weighted sum (partial sum))
and to generate an internal partial sum, and the processing element and the additional processing element are arranged in a same PE array (Huang: Fig. 7, e.g., shows output p_out (internal partial sum). PEs are arranged in the same PE array).
With regards to Claim 3, Huang in view of Raha in view of Beaumont-Smith teach:
The apparatus of claim 2, wherein the adder is further configured to perform an accumulation operation with the internal partial sum and an external partial sum, and the external partial sum is an output of a different PE array (Huang: Fig. 8, e.g., receives data from buses 822 and 824 (external partial sum). Accumulator accumulates output of multiplier (input partial sum) and data from bus 824; ¶0100, e.g., PEs may receive inputs through input bus 824 from external circuitry (different PE array)).
With regards to Claim 5, Huang in view of Raha in view of Beaumont-Smith teach:
The apparatus of claim 1, wherein the multiplier and the accumulator are in an electronic circuit that is configured to perform the series of multiplication operations and the accumulation operation (Huang: ¶0089, e.g., Each PE includes multiplier circuit (electronic circuit); Raha: Fig. 21, e.g., multiplier includes an accumulator).
The motivation to combine provided with respect to claim 1 applies equally to claim 5.
With regards to Claim 6, Huang in view of Raha in view of Beaumont-Smith teach:
The apparatus of claim 5, wherein the electronic circuit is configured to accumulate a first floating-point number with a second floating-point number (Raha: ¶0076, e.g., Fig. 21 shows floating point execution (including accumulation) of floating point numbers (first and second floating point numbers)) by:
changing a first exponent in the first floating-point number based on a second exponent in the second floating-point number before determining whether the second exponent is larger than the first exponent (Beaumont-Smith: Introduction Page 1, second column first bullet point, e.g., additions can be performed using extra adders to compute speculative results for exponent subtraction (Ea - Eb and Eb - Ea) from first and second floating point numbers; Introduction Pages 1 and 2, second column (page 1) and first column (page 2) second bullet point, e.g., shifting of significands can be done in parallel with the exponent difference computation).
The motivation to combine provided with respect to claim 1 applies equally to claim 6.
With regards to Claim 7, Huang in view of Raha in view of Beaumont-Smith teach:
The apparatus of claim 5, wherein:
the electronic circuit is configured to accumulate a first floating-point number with a second floating-point number (Huang: ¶0089, e.g., Each PE includes multiplier-accumulator circuit (electronic circuit)),
the first path in the electronic circuit is configured to adjust a first exponent in the first floating-point number based on a second exponent in the second floating-point number (Beaumont-Smith: Third page, Fig. 1, e.g., path to the right (first path) receiving inputs Ea and Eb (first and second exponents)),
the second path in the electronic circuit is configured to remove one or more bits having zero values from the first floating-point number or the second floating-point number (Beaumont-Smith: Third page, Fig. 1, e.g., path to the left (second path) aligns significand by shifting (removes one or more bits having zero values; First and second page, second bullet point.)), and
the second path is separate from the first path (Beaumont-Smith: Third page, Fig. 1, e.g., shows path to the left (receiving mantissas) and path to the right (receiving exponents)).
The motivation to combine provided with respect to claim 1 applies equally to claim 7.
With regards to Claim 8, Huang in view of Raha in view of Beaumont-Smith teach:
The apparatus of claim 1, wherein:
the concatenating module is further configured to generate a weight operand of the FPMAC operation (Raha: Fig. 21, e.g., each portion of weight operands (first and second portions) is stored in Subbanks SB0 and SB1 (first and second storage unit) of Register File RF (memory) respectively),
the weight operand including a plurality of bits (Raha: Fig. 21, e.g., shows plurality of bits for both weight and input operands),
a first portion of the weight operand is stored in a second storage unit of the memory,
a second portion of the weight operand is stored in a third storage unit of the memory (Raha: Fig. 21, e.g., shows both storage units IF RF SB0 and IF RF SB1 to store first and second portions of weight operands),
and a multiplication operation in the series is with a bit of the input operand and a bit of the weight operand (Huang: Fig. 7, e.g., weights and inputs are inputted to multiplier).
The motivation to combine provided with respect to claim 1 applies equally to claim 8.
With regards to Claim 9, Huang in view of Raha in view of Beaumont-Smith teach:
The apparatus of claim 1, wherein the first portion and the second portion include same bits arranged in a same sequence (Raha: Fig. 21, e.g., sequence of bit numbers are identical).
The motivation to combine provided with respect to claim 1 applies equally to claim 9.
With regards to Claim 11, Huang teaches:
A method, comprising:
transferring [an input operand] of a floating-point multiply-accumulate (FPMAC) operation (¶0100, e.g., PEs perform floating point MAC operations (FPMAC operation)) … to an input register file in a processing element, bits in the input operand having a first sequence in the input register file (¶0102, e.g., Each PE may store inputs (input operand) in sequential logic (first sequence) registers (register file); Fig. 7, e.g., shows flip-flops (registers) to store input data i (input register file));
transferring [a weight operand] of the FPMAC operation (¶0100, e.g., PEs perform floating point MAC operations (FPMAC operation)) … to a weight register file in the processing element, bits in the weight operand having a second sequence in the input register file (¶0102, e.g., Each PE may store inputs (weight operand) in sequential logic (second sequence) registers (register file); Fig. 7, e.g., shows flip-flops (registers) to store weight data w (input register file));
feeding an FPMAC unit of the processing element with the bits in the input operand and the bits in the weight operand based on the first sequence and the second sequence (Fig. 7, e.g., shows multiplier and adder (FPMAC unit) receiving input operand i and weight operand w and flip-flops (which act as registers) to store input data; ¶0102, e.g., Each PE may store inputs (input operand) in sequential logic (first and second sequence) registers); and
performing, by the FPMAC unit, the FPMAC operation based on the input operand and the weight operand to generate an individual partial sum of the processing element (Fig. 8, e.g., shows multiplier and adder (FPMAC unit); ¶0101, e.g., Multiplier 823 may generate multiplication product and Added 825 may add weighted input (results of series of multiplication operations) (multiplier and adder together perform FPMAC operation) and generates partial weighted sum (individual partial sum));
computing, by an adder coupled with the FPMAC unit, a first sum from the individual partial sum of the processing element (Fig. 8, e.g., Adder 825 adds multiplier output (individual partial sum)) …
Huang does not teach:
transferring a first portion of an input operand of a floating-point multiply-accumulate (FPMAC) operation and a second portion of the input operand from two separate storage units of a memory …
transferring a first portion of a weight operand of the FPMAC operation and a second portion of the weight operand from two other separate storage units of the memory …
computing, by an adder coupled with the FPMAC unit, a first sum from the individual partial sum of the processing element in a first accumulation cycle;
computing, by the adder, a second sum in a second accumulation cycle; and
normalizing, by the adder, an exponent of the first sum in the second accumulation cycle.
However, Raha teaches:
transferring a first portion of an input operand of a floating-point multiply-accumulate (FPMAC) operation and a second portion of the input operand from two separate storage units of a memory (Fig. 21, e.g., each portion of input operands (first and second portions) is stored in Subbanks SB0 and SB1 (first and second storage unit) of Register File RF (memory) respectively; ¶0076, e.g., subbanks (storage units) store each bitmap (first and second portions)) …
transferring a first portion of a weight operand of the FPMAC operation and a second portion of the weight operand from two other separate storage units of the memory (Fig. 21, e.g., each portion of weight operands (first and second portions) is stored in Subbanks SB0 and SB1 (first and second storage unit) of Register File RF (memory) respectively; ¶0076, e.g., subbanks (storage units) store each bitmap (first and second portions)) …
Therefore, it would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to which said subject matter pertains to combine the concatenating module and register subbanks as taught by Raha with the Processing Element as taught by Huang. One would have been motivated to combine these references because both references disclose sequential FPMAC operations, and Raha enhances the model of Huang by providing "for a DNN processing engine (PE) that can support MAC operations of different precisions, (INT8/4/2/1 to FP16/BF16) while using a common low overhead sparsity acceleration logic." ¶0038
Huang in view of Raha do not teach:
computing, by an adder coupled with the FPMAC unit, a first sum from the individual partial sum of the processing element in a first accumulation cycle;
computing, by the adder, a second sum in a second accumulation cycle; and
normalizing, by the adder, an exponent of the first sum in the second accumulation cycle.
However, in the same field of endeavor, Beaumont-Smith teaches a 3-cycle Floating Point Adder that uses 3 pipelining stages. Beaumont-Smith explains “the first stage is an exponent subtraction, significand swap, unpacking and alignment shift, the second stage performs the significand addition and the third stage performs the post normalisation shift.” See Beaumont-Smith, second page, Section 2. “A 3-cycle Floating Point Adder using a Flagged Prefix Integer Adder”. Since the 3-cycle Floating Point Adder uses a pipelining method, and has a second stage that performs significand addition and a third stage that performs post normalization shift (normalizing), one ordinarily skilled in the art could determine that the result of the second stage (first sum) of a first cycle would be normalized, in a second cycle, in the third pipelining stage while the second stage performs another summation operation (second sum).
Therefore, it would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to which said subject matter pertains to combine the 3-cycle Floating Point Adder as taught by Beaumont-Smith with the adder 825 as taught by Huang in view of Raha. One would have been motivated to combine these references because both references disclose floating point accumulation operations, and Beaumont-Smith enhances the model of Huang in view of Raha because the adder would use “minimal hardware for use in a parallel processor with multiple FPUs” (Beaumont-Smith: 1. Introduction), and would allow for normalization of floating point values after a summation operation.
With regards to Claim 12, Huang in view of Raha in view of Beaumont-Smith teach:
The method of claim 11, wherein the first portion of the input operand includes a first half of the bits in the first sequence, and the second portion of the input operand includes a second half of the bits in the first sequence (Raha: ¶0076; Fig. 21, e.g., shows two 8 bit input sequencies (first and second halves of the input operand) to be concatenated to for a 16 bit value).
The motivation to combine provided with respect to claim 11 applies equally to claim 12.
With regards to Claim 13, Huang in view of Raha in view of Beaumont-Smith teach:
The method of claim 11, wherein the first portion of the weight operand includes a first half of the bits in the second sequence, and the second portion of the weight operand includes a second half of the bits in the second sequence (Raha: ¶0076, e.g., Fig. 21, e.g., shows two 8 bit weight sequencies (first and second halves of the weight operand) to be concatenated to for a 16 bit value).
The motivation to combine provided with respect to claim 11 applies equally to claim 13.
With regards to Claim 15, Huang in view of Raha in view of Beaumont-Smith teach:
The method of claim 11, wherein computing the first sum comprises:
generating an internal partial sum by accumulating the individual partial sum with an output of another processing element (Huang: Fig. 7, e.g., shows output p_out (internal partial sum). Receives p_in (output of another processing element); ¶0090).
With regards to Claim 17, Huang in view of Raha in view of Beaumont-Smith teach:
The method of claim 11, wherein the first portion and the second portion of the input operand have same bits arranged in a same sequence (Raha: Fig. 21, e.g., sequence of bit numbers are identical).
The motivation to combine provided with respect to claim 11 applies equally to claim 17.
With regards to Claims 21 and 22, they are media versions of the claimed method above (claims 11 and 12 respectively), wherein all claim limitations also have been addressed and/or covered in cited areas. Thus, accordingly, these claims are rejected for at least the same reasons therein.
Claims 4, 14, 16, and 23 are rejected under 35 U.S.C. 103 as being unpatentable over Huang, in view of Raha, in view of Beaumont-Smith, further in view of Shibo Wang in NPL: BFloat16: The secret to high performance on Cloud TPUs (https://cloud.google.com/blog/products/ai-machine-learning/bfloat16-the-secret-to-high-performance-on-cloud-tpus), hereinafter “Wang”.
With regards to Claim 4, Huang in view of Raha in view of Beaumont-Smith teach the apparatus of claim 2. Huang in view of Raha in view of Beaumont-Smith do not teach:
wherein a bit precision of the results of the series of multiplication operations is different from a bit precision of the internal partial sum.
However, Wang teaches:
wherein a bit precision of the results of the series of multiplication operations is different from a bit precision of the internal partial sum (Section “Choosing bfloat16”, Third paragraph, e.g., Multiplication operations are performed in bfloat 16, while accumulations are performed in FP32).
Therefore, it would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to which said subject matter pertains to combine configuration of using different precisions in the multiplication and accumulation operations as taught by Wang with the Processing Element array as taught by Huang in view of Raha in view of Beaumont-Smith. One would have been motivated to combine these references because both references disclose PE arrays for MAC operations, and Wang enhances the model of Huang in view of Raha in view of Beaumont-Smith by allowing accumulations to be represented using a larger precision for better accuracy.
With regards to Claim 14, Huang in view of Raha in view of Beaumont-Smith teach the method of claim 11. Huang in view of Raha in view of Beaumont-Smith do not teach:
wherein the individual partial sum has a different bit precision from the input operand or the weight operand.
However, Wang teaches:
wherein the individual partial sum has a different bit precision from the input operand or the weight operand (Section “Choosing bfloat16”, Third paragraph, e.g., Multiplication operations are performed in bfloat 16, while accumulations are performed in FP32).
Therefore, it would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to which said subject matter pertains to combine configuration of using different precisions in the multiplication and accumulation operations as taught by Wang with the Processing Element array as taught by Huang in view of Raha in view of Beaumont-Smith. One would have been motivated to combine these references because both references disclose PE arrays for MAC operations, and Wang enhances the model of Huang in view of Raha in view of Beaumont-Smith by allowing accumulations to be represented using a larger precision for better accuracy.
With regards to Claim 16, Huang in view of Raha in view of Beaumont-Smith teach the method of claim 15. Huang in view of Raha in view of Beaumont-Smith do not teach:
wherein the internal partial sum has a same bit precision as the individual partial sum.
However, Wang teaches:
wherein the internal partial sum has a same bit precision as the individual partial sum (Wang: “Choosing bfloat16”, Third paragraph, e.g., Accumulation operations are performed in FP32).
Therefore, it would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to which said subject matter pertains to combine configuration of using the same precision for accumulation operations as taught by Wang with the Processing Element array as taught by Huang in view of Raha in view of Beaumont-Smith. One would have been motivated to combine these references because both references disclose PE arrays for MAC operations, and Wang enhances the model of Huang in view of Raha in view of Beaumont-Smith by allowing accumulations to be represented using a larger precision for better accuracy. Combination would result in the internal partial sum and the individual partial sum to have the same bit precision; hence this combination teaches claim 16 in its entirety.
With regards to Claim 23, it is a media version of the claimed method above (claim 14), wherein all claim limitations also have been addressed and/or covered in cited areas. Thus, accordingly, this claim is rejected for at least the same reasons therein.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Huang, in view of Raha, in view of Beaumont-Smith, further in view of Volpe (U.S. Patent No.: US 11113233 B1), hereinafter “Volpe”, and further in view of Hardik Sharma in NPL: Bit Fusion: Bit-Level Dynamically Composable Architecture for Accelerating Deep Neural Networks. (https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8416871&tag=1), hereinafter “Sharma”.
With regards to Claim 10, Huang in view of Raha in view of Beaumont-Smith teach:
The apparatus of claim 1, wherein:
the input operand is a first input operand, the multiplier and the accumulator constitute a first FPMAC unit of the processing element (Huang: Fig. 7, e.g., shows input operand i, and multiplier and accumulator (first FPMAC unit) inside the PE),
… a second FPMAC unit, a third FPMAC unit, and a fourth FPMAC unit (Huang, e.g., Fig. 8, e.g., shows PEs 820b (second FPMAC unit), PE next to 820a (third FPMAC unit), and PE next to 820b (fourth FPMAC unit)),
the first FPMAC unit is configured to perform an FPMAC operation …
the second FPMAC unit is configured to perform an FPMAC operation …
the third FPMAC unit is configured to perform an FPMAC operation …
the fourth FPMAC unit is configured to perform an FPMAC operation … (Huang: ¶0100, e.g., PEs perform floating point MAC operations (FPMAC))
Huang in view of Raha in view of Beaumont-Smith do not explicitly teach:
the processing element further comprises a second FPMAC unit, a third FPMAC unit, and a fourth FPMAC unit,
the first FPMAC unit is configured to perform an FPMAC operation with the first input operand and a first weight operand,
the second FPMAC unit is configured to perform an FPMAC operation with the first input operand and a second weight operand,
the third FPMAC unit is configured to perform an FPMAC operation with a second input operand and the first weight operand, and
the fourth FPMAC unit is configured to perform an FPMAC operation with the second input operand and the second weight operand.
However, Volpe teaches:
the first FPMAC unit is configured to perform an FPMAC operation with the first input operand and a first weight operand (Fig. 1E, e.g., PE 162a receives weight set 1 (first weight operand) and input data element (first input operand)),
the second FPMAC unit is configured to perform an FPMAC operation with the first input operand and a second weight operand (Fig. 1E, e.g., PE 162b receives weight set 2 (second weight operand) and input data element (first input operand)),
the third FPMAC unit is configured to perform an FPMAC operation with a second input operand and the first weight operand (Fig. 1E, e.g., PE below 162a receives weight set 1 (first weight operand) and input data element (second input operand); Column 3 Lines 5-8, e.g., Each row corresponds to a specific input data element), and
the fourth FPMAC unit is configured to perform an FPMAC operation with the second input operand and the second weight operand (Fig. 1E, e.g., PE below 162b receives weight set 2 (second weight operand) and input data element (second input operand); Column 3 Lines 5-8, e.g., Each row corresponds to a specific input data element).
Therefore, it would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to which said subject matter pertains to combine the PEs performing MAC operations using two operands from input and weight data as taught by Volpe with Processing Element array as taught by Huang in view of Raha in view of Beaumont-Smith. One would have been motivated to combine these references because both references disclose floating point MAC operations in PE arrays, and Volpe enhances the model of Huang in view of Raha in view of Beaumont-Smith by allowing for FPMAC operations to be carried out through the array.
Huang in view of Raha in view of Beaumont-Smith in view of Volpe do not teach:
the processing element further comprises a second FPMAC unit, a third FPMAC unit, and a fourth FPMAC unit,
However, in the same field of endeavor, Sharma teaches how PEs can be fused into a single Fused Processing Engine (F-PE). Sharma explains: “As such, Bit Fusion is a collection of bit-level computational elements, called BitBricks, that dynamically compose to logically construct Fused Processing Engines (Fused-PE) that execute DNN operations with the required bitwidth”. See Section II. Bit Fusion Architecture, First Paragraph. Combining this method with the PE array as taught by Huang in view of Raha in view of Volpe would cause for the PEs (where each PE includes a FPMAC unit) to be fused into one F-PE.
Therefore, it would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to which said subject matter pertains to combine PEs fused together to form one fused PE as taught by Sharma with the Processing Element array as taught by Huang in view of Raha in view of Beaumont-Smith in view of Volpe. One would have been motivated to combine these references because both references disclose floating point MAC operations, and Sharma enhances the model of Huang in view of Raha in view of Beaumont-Smith in view of Volpe because "Fused-PEs provide bit-level flexibility for multiply-adds" (Sharma: Section II. Bit Fusion Architecture, First Paragraph).
Claims 18-20 and 24-25 are rejected under 35 U.S.C. 103 as being unpatentable over Huang, in view of Raha, in view of Beaumont-Smith, further in view of Volpe, and further in view of Das Sarma (U.S. Patent Application Publication No.: US 20200349216 A1), hereinafter “Das Sarma”.
With regards to Claim 18, Huang in view of Raha in view of Beaumont-Smith teach:
The method of claim 11, …
the method further comprises performing FPMAC operations based on the input matrix and the weight matrix (Huang: ¶0044).
Huang in view of Raha in view of Beaumont-Smith do not explicitly teach:
the input operand is a first input operand of an input matrix, the input matrix further comprising a second input operand,
the weight operand is a first weight operand of a weight matrix, the weight matrix further comprising a second weight operand, and
the method further comprises performing FPMAC operations based on the input matrix and the weight matrix.
However, Das Sarma teaches:
the input operand is a first input operand of an input matrix …
the weight operand is a first weight operand of a weight matrix (¶0045, e.g., Each cell (processing element) receives input operands from two input matrices (input and weight matrix)) …
Therefore, it would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to which said subject matter pertains to combine the input and weight operands to come from an input and weight matrix respectively as taught by Das Sarma with the Processing Element array as taught by Huang in view of Raha in view of Beaumont-Smith. One would have been motivated to combine these references because both references disclose PE arrays for FPMAC operations, and Das Sarma enhances the model of Huang in view of Raha in view of Beaumont-Smith by allowing for matrix-to-matrix multiplication.
Huang in view of Raha in view of Das Sarma do not explicitly teach:
the input matrix further comprising a second input operand,
the weight matrix further comprising a second weight operand,
However, Volpe teaches:
the input matrix further comprising a second input operand (Fig. 1E, e.g., shows first and second input data elements corresponding to first and second rows respectively),
the weight matrix further comprising a second weight operand (Fig. 1E, e.g., shows first and second input data elements corresponding to first and second rows respectively),
Therefore, it would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to which said subject matter pertains to combine the PEs performing MAC operations using two operands from input and weight data as taught by Volpe with Processing Element array as taught by Huang in view of Raha in view of Beaumont-Smith in view of Das Sarma. One would have been motivated to combine these references because both references disclose floating point MAC operations in PE arrays, and Volpe enhances the model of Huang in view of Raha in view of Beaumont-Smith in view of Das Sarma by allowing for FPMAC operations to be carried out through the array.
With regards to Claim 19, Huang in view of Raha in view of Beaumont-Smith in view of Das Sarma in view of Volpe teach:
The method of claim 18, wherein the FPMAC unit is a first FPMAC unit of the processing element, the FPMAC operation is a first FPMAC operation of the FPMAC operations (Huang: Fig. 7, e.g., shows input operand i, and multiplier and accumulator (first FPMAC unit) inside processing element), and
performing the FPMAC operations based on the input matrix and the weight matrix (Huang: ¶0044, e.g., Mac operations are performed in input and weight matrices) comprises:
performing, by a second FPMAC unit of the processing element, a second FPMAC operation of the FPMAC operations with the first input operand and the second weight operand (Volpe: Fig. 1E, e.g., PE 162b receives weight set 2 (second weight operand) and input data element (first input operand));
performing, by a third FPMAC unit of the processing element, a third FPMAC operation of the FPMAC operations with the second input operand and the first weight operand (Volpe: Fig. 1E, e.g., PE below 162a receives weight set 1 (first weight operand) and input data element (second input operand); Column 3 Lines 5-8, e.g., Each row corresponds to a specific input data element); and
performing, by a fourth FPMAC unit of the processing element, a fourth FPMAC operation of the FPMAC operations with the second input operand and the second weight operand (Volpe: Fig. 1E, e.g., PE below 162b receives weight set 2 (second weight operand) and input data element (second input operand); Column 3 Lines 5-8, e.g., Each row corresponds to a specific input data element).
The motivation to combine provided with respect to claim 18 applies equally to claim 19.
With regards to Claim 20, Huang in view of Raha in view of Beaumont-Smith in view of Das Sarma in view of Volpe teach:
The method of claim 19, further comprising:
accumulating results of the FPMAC operations (Huang: ¶0092).
With regards to Claims 24 and 25, they are media version of the claimed method above (claims 18 and 19 respectively), wherein all claim limitations also have been addressed and/or covered in cited areas. Thus, accordingly, this claim is rejected for at least the same reasons therein.
Prior Art Made of Record
US 12423058 B2 – teaches a PE array that uses a multiplier and adder in each processing element. See Fig. 1A, 2A and 2B, and corresponding description.
US 12443841 B2 – teaches a normalizer/rounder receiving outputs from adder. See Fig. 20 and corresponding description.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CARLOS H DE LA GARZA whose telephone number is (571)272-0474. The examiner can normally be reached Monday-Friday 9:30AM-6PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Caldwell can be reached at (571) 272-3702. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/C.H.D./
Carlos H. De La GarzaExaminer, Art Unit 2182 (571)272-0474
/ANDREW CALDWELL/Supervisory Patent Examiner, Art Unit 2182