DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
The present application, 18222989, filed on 07/17/2023 is a Continuation of 17740057, filed on 05/09/2022 now abandoned is a Continuation of 15644526, filed 07/07/2017, now U.S. Patent # 11328037.
Remarks
A prior art rejection under 35 U.S.C. 103 was made in the parent application which was overcome after a declaration under 37 C.F.R. 1.132 was submitted. Examiner would like to point out that affidavits or declarations, such as those submitted under 37 CFR 1.132 filed during the prosecution of the prior nonprovisional application do not automatically become a part of a continuation. If Applicant wants the declaration to be part of the present application, Applicant must include a copy of the original declaration filed in the prior nonprovisional application. A copy of the original declaration does not to be resigned/re-executed. See MPEP 201.06(c) IX for more information.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 07/17/2023 and 07/25/2025 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner except for the reference that is struck-through because the non-patent literature document of Ling et al. “Creating High Performance Applications with Intel’s FPGA SDK for OpenCL” was already cited and considered in the previous IDS submission.
Drawings
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(4) because reference character “11” has been used to designate both integrated circuit in Figs. 1 and 11-12 and an example of banking in Fig. 4.
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5) because they do not include the following reference sign(s) mentioned in the description:
A. block banking 39 in Fig. 4 mentioned in paragraph [0021] lines 9-10.
B. 100 mentioned in paragraph [0027] line 10
C. drain bank 98 in Fig. 7 mentioned in paragraph [0028] lines 9-10
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Specification
The specification is objected to under 37 C.F.R. 1.74, which requires the detailed description to refer to the different parts of the figures by use of reference letters or reference numerals. Implicit in this rule is that the detailed description correctly reference the figures. In this application the figures and detailed description are inconsistent as explained below.
A. In paragraph [0019], the disclosure mentions that the "B" block 44 is subdivided into B1, B2, and B3. However, Fig. 3 shows the "B" block is subdivided into B1, B1, and B1.
B. In paragraph [0028], the specification refers to 112 as drain bank, however 112 is labeled as drain control in Fig. 7.
C. In paragraph [0028], the specification refers to 104 as an accumulator control block, however 104 is labeled as account control in Fig. 7.
Claim Objections
Claims 2-8, 10 and 14-21 are objected to under 37 C.F.R. 1.71(a) which requires “full, clear, concise, and exact terms” as to enable any person skilled in the art or science to which the invention or discovery appertains, or with which it is most nearly connected, to make and use the same. The following should be corrected.
A. In claim 2 lines 2-3, “respective dot product circuitry and accumulator circuitry” should read “respective dot product circuitry and respective accumulator circuitry” instead for better clarity. Claim 14 recites a similar limitation and is objected to for the same reason. Claims 3-8 inherit the same deficiency as claim 2 by reason of dependence. Claims 15-21 inherit the same deficiency as claim 14 by reason of dependence.
B. In claim 10 lines 11-12, “a plurality of segments of the first block” should read “the plurality of segments of the first block” instead.
C. In claim 11 line 13, “a plurality of segments of the first block” should read “the plurality of segments of the first block” instead.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 6-7, 11 and 18-19 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 6 recites “before it is provided to the respective processing element” in line 6. It is unclear what it is supposed to be referring. For purposes of examination, this is interpreted as before the data is provided to the respective processing element. Claim 18 recites a similar limitation and is rejected for the same reason. Claim 7 inherit the same deficiency as claim 6 by reason of dependence. Claim 19 inherit the same deficiency as claim 18 by reason of dependence
Further, claim 6 recites “before it is provided to the respective processing element” in line 12. It is unclear what it is supposed to be referring. Further, it is unclear whether the respective processing element recited in line 12 is supposed to refer to the respective processing element in the outermost row or to the respective processing element in the outermost column. For purposes of examination, this is interpreted as before the data is provided to the respective processing element in the outermost column. Claim 18 recites a similar limitation and is rejected for the same reason. Claim 7 inherit the same deficiency as claim 6 by reason of dependence. Claim 19 inherit the same deficiency as claim 18 by reason of dependence.
Claim 11 recites “first block” in line 2. It is unclear whether this is supposed to be interpreted to refer to the first block of the first matrix or to the first block of the second matrix. For purposes of examination, this is interpreted to refer to the first block of the first matrix.
Further, claim 11 recites “the second block” in line 2. There is insufficient antecedent basis for this limitation in the claim. For purposes of examination, this is interpreted to refer to the first block of the second matrix.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 2-21 are rejected under 35 U.S.C. 103 as being unpatentable over Shalev et al. (US 10853448 B1), hereinafter Shalev, in view of Zhang et al. (US 20180314671 A1), hereinafter Zhang. Shalev and Zhang are cited in the IDS submitted on 07/17/2023.
Regarding claim 2, Shalev teaches circuitry of an integrated circuit, comprising:
a plurality of processing elements (Shalev Fig. 1 and col 5 lines 32-34 plurality of processing elements - array of processing elements 24);
loading circuitry to retrieve a first block of a first matrix and a first block of a second matrix from a memory (Shalev Figs. 1, 3-8b and col 6 lines 17-19; loading circuitry – load units 36; first block of a first matrix – vector/column of matrix A or B; first block of a second matrix – vectors/rows of matrix B or A; memory – memory 34; col 5 lines 55-56); and
feed the respective segments of the first block of the first matrix and the respective segments of the first block of the second matrix to the plurality of processing elements interleaved over time (Shalev Figs. 3-8b and col 6 lines 19-23 “These values are then distributed among processing elements 24 by data manipulation unit 40. Examples of these data extraction, manipulation and distribution functions are described hereinbelow with reference to FIGS. 3-8B”; col 8 line 65 to col 9 line 16 “data access logic 31 partitions input matrix 99 into two adjoining tiles, A0 and A1. In addition, data access logic 31 further partitions each of these tiles by assigning, in interleaved sequence, successive columns of each tile to different, alternating sub-matrices. (Alternatively or additionally, the tiles could be partitioned by rows)”).
Shalev does not explicitly teach a plurality of processing elements comprising respective dot product circuitry and accumulator circuitry; and feeder circuitry comprising a plurality of banks to store respective segments of the first block of the first matrix and respective segments of the first block of the second matrix.
However, on the same field of endeavor, Zhang discloses a circuitry comprising plurality of processing elements comprising respective dot product circuitry and accumulator circuitry (Zhang Figs. 2-3 and paragraphs [0028, 0038 and 0040]; plurality of processing elements - two-dimensional array of processing elements; dot product circuitry – multipliers 308-310; accumulator circuitry - combination of an adder 314, a register 316 and a C cyclic accumulation shift register 318); and feeder circuitry comprising a plurality of banks to store respective segments of a first matrix and respective segments of a second matrix (Zhang Figs. 2 and 4 and paragraphs [0031-0032] feeder circuitry – input buffer and weight buffer; paragraph [0042] “each of buffer 0 404 and buffer 1 406 is comprised of an array of buffer segments”).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention, to modify Shalev using Zhang and configure each processing elements to include respective dot product circuitry and accumulator circuitry in order to implement the multiply-accumulate operation (Zhang paragraph [0040]). As shown in Fig. 2 of Shalev each processing unit includes an FMA, and the multiplier-accumulator arrangement of Zhang can be used to perform the function of the FMA as the FMA also includes a multiplier-adder structure (Shalev col 4 lines 11-12 and col 4 lines 43-44). Further, col 6 lines 47-48 also discloses that other sorts of multiplier accumulators may alternatively be used in lieu of the FMA 50. Further, configure the matrix multiplication engine to include weight and input buffers for storing respective segments of the first block of matrix A and respective segments of the first block of matrix B and feed the respective segments of the first block of matrix A and the respective segments of the first block of matrix B to the plurality of processing elements interleaved over time for data reuse in order and to reduce memory access (Zhang paragraph [0048] and Shalev col 4 lines 36-40).
Therefore, the combination of Shalev as modified in view of Zhang teaches a plurality of processing elements comprising respective dot product circuitry and accumulator circuitry; and feeder circuitry comprising a plurality of banks to store respective segments of the first block of the first matrix and respective segments of the first block of the second matrix.
Regarding claim 3, Shalev as modified in view of Zhang teaches the limitations of claim 2 as stated above. Further, Shalev as modified in view of Zhang teaches wherein the respective accumulator circuitry of the plurality of processing elements is controllable to be drained upon accumulating results of dot products of the respective segments of the first block of the first matrix and the respective segments of the first block of the second matrix obtained using the respective dot product circuitry of the plurality of processing elements (Zhang paragraph [0040] “Once the data elements are ready in C cyclic accumulation shift register 318, they are then shifted out into a C output shifting register set 320. Finally, all the data elements in C output shifting register set 320 are shifted out of the PE via an output channel C_out 330 to the bottom neighboring PEs”).
Regarding claim 4, Shalev as modified in view of Zhang teaches the limitations of claim 3 as stated above. Further, Shalev as modified in view of Zhang teaches wherein the accumulated results of the dot products of the respective segments of the first block of the first matrix and the respective segments of the first block of the second matrix correspond to a dot product of the first block of the first matrix and the first block of the second matrix (Shalev Figs. 3-8B, col 8 lines 19-26, and col 9 lines 14-28).
Regarding claim 5, Shalev as modified in view of Zhang teaches the limitations of claim 2 as stated above. Further, Shalev as modified in view of Zhang teaches wherein the plurality of processing elements are arranged in a (Shalev col 5 line 32).
Shalev does not explicitly teach wherein the plurality of processing elements are arranged in a systolic array.
However, on the same field of endeavor, Zhang discloses a plurality of processing elements are arranged in a systolic array (Zhang Fig. 2 and paragraph [0028]).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention, to modify Shalev using Zhang arrange the plurality of processing elements in a systolic array architecture in order to reduce external memory access by passing the matrix A and matrix B data of one PE to a neighboring PE (Zhang paragraph 0035]).
Therefore, the combination of Shalev as modified in view of Zhang teaches wherein the plurality of processing elements are arranged in a systolic array.
Regarding claim 6, Shalev as modified in view of Zhang teaches the limitations of claim 2 as stated above. Further, Shalev as modified in view of Zhang teaches wherein the feeder circuitry comprises:
an array of column feeders respectively (Zhang Figs. 2 and 4; array of column feeders – input buffer):
coupled to a respective processing element of an outermost row of the plurality of processing elements (Zhang Fig. 2 outermost row – top row); and
comprising a respective buffer memory to serve as a bank to temporarily store data before it is provided to the respective processing element or to another column feeder (Zhang Figs. 2 and 4 and paragraphs [0032, 0041-0042] respective buffer memory – buffer 0 and/or buffer 1); and
an array of row feeders respectively (Zhang Figs. 2 and 4; array of row feeders – weight buffer):
coupled to a respective processing element of an outermost column of the plurality of processing elements (Zhang Fig. 2 outermost column – left column); and
comprising a respective buffer memory to serve as a bank to temporarily store data before it is provided to the respective processing element or to another row feeder (Zhang Figs. 2 and 4 and paragraphs [0032, 0041-0042] respective buffer memory – buffer 0 and/or buffer 1).
Regarding claim 7, Shalev as modified in view of Zhang teaches the limitations of claim 6 as stated above. Further, Shalev as modified in view of Zhang teaches wherein feeding the respective segments of the first block of the first matrix and the respective segments of the first block of the second matrix to the plurality of processing elements interleaved over time comprises (Shalev Figs. 5a-5b and col 8 lines 64 to col 9 lines 28; Figs. 8a-8b and col 10 line 58 to col 11 line 17):
the array of column feeders feeding a first segment of the respective segments of the first block of the first matrix as the array of row feeders feeds the respective segments of the first block of the second matrix (Shalev Figs. 8a-8b and col 10 line 58 to col 11 line 17; first segment of the first block of the first matrix – 170; respective segments of the first block of the second matrix – 166 and 168); and
the array of column feeders feeding a second segment of the respective segments of the first block of the first matrix as the array of row feeders feeds the respective segments of the first block of the second matrix (Shalev Figs. 8a-8b and col 10 line 58 to col 11 line 17; second segment the first block of the first matrix – 172).
Regarding claim 8, Shalev as modified in view of Zhang teaches the limitations of claim 2 as stated above. Further, Shalev as modified in view of Zhang teaches wherein the first block of the first matrix corresponds to a column of the first matrix and the first block of the second matrix corresponds to a row of the second matrix (Shalev Figs. 3-8B and col 6 lines 18-23; the vectors are the rows/columns of matrix A and/or B).
Regarding claim 9, Shalev teaches a method comprising:
loading a first block of a first matrix and a first block of a second matrix from memory (Shalev Figs. 1, 3-8b and col 6 lines 17-19; first block of a first matrix – vector/column of matrix A or B; first block of a second matrix – vectors/rows of matrix B or A; memory – memory 34; col 5 lines 55-56);
feeding different segments of the first block of the first matrix and the first block of the second matrix at different times (Shalev Figs. 1 and 3-8b and col 6 lines 19-23 “These values are then distributed among processing elements 24 by data manipulation unit 40. Examples of these data extraction, manipulation and distribution functions are described hereinbelow with reference to FIGS. 3-8B”; col 8 line 65 to col 9 16 “data access logic 31 partitions input matrix 99 into two adjoining tiles, A0 and A1. In addition, data access logic 31 further partitions each of these tiles by assigning, in interleaved sequence, successive columns of each tile to different, alternating sub-matrices. (Alternatively or additionally, the tiles could be partitioned by rows)”; plurality of processing elements - array of processing elements 24); and
obtaining a matrix multiplication product of the first block of the first matrix and the first block of the second matrix by accumulating matrix multiplications of the different segments of the first block of the first matrix and the first block of the second matrix at the different times using the plurality of processing elements (Shalev Figs. 3-8b, col 8 line 65 to col 9 line 16, and col 10 line 58 to col 11 line 17; matrix multiplication product – output 111/164; different segments of the first block of the first matrix – A0 and A1 or 166 and 168 or 170 and 172).
Shalev does not explicitly teach loading a first block of a first matrix and a first block of a second matrix from memory into feeder circuitry; and feeding different segments of the first block of the first matrix and the first block of the second matrix at different times from the feeder circuitry into a plurality of processing elements.
However, on the same field of endeavor, Zhang discloses a circuitry comprising feeder circuitry to store respective segments of a first matrix and respective segments of a second matrix; and plurality of processing elements receiving the respective segments of the first matrix and respective segments of the second matrix from the feeder circuitry (Zhang Figs. 2 and 4 and paragraphs [0031-0032] feeder circuitry – input buffer and weight buffer).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention, to modify Shalev using Zhang and configure the matrix multiplication engine to include weight and input buffers for storing the first block of matrix A and the first block of matrix B and feed respective segments of the first block of matrix A and the first block of matrix B to the plurality of processing elements interleaved over time for data reuse in order and to reduce memory access (Zhang paragraph [0048] and Shalev col 4 lines 36-40).
Therefore, the combination of Shalev as modified in view of Zhang teaches loading a first block of a first matrix and a first block of a second matrix from memory into feeder circuitry; and feeding different segments of the first block of the first matrix and the first block of the second matrix at different times from the feeder circuitry into a plurality of processing elements.
Regarding claim 10, Shalev as modified in view of Zhang teaches the limitations of claim 9 as stated above. Further, Shalev as modified in view of Zhang teaches wherein feeding the different segments of the first block of the first matrix and the first block of the second matrix and obtaining the matrix multiplication product comprises (Shalev Figs. 8a-8b and col 10 line 58 to col 11 line 17):
feeding, into the plurality of processing elements, a first segment of a plurality of segments of the first block of the first matrix to the plurality of processing elements and feeding a first segment of a plurality of segments of the first block of the second matrix to the plurality of processing elements (Shalev Figs. 8a-8b and col 10 line 58 to col 11 line 17 first segment of a plurality of segments of the first block of the first matrix – 170; first segment of a plurality of segments of the first block of the second matrix – 166);
performing a first matrix multiplication of the first segment of the first block of the first matrix and the first segment of the first block of the second matrix using the plurality of processing elements at a first time (Shalev Figs. 8a-8b and col 10 line 58 to col 11 line 17 first time – cycle 1);
feeding, into the plurality of processing elements, a second segment of a plurality of segments of the first block of the first matrix to the plurality of processing elements and feeding the first segment of a plurality of segments of the first block of the second matrix to the plurality of processing elements (Shalev Figs. 8a-8b and col 10 line 58 to col 11 line 17 second segment of a plurality of segments of the first block of the first matrix – 172);
performing a second matrix multiplication of the second segment of the first block of the first matrix and the first segment of the first block of the second matrix using the plurality of processing elements at a second time (Shalev Figs. 8a-8b and col 10 line 58 to col 11 line 17 second time – cycle 2); and
accumulating results of the first matrix multiplication and the second matrix multiplication (Shalev Figs. 8a-8b and col 10 line 58 to col 11 line 17 results – C0 and C1).
Regarding claim 11, Shalev as modified in view of Zhang teaches the limitations of claim 9 as stated above. Further, Shalev as modified in view of Zhang teaches wherein the first block comprises a first row of the first matrix and the second block comprises a first column of the second matrix (Shalev Figs. 3-8B and col 6 lines 18-23; the vectors are the rows/columns of matrix A and/or B).
Regarding claim 12, Shalev as modified in view of Zhang teaches the limitations of claim 9 as stated above. Further, Shalev as modified in view of Zhang teaches wherein obtaining the matrix multiplication product comprises performing a plurality of dot product operations in the plurality of processing elements arranged in a (Shalev col 4 line 60-67 and col 5 lines 31-33 “Computations in engine 20 are carried out by an execution unit 22, comprising an array of processing elements 24, which perform multiply and accumulate operations”).
Shalev does not explicitly teach the plurality of processing elements arranged in a systolic array.
However, on the same field of endeavor, Zhang discloses a plurality of processing elements arranged in a systolic array (Zhang Fig. 2 and paragraph [0028]).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention, to modify Shalev using Zhang and arrange the plurality of processing elements in a systolic array architecture in order to reduce external memory access by passing the matrix A and matrix B data of one PE to a neighboring PE (Zhang paragraph 0035]).
Therefore, the combination of Shalev as modified in view of Zhang teaches the plurality of processing elements arranged in a systolic array.
Regarding claim 13, Shalev as modified in view of Zhang teaches the limitations of claim 12 as stated above. Further, Shalev as modified in view of Zhang teaches
wherein obtaining the matrix multiplication product comprises accumulating results of the plurality of dot product operations in the plurality of processing elements (Shalev Figs. 3-8b and col 9 lines 14-28; col 11 lines 9-17).
Regarding claims 14-20, they are directed to an article of manufacture comprising tangible, non-transitory, machine-readable media comprising instructions to implement following circuitry of claims 2-8 respectively. Claims 2-8 analysis applies equally to claims 14-20 respectively.
Regarding claim 21, Shalev as modified in view of Zhang teaches the limitations of claim 14 as stated above. Further, Shalev as modified in view of Zhang teaches wherein the circuitry is configured to be operated according to a method comprising: loading the first block of the first matrix and the first block of the second matrix from the memory into the feeder circuitry using the loading circuitry (Shalev Figs. 1, 3-8b and col 6 lines 17-19);
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Carlo Waje whose telephone number is (571)272-5767. The examiner can normally be reached 9:00-6:00 M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Caldwell can be reached at (571) 272-3702. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Carlo Waje/Examiner, Art Unit 2182 (571)272-5767