Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Remarks
In ¶ [0416] of the specification, “moving the data is a different order” should be “moving the data in a different order”.
In Fig. 31A of the drawings, the inputs of from elements 2810-1 and 2810-7 that are stored inside element 2807b and 2807h respectively (the numbers between the square brackets) are the incorrect values. The series of “[50]” in element 2807b should be a series of “[58]”. The series of “[8]” in element 2807h should be a series of “[16]”.
Claim Rejections – 35 USC §103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 26, 27, 30, 32-37, 40-42, and 45 are rejected under 35 U.S.C. 103 as being unpatentable over Dally et al. (US 20210089864 A1) in view of Li et al. (US 20200409664 A1) and Nielsen (NPL – How the backpropagation algorithm works).
Regarding claim 26, Dally teaches a memory interface (Figure 2A; element 205) and a matrix acceleration hardware that is communicably connected to the memory interface and can perform matrix multiplication (Fig. 2A; array of processing elements 210).
However, Dally doesn’t teach a matrix acceleration hardware communicably coupled to the memory interface and comprising at least one storage buffer, wherein the matrix acceleration hardware is to: access a source matrix on which to perform a transpose operation; load, in accordance with the transpose operation, the source matrix as a transposed matrix to the at least one storage buffer of the matrix acceleration hardware; and utilize the transposed matrix for [the] matrix multiplication operation supported by Dally.
Li teaches a matrix acceleration hardware comprising at least one storage buffer (Fig. 4; element 420), wherein the matrix acceleration hardware is to: access a source matrix (Fig. 4; element 430) on which to perform a transpose operation; load, in accordance with the transpose, the source matrix as a transposed matrix to the at least one storage buffer (Fig. 4; element 420 and 430; ¶ [0074] – [0075]) of the matrix acceleration hardware.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the matrix acceleration hardware of Dally to perform matrix transposition as taught by Li, before utilizing the transposed matrix for a matrix multiplication operation as taught by Dally. It would’ve been obvious because a fundamental equation when performing backpropagation is the equation for finding the error in a neural network layer in terms of the next layer, which includes multiplying the transposed weight matrix with the error of the next layer (Nielsen; § “The four fundamental equations behind backpropagation”: § “An equation for the error
δ
l
in terms of the error in the next layer,
δ
l
+
1
”).
Regarding claim 27, the combination of Dally in view Li and Nielsen teaches all the limitations of claim 26. Li further teaches the transposed matrix being stored in the at least one storage buffer (Li; Fig. 4; element 420; ¶ [0075]), while Dally further teaches additional multiplication operations of the matrix acceleration hardware (Fig. 2A; set of PE 210), which the transposed matrix taught by Li will be used for.
Regarding claim 30, the combination of Dally in view Li and Nielsen teaches all the limitations of claim 26. Dally in view of Li and Nielsen further teaches the matrix acceleration hardware further reading data from the at least one storage buffer in a transpose order (Li; Fig. 4; element 420; ¶ [0074] – [0075]).
Regarding claim 32, the combination of Dally in view Li and Nielsen teaches all the limitations of claim 26. The combination of Dally in view Li and Nielsen further teaches the matrix multiplication operation references a dimensionality (Dally; fig. 2A; array of processing elements 210; ¶ [0049]) of the source matrix (Li; Fig. 4; element 420; ¶ [0074]-[0075]).
Regarding claim 33, the combination of Dally in view Li and Nielsen teaches all the limitations of claim 26. Dally further teaches a matrix acceleration hardware being comprised in a GPU (Figure 6; element 600; ¶ [0147]).
Regarding claim 34, the combination of Dally in view Li and Nielsen teaches all the limitations of claim 26. The combination of Dally in view of Li and Nielsen further teaches the matrix acceleration hardware comprising a tensor core hardware (Dally, figure 2A, set of PE 210; Li, figure 4, ¶ [0074] – [0075]).
Regarding claim 35, the combination of Dally in view Li and Nielsen teaches all the limitations of claim 34. Li further teaches the source matrix being accessed from tensor memory of the tensor core hardware (Fig. 4; element 430).
Regarding claim 36, it is a method claim corresponding to apparatus claim 26. It is rejected for the same reasons.
Regarding claim 37, it is a method claim corresponding to apparatus claim 27. It is rejected for the same reasons.
Regarding claim 40, it is a method claim corresponding to apparatus claim 30. It is rejected for the same reasons.
Regarding claim 41, Dally further teaches a memory to store a block of data (Dally; ¶ [0045]) and communicably coupled to the memory interface (Figure 2A; element 205). The other limitations of claim 41 are the same limitations as recited by claim 26 and are rejected accordingly.
Regarding claim 42, the combination of Dally in view Li and Nielsen teaches all the limitations of claim 41. The other limitations of claim 42 are limitations also recited by claim 27 and are rejected accordingly.
Regarding claim 45, the combination of Dally in view Li and Nielsen teaches all the limitations of claim 41. The other limitations of claim 42 are limitations also recited by claim 30 and are rejected accordingly.
Claims 28, 38, and 43 are rejected under 35 U.S.C. 103 as being unpatentable over Dally in view of Li and Nielsen as applied to claim 26 above, and further in view of Bradford et al. (US 10649772 B2).
Regarding claim 28, Dally in view of Li and Nielsen teaches all the limitations of claim 26. The combination of Dally in view of Li and Nielsen teach the matrix acceleration hardware [generating] the transposed matrix. However, the combination of Dally in view of Li and Nielsen doesn’t teach the matrix acceleration hardware applying write controls to the data to load the data in a scrambled order.
Bradford teaches applying write controls to the data to load the data in a scrambled order (Figure 3B; elements 352, 357A-D, and 356A-D; ¶ [0057]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the matrix acceleration hardware of the combination of Dally in view of Li and Nielsen to apply the write control to the data to load the data in a scrambled order as taught by Bradford. It would’ve been obvious because the source matrix can be transposed at a lower cost and die area (Bradford; ¶ [0054]).
Regarding claim 38, the combination of Dally in view Li and Nielsen teaches all the limitations of claim 36. The other limitations of claim 38 are limitations also recited by claim 28 and are rejected accordingly.
Regarding claim 43, the combination of Dally in view Li and Nielsen teaches all the limitations of claim 41. The other limitations of claim 42 are limitations also recited by claim 28 and are rejected accordingly.
Claims 29, 31, 39, and 44 are rejected under 35 U.S.C. 103 as being unpatentable over Dally in view of Li and Nielsen as applied to claim 26 above, and further in view of Mellempudi et al. (US 20220318013 A1).
Regarding claim 29, the combination of Dally in view of Li and Nielsen teaches the apparatus of claim 26:
wherein the matrix acceleration hardware has an array of processing elements (Dally; Fig. 2A; array of processing elements 210) comprising multiply-accumulate (MAC) circuits to support matrix acceleration operations, and wherein each MAC circuit comprises multiplier circuits (Dally; Fig. 2C; element 240) and at least one adder (Dally; Fig. 2C; element 255).
However, the combination of Dally in view of Li and Nielsen doesn’t teach each MAC circuit having shifters.
Mellempudi teaches each MAC circuit having shifters (Fig. 17A; elements 1715b-1715e).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the MAC circuits taught by the combination of Dally in view of Li and Nielsen to include shifters as taught by Mellenpudi. It would have been obvious because shifters normalize the outputs of multipliers (Mellempudi; ¶ [0238]).
Regarding claim 31, the combination of Dally in view of Li and Nielsen teaches all the limitations of claim 26. However, the combination of Dally in view of Li and Nielsen doesn’t teach the matrix multiplication operation comprising an asynchronous instruction.
Mellempudi teaches a matrix multiplication operation comprising an asynchronous instruction (Fig. 18A; element 1800; ¶ [0249]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify matrix operations taught by the combination of Dally in view of Li and Nielsen to include an asynchronous instruction as taught by Mellenpudi. It would have been obvious because a systolic depth (sdepth) in the asynchronous instruction can be used to specify the number systolic layers to use to process input data (Mellempudi; ¶ [0251]).
Regarding claim 39, it is a method claim corresponding to apparatus claim 29. It is rejected for the same reasons.
Regarding claim 44, the combination of Dally in view Li and Nielsen teaches all the limitations of claim 41. The other limitations of claim 44 are the limitations also recited by claim 29 and are rejected accordingly (including the teachings of Mellempudi).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Goossaert (NPL – “Implementing a Key-Value Store – Part 7: Optimizing Data Structures for SSDs”) teaches sequential addresses in the logical space of a solid state drive (SSD) isn’t always sequential in the physical space (§ Data is never really sequential). Goossaert also teaches memory mapping, which maps an area of a file directly into an address in a memory (§ Memory Mapping).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to David Jiarui Liu whose telephone number is (571) 270-3004. The examiner can normally be reached M-Th, 7:30 am to 5 pm ET, F, 8:30 am to 1 pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Caldwell can be reached at (571) 272-3702. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAVID J LIU/Examiner, Art Unit 2182
/ANDREW CALDWELL/Supervisory Patent Examiner, Art Unit 2182
