DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
The lengthy specification has not been checked to the extent necessary to determine the presence of all possible minor errors. Applicant’s cooperation is requested in correcting any errors of which applicant may become aware in the specification.
Drawings
The applicant’s submitted drawings appear to be acceptable for examination purposes. Applicant’s cooperation is requested in correcting any errors of which applicant may become aware in the drawings.
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. 16/317,497, filed on 11 January 2019.
Information Disclosure Statement
The information disclosure statement filed 14 March 2024 fails to comply with 37 CFR 1.98(a)(3)(i) because it does not include a concise explanation of the relevance, as it is presently understood by the individual designated in 37 CFR 1.56(c) most knowledgeable about the content of the information, of each reference listed that is not in the English language. It has been placed in the application file, but the information referred to therein has not been considered.
Claim Objections
Applicant is advised that should claims 1-8 be found allowable, claims 9-16 will be objected to under 37 CFR 1.75 as being a substantial duplicate thereof. When two claims in an application are duplicates or else are so close in content that they both cover the same thing, despite a slight difference in wording, it is proper after allowing one claim to object to the other as being a substantial duplicate of the allowed claim. See MPEP § 608.01(m). In this case there is a slight difference in wording, but the “plurality of on-chip memories” and “plurality of memories” are substantially the same things in the context of the claim language, and the “plurality of compute tiles arranged in a two-dimensional array” and the “plurality of matrix processors coupled in a two-dimensional array” are substantially the same things in the context of the claim language, while the rest of the claimed invention appears to be the same.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 6 and 14 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The term “final” in claim 6 is a relative term which renders the claim indefinite. The term “final” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention.
The term “final” in claim 14 is a relative term which renders the claim indefinite. The term “final” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 2, 8-10, and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ginzburg (US 2011/0153707 – cited in an IDS) in view of Wilson (US 6,728,862).
As per claim 1, Ginzburg teaches an apparatus comprising:
a plurality of on-chip memories [each processing unit can include multiple memories, including registers, caches, etc. (figs. 1, 7, etc.)], one or more of the on-chip memories to store data elements of a first matrix and a second matrix [matrices can be stored in the registers (paras. 0031-34; fig. 3; etc.)];
a plurality of compute tiles, each compute tile to perform matrix multiplication operations [the computer system can include multiple cores, each of which can include multiple processing units (paras. 0022, 0063; fig. 8; etc.) which include elements for performing matrix multiplication operations (paras. 0018-19; figs. 4-7; etc.); where each core or each processing unit are both within the broadest reasonable interpretation of compute tiles performing the matrix multiplication operations];
a plurality of interconnects to each compute tile of the plurality of compute tiles to a corresponding subset of on-chip memories of the plurality of on-chip memories [each of the cores and each of the processing units include interconnects connecting them to other units (a corresponding subset) and to multiple levels of memory (figs. 1, 7, 8, etc.)];
an external memory interface to couple the plurality of on-chip memories to an external memory [the cores and processing units are connected, via interconnections/interface to external memories (figs. 1, 7, 8, etc.)]; and
a first compute tile of the plurality of compute tiles comprising:
an instruction memory to store an instruction indicating the data elements of the first matrix and the second matrix [the processing unit stores instructions in and receives instructions from a local cache/memory (para. 0027; fig. 1; etc.) which instructions can include 2D matrix multiply-add operations indicating memory locations of the matrix data elements (para. 0028, etc.)];
a local memory to store a respective subset of the data elements of the first matrix and the second matrix [the data elements of the matrices are stored in registers or memory locations pointed at by the fields of the matrix multiply-add instruction(s) (paras. 0027-28; etc.) where each MMAU sub-unit computes a row of the result matrix to generate a portion of the row ordered result (para. 0039, etc.); where each register and/or memory location stores a respective subset of the data elements of each matrix];
a decoder to decode the instruction [the processing units include decoders to decode instructions to determine the operations to be performed (para. 0027; fig. 1; etc.)], and
multiply-accumulate circuitry to execute the instruction over multiple execution lanes to generate multiple corresponding accumulated values [each execution unit includes multiply add units (paras. 0035-36; figs. 4-5) which performs the multiply add operations over multiple execution lanes (figs. 5-6, etc.)], the multiply-accumulate circuitry comprising:
a plurality of multipliers to multiply the respective subset of the data elements of the first matrix by corresponding data elements of the second matrix to generate a corresponding plurality of products [the processing units include multipliers to multiply data elements of the first and second matrices (paras. 0035-37; figs. 5-6; etc.)]; and
a plurality of adders to add one or more products of the plurality of products to a corresponding accumulated value to generate a corresponding new accumulated value [the processing units include multipliers and adders to multiply data elements of the first and second matrices and adding to intermediate results (accumulated values) (paras. 0035-37, 0043-44; claim 14; figs. 5-6; etc.)].
While Ginzburg teaches a computer system including multiple cores that include multiple processing units (see above), it has not been relied upon for teaching the plurality of compute tiles arranged in a two-dimensional array.
Wilson teaches a plurality of compute tiles arranged in a two-dimensional array [an array of processor elements (col. 4, lines 49-52; figs. 1, 3; etc.)]; [and]
a plurality of interconnects to each compute tile of the plurality of compute tiles to a corresponding subset of on-chip memories of the plurality of on-chip memories [an array of interconnected processor elements (col. 4, lines 49-52; figs. 1, 3; etc.) which include local connectivity between processor elements (col. 10, line 59 to col. 11, line 5; fig. 3; etc.)].
Ginzburg and Wilson are analogous art, as they are within the same field of endeavor, namely processing element architectures for performing matrix operations.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to arrange the processing elements in a two-dimensional array including specified interconnections between selected processing elements, as taught by Wilson, for arranging the processing units/cores in the system taught by Ginzburg.
Wilson provides motivation as [This architecture provides a number of advantages: it provides direct access to a good number of local processor elements in the horizontal and vertical directions of the array, supporting many typical imaging (matrix) operations and, it provides indirect access (through two steps) to an even larger area without incurring the logic and wiring overhead of a direct connection (col. 12, lines 30-36; etc.)]. Additionally, it has been held that rearranging parts of an invention (i.e. arranging the cores and/or processing elements into a 2-D array) involves only routine skill in the art. In re Japikse, 86 USPQ 70.
As per claim 2, Ginzburg/Wilson teaches wherein each compute tile of the plurality of compute tiles includes a scalar unit comprising:
a plurality of scalar registers to store scalar data elements [each processing unit includes multiple register files and can include multiple functional units comprising one or more functional units, such as, for example, arithmetic, logic, and/or floating-point units (Ginzburg: para. 0024; figs. 1, 4; etc.); where performing arithmetic, logic, or floating point operations on a 16bit register operand is a scalar operation], and
a scalar arithmetic logic unit (ALU) to execute scalar instructions using the scalar data elements [each processing unit includes multiple register files and can include multiple functional units comprising one or more functional units, such as, for example, arithmetic, logic, and/or floating-point units (Ginzburg: para. 0024; figs. 1, 4; etc.); where performing arithmetic, logic, or floating point operations on a 16bit register operand is a scalar operation].
As per claim 8, Ginzburg/Wilson teaches wherein the data elements of the first matrix and the second matrix are floating-point values [the data elements operated on by the processing units can include floating-point data elements (Ginzburg: paras. 0037, 0044, etc.)].
As per claim 9, see the rejection of claim 1, above, wherein the on-chip memories are a plurality of memories and the compute tiles are matrix processors (as they are performing matrix multiplication).
As per claim 10, see the rejection of claim 2 above.
As per claim 16, see the rejection of claim 8 above.
Claim(s) 3-6 and 11-14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ginzburg and Wilson as applied to claims 1 and 9 above, and further in view of Henry (US 2017/0103321 – cited in an IDS).
As per claim 3, Ginzburg/Wilson teaches the apparatus of claim 1, above.
While Ginzburg/Wilson teaches performing matrix multiply-add instructions on multiple matrices (see above), it has not been relied upon for teaching wherein the data elements of the second matrix comprise neural network weights.
Henry teaches wherein the data elements of the second matrix comprise neural network weights [the multiply-accumulate operations perform convolutions on the input- and weight matrices (paras. 0174, 0180, 188-192, etc.) in addition to storing and operating on other kinds of matrices (para. 0053, etc.)].
Ginzburg/Wilson and Henry are analogous art, as they are within the same field of endeavor, namely processing element architectures for performing matrix operations.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to utilize the array of processing elements/weights for performing matrix operations on neural network weights and inputs, as taught by Henry, in the array of processing elements performing matrix operations in the system taught by Ginzburg/Wilson.
Henry provides motivation as [Recently, there has been a resurgence of interest in artificial neural networks (ANN), and such research has commonly been termed deep learning, computer learning and similar terms. The increase in general-purpose processor computation power has given rise to the renewed interest that waned a couple of decades ago. Recent applications of ANNs have included speech and image recognition, along with others. There appears to be an increasing demand for improved performance and efficiency of computations associated with ANNs (para. 0003, etc.)].
As per claim 4, Ginzburg/Wilson/Henry teaches wherein the data elements of the first matrix indicate input features [the multiply-accumulate operations perform convolutions on the input- and weight matrices (Henry: paras. 0174, 0180, 188-192, etc.) in addition to storing and operating on other kinds of matrices (Henry: para. 0053, etc.)].
As per claim 5, Ginzburg/Wilson teaches logic to compute an activation function based on the new accumulated value [the processing units are used to perform multiply-accumulate-activation function computations (Henry: paras. 0053-56; figs. 6B, 11; etc.) which multiplies the input elements, adds all the products to generate a sum, then performs an activation function on the sum (Henry: paras. 0060-61, etc.)].
As per claim 6, Ginzburg/Wilson teaches wherein the new accumulated value comprises a final accumulated value [the processing units include multipliers and adders to multiply data elements of the first and second matrices and adding to intermediate results (accumulated values) to produce a final output (Ginzburg: paras. 0035-37, 0043-44; claim 14; figs. 5-6; etc.); and performs multiply-accumulate-activation function computations (Henry: paras. 0053-56; figs. 6B, 11; etc.) which multiplies the input elements, adds all the products to generate a sum, then performs an activation function on the sum (Henry: paras. 0060-61, etc.)].
As per claim 11, see the rejection of claim 3 above.
As per claim 12, see the rejection of claim 4 above.
As per claim 13, see the rejection of claim 5 above.
As per claim 14, see the rejection of claim 6 above.
Claim(s) 7 and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ginzburg and Wilson as applied to claims 1 and 9 above, and further in view of Hansen (US 2015/0378734 – cited in an IDS).
As per claim 7, Ginzburg/Wilson teaches the apparatus of claim 1, above.
While Ginzburg/Wilson teaches performing a matrix multiply-add instruction on multiple matrices (see above), it has not been relied upon for teaching wherein the instruction is to indicate a first size associated with the first matrix and a second size associated with the second matrix.
Hansen teaches wherein the instruction is to indicate a first size associated with the first matrix and a second size associated with the second matrix [the EXWIDEMATRIX instruction can be an EXWIDEMATRIX-MULTIPLY instruction which performs matrix-vector or matrix-matrix multiplication, using size and shape parameters from the operand descriptors to indicate the size and shape of the operand matrices (paras. 0040-45, 0052-54, 0058; claim 5; etc.) for execution by a matrix execution functional unit (fig. 7, etc.)].
Ginzburg/Wilson and Hansen are analogous art, as they are within the same field of endeavor, namely processing element architectures for performing matrix operations.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to utilize a single instruction for performing matrix multiplication, including indicating the sizes of the matrices for a matrix functional unit, as taught by Hansen, in the system utilizing a matrix multiply-add instruction to perform multiplication and addition for two matrices in the system taught by Ginzburg/Wilson.
Hansen provides motivation as [by allowing the instruction to specify the size and shape of the matrices, efficiency can be increased, while also allowing more flexibility (paras. 0024-27, etc.)].
As per claim 15, see the rejection of claim 7 above.
Conclusion
The following is a summary of the treatment and status of all claims in the application as recommended by M.P.E.P. 707.07(i): claims 1-16 are rejected.
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Eichenberger (US 2011/0055517 – cited in an IDS) – discloses fused multiply-add instructions and units for matrix multiplication including complex matrix multiplication.
Brothers (US 2017/0011288 – cited in an IDS) – discloses a matrix multiply unit array for performing convolutions.
Nasiri et al. (Modified Fused Multiply-Accumulate Chained Unit, Sept 2014, pgs. 889-892 – cited in an IDS) – discloses pipelined fused multiply add units forming a multiply-accumulation chained unit.
Abdallah (US 2016/0026486 – cited in an IDS) – discloses decoding macro instructions into micro instructions and optimizing micro instructions.
Hancock (US 2013/0268794 – cited in an IDS) – discloses decoding macro instructions into micro instructions and optimizing micro instructions.
Hansen (US 7,843,459 – cited in an IDS) – discloses a matrix multiplication instruction including indicating the size of the matrix elements to be multiplied.
Ginzburg (US 2011/0153707 – cited in an IDS) – discloses using a single matrix multiply instruction.
Ge (US 2014/0237010 – cited in an IDS) – discloses a matrix multiplication instruction.
Shifer (US 2014/0089635 – cited in an IDS) – discloses a multicore processor including matrix multiplication logic as well as decoding macroinstructions into multiple micro-instructions.
Gopal (US 2014/0006753 – cited in an IDS) – discloses a matrix multiply accumulate instruction, as well as decoding macroinstructions into micro-operations.
Kojima (US 5,038,312 – cited in an IDS) – discloses a data processing system including decoding macro-instructions into micro-instructions for matrix-vector multiplication.
Vangal et al. (An 80-Tile Sub-100-W TeraFLOPS Processor in 65-nm CMOS, Jan 2008, pgs. 29-41) – discloses an array of floating point cores including floating-point multiply accumulators for matrix operations, and local instruction and data memories.
Shafiee et al. (ISAAC: A Convolutional Neural Network Accelerator with In-Situ Analog Arithmetic in Crossbars, June 2016, pgs. 14-26) – discloses a CNN accelerator with an array of processing tiles including neural functional units and local memory for performing multiply-accumulate (dot product) operations.
Luo et al. (DaDianNao: A Neural Network Supercomputer, May 2016, pgs. 73-88) – discloses a DNN/CNN accelerator including an array of nodes, which include arrays of tiles, including performing multiply-accumulate operations.
The examiner requests, in response to this Office action, that support be shown for language added to any original claims on amendment and any new claims. That is, indicate support for newly added claim language by specifically pointing to page(s) and line number(s) in the specification and/or drawing figure(s). This will assist the examiner in prosecuting the application.
When responding to this office action, Applicant is advised to clearly point out the patentable novelty which he or she thinks the claims present, in view of the state of the art disclosed by the references cited or the objections made. He or she must also show how the amendments avoid such references or objections. See 37 CFR 1.111(c).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GEORGE GIROUX whose telephone number is (571)272-9769. The examiner can normally be reached M-F 10am-6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Omar Fernandez Rivas can be reached at 571-272-2589. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GEORGE GIROUX/Primary Examiner, Art Unit 2128
Prosecution Insights