DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This Office Action is in response to applicant’s communication filed 18 June 2025, in response to the Office Action mailed 19 December 2024. The applicant’s remarks and any amendments to the claims or specification have been considered, with the results that follow.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-23 is/are rejected under 35 U.S.C. 103 as being unpatentable over Henry (US 2017/0103321) in view of Hansen (US 2015/0378734).
As per claim 1, Henry teaches an apparatus comprising: a first parallel compute device comprising a plurality of chips, the plurality of chips including a first chip coupled to a first memory device and a second chip coupled to a second memory device [a processing core connected to multiple memory systems (fig. 1, etc.) which is one of multiple processing cores in a multi-core processor connected to individual and shared memory (para. 0052, etc.) and where each core is on its own chip or die (para. 0044, etc.); where the multi-core processor is the first parallel compute device comprising the plurality of chips, where each core is a chip connected to a memory device]; a first interconnect to couple the first chip and the second chip [the cores may be connected by a bus interface and busses to shared cache and system memory (para. 0052; fig. 1; etc.)]; and an interface to couple the first parallel compute device to one or more additional parallel compute devices [the cores may be connected by a bus interface and busses to shared cache and system memory (para. 0052; fig. 1; etc.)]; the first chip comprising: a first scalar register file to store data related to control flow operations [each processor chip includes media registers, general purpose registers, and control & status registers (fig. 1, etc.) which can be used for control flow instruction execution, including branches with target addresses, call or return instructions, etc. (paras. 0045, 0052, etc.)]; first scalar execution circuitry to execute one or more scalar instructions to perform control flow operations in accordance with the data to control execution of an instruction sequence [each processor chip includes a branch unit that can be used for control flow instructions including branches (paras. 0045, 0052, etc.)], the control flow operations including program loops, branches, and address calculation [each processor chip includes a branch unit that can be used for control flow instructions including branches, which include branch target address calculation, (paras. 0045, 0052, etc.) as well as loop instructions and instruction address calculation (paras. 0056, 0187-188, etc.)], and the instruction sequence including a first matrix multiply instruction [the units execute multiply-accumulate instructions operating on sub-matrices of the DR (paras. 0187-189; fig. 26A; etc.)]; a first decoder to decode the instructions of the instruction sequence including the first matrix multiply instruction [for each processor chip instructions including the multiply-accumulate instruction are decoded by the sequencer to create microinstructions, which are themselves decoded by the decoder (paras. 0237, 0277-279, etc.)], and a first matrix processing unit comprising a first array of processing elements to perform a plurality of parallel fused multiply-accumulate operations in accordance with the first matrix multiply instruction [each core includes a number of neural processing units (NPUs) that perform the instruction in parallel (para. 0071, etc.) including the multiply-accumulate instruction (paras. 0187-189, etc.)] to multiply first data elements of the first matrix by corresponding second data elements of the second matrix to generate a corresponding plurality of products, and to add the plurality of products to corresponding accumulated values to generate a result matrix [each NPU includes multipliers, adders, and an accumulator to multiply elements of the matrices and add the plurality of products to corresponding accumulated values to generate the result (figs. 2, 7, 11; paras. 0112-114, 0146-150, 0187-192; etc.)].
While Henry teaches multiply-accumulate instructions that operate on rows of the matrices (see above), and each row could be considered a matrix in itself, Henry has not been relied upon for the first matrix multiply instruction to specify multiplication of a first matrix by a second matrix, and to indicate a first two-dimensional size associated with the first matrix and a second two-dimensional size associated with the second matrix.
Hansen teaches the first matrix multiply instruction to specify multiplication of a first matrix by a second matrix, and to indicate a first two-dimensional size associated with the first matrix and a second two-dimensional size associated with the second matrix [the EXWIDEMATRIX instruction can be an EXWIDEMATRIX-MULTIPLY instruction which performs matrix-vector or matrix-matrix multiplication, using size and shape parameters from the operand descriptors to indicate the size and shape of the operand matrices (paras. 0040-45, 0052-54, 0058; claim 5; etc.) for execution by a matrix execution functional unit (fig. 7, etc.)].
Henry and Hansen are analogous art, as they are within the same field of endeavor, namely processing units for performing matrix operations.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to utilize a single instruction for performing matrix multiplication, including the sizes of the matrices, and using a matrix functional unit, as taught by Hansen, in the system utilizing multiply-accumulate instructions to perform multiplication and addition for two matrices in the system taught by Henry.
Hansen provides motivation as [by allowing the instruction to specify the size and shape of the matrices, efficiency can be increased, while also allowing more flexibility (paras. 0024-27, etc.)].
As per claim 2, Henry/Hansen teaches wherein the first two-dimensional size associated with the first matrix is different from the second two-dimensional size associated with the second matrix [the EXWIDEMATRIX instruction can be an EXWIDEMATRIX-MULTIPLY instruction which performs matrix-vector or matrix-matrix multiplication, using size and shape parameters from the operand descriptors to indicate the size and shape of the operand matrices (Hansen: paras. 0040-45, 0052-54, 0058; claim 5; etc.)].
As per claim 3, Henry/Hansen teaches wherein the second chip comprises: a second scalar register file to store data related to control flow of a first program or a second program [each processor chip includes media registers, general purpose registers, and control & status registers (Henry: fig. 1, etc.) which can be used for control flow instruction execution, including branches with target addresses, call or return instructions, etc. (Henry: paras. 0045, 0052, etc.)]; second scalar execution circuitry to execute one or more scalar instructions to perform control flow operations related to execution of the second program in accordance with the data related to the control flow of the first program or the second program, the control flow operations including program loops, branches, and address calculations [each processor chip includes a branch unit that can be used for control flow instructions including branches (Henry: paras. 0045, 0052, etc.)]; a second decoder to decode a second matrix multiply instruction, the second matrix multiply instruction to specify multiplication of a third matrix by a fourth matrix, and to indicate a third two-dimensional size associated with the third matrix and a fourth two-dimensional size associated with the fourth matrix [for each processor chip instructions including the multiply-accumulate instruction are decoded by the sequencer to create microinstructions, which are themselves decoded by the decoder (Henry: paras. 0237, 0277-279, etc.); using a an EXWIDEMATRIX-MULTIPLY instruction which performs matrix-vector or matrix-matrix multiplication, including size and shape parameters from the operand descriptors to indicate the size and shape of the operand matrices (Hansen: paras. 0040-45, 0052-54, 0058; claim 5; etc.) for execution by a matrix execution functional unit (Hansen fig. 7, etc.)]; and a second matrix processing unit comprising a second array of processing elements to perform a second plurality of parallel fused multiply-accumulate operations in accordance with the second matrix multiply instruction [each core includes a number of neural processing units (NPUs) that perform the instruction in parallel (Henry: para. 0071, etc.) including the multiply-accumulate instruction (Henry: paras. 0187-189, etc.) using a an EXWIDEMATRIX-MULTIPLY instruction which performs matrix-vector or matrix-matrix multiplication, including size and shape parameters from the operand descriptors to indicate the size and shape of the operand matrices (Hansen: paras. 0040-45, 0052-54, 0058; claim 5; etc.) for execution by a matrix execution functional unit (Hansen fig. 7, etc.)] to multiply third data elements of the third matrix by corresponding fourth data elements of the fourth matrix to generate a second corresponding plurality of products, and to add the second corresponding plurality of products to second corresponding accumulated values to generate a second result matrix [each NPU includes multipliers, adders, and an accumulator to multiply elements of the matrices and add the plurality of products to corresponding accumulated values to generate the result (Henry: figs. 2, 7, 11; paras. 0112-114, 0146-150, 0187-192; etc.); and using an EXWIDEMATRIX-MULTIPLY instruction which performs matrix-vector or matrix-matrix multiplication, including size and shape parameters from the operand descriptors to indicate the size and shape of the operand matrices (Hansen: paras. 0040-45, 0052-54, 0058; claim 5; etc.) for execution by a matrix execution functional unit (Hansen fig. 7, etc.)].
As per claim 4, Henry/Hansen teaches wherein the first data elements comprise convolution input elements [the multiply-accumulate operations perform convolutions on the input and weight matrices (Henry: paras. 0174, 0180, 188-192, etc.)].
As per claim 5, Henry/Hansen teaches wherein the second data elements comprise neural network weights [the multiply-accumulate operations perform convolutions on the input and weight matrices (Henry: paras. 0174, 0180, 188-192, etc.)].
As per claim 6, Henry/Hansen teaches wherein the first matrix multiply instruction comprises a first operand to identify the first data elements and a second operand to identify the second data elements [the EXWIDEMATRIX instruction can be an EXWIDEMATRIX-MULTIPLY instruction which performs matrix-vector or matrix-matrix multiplication, using size and shape parameters from the operand descriptors to indicate the size and shape of the operand matrices (Hansen: paras. 0040-45, 0052-54, 0058; claim 5; etc.) for execution by a matrix execution functional unit (Hansen: fig. 7, etc.)].
As per claim 7, Henry/Hansen teaches wherein the first operand identifies the first data elements in a first one or more registers and the second operand identifies the second data elements in a second one or more registers [the EXWIDEMATRIX instruction can be an EXWIDEMATRIX-MULTIPLY instruction which performs matrix-vector or matrix-matrix multiplication, using size and shape parameters from the operand descriptors to indicate the size and shape of the operand matrices (Hansen: paras. 0040-45, 0052-54, 0058; claim 5; etc.) for execution by a matrix execution functional unit (Hansen: fig. 7, etc.); where the matrices may be stored in dedicated registers (Hansen: para. 0028, etc.)].
As per claim 8, Henry/Hansen teaches an instruction fetch unit to fetch the first matrix multiply instruction [an instruction fetch unit fetches instructions to an instruction cache (Henry: fig. 1, etc.)]; a decoder to decode the first matrix multiply instruction to generate parallel multiply-add operations [for each processor chip instructions including the multiply-accumulate instruction are decoded by the sequencer to create microinstructions, which are themselves decoded by the decoder (Henry: paras. 0237, 0277-279, etc.); using a an EXWIDEMATRIX-MULTIPLY instruction which performs matrix-vector or matrix-matrix multiplication (Hansen: paras. 0040-45, 0052-54, 0058; claim 5; etc.)]; and a scheduler to schedule the parallel multiply-add operations for execution by at least a portion of the array of processing elements [for each processor chip instructions including the multiply-accumulate instruction are decoded by the sequencer to create microinstructions, which are themselves decoded by the decoder, as well as providing scheduling of the operations (Henry: paras. 0047, 0056, 0237, 0277-279, etc.); using a an EXWIDEMATRIX-MULTIPLY instruction which performs matrix-vector or matrix-matrix multiplication (Hansen: paras. 0040-45, 0052-54, 0058; claim 5; etc.)].
As per claim 9, Henry/Hansen teaches wherein the first memory device and the second memory device comprise high bandwidth memory (HBM) devices [the memory subsystem facilitates high bandwidth data transfers (Henry: para. 0058, etc.); making it a high bandwidth memory device].
As per claim 10, Henry/Hansen teaches wherein each of the first parallel compute device and the one or more additional parallel compute devices are to access a system memory using a shared address range [the processor chips can share a memory (Henry: para. 0052, etc.) using a shared memory address space (Hansen: para. 0065; etc.)].
As per claim 11, Henry/Hansen teaches coherency logic to ensure coherency of data in the system memory which is shared between the first parallel compute device and the one or more additional parallel compute devices [the system includes a cache coherency scheme to monitor shared memory and maintain coherency (Hansen: para. 0026; etc.)].
As per claim 12, Henry/Hansen teaches wherein the first program comprises one or more machine learning tasks, wherein separate portions of the one or more machine learning tasks are to be executed by the first parallel compute device and the one or more additional parallel compute devices [the multiply-accumulate operations perform convolutions on the input and weight matrices for a convolutional neural network (Henry: paras. 0174, 0180, 188-192, 0201, etc.)].
As per claim 13, Henry/Hansen teaches an apparatus comprising: a compute processor package including a first die and a second die, the first die coupled to a first memory device and the second die coupled to a second memory device [a processing core connected to multiple memory systems (Henry: fig. 1, etc.) which is one of multiple processing cores in a multi-core processor connected to individual and shared memory (Henry: para. 0052, etc.) and where each core is on its own chip or die (Henry: para. 0044, etc.); where the multi-core processor is the first parallel compute device comprising the plurality of chips, where each core is a chip connected to a memory device]; a first interconnect to couple the first die and the second die [the cores may be connected by a bus interface and busses to shared cache and system memory (Henry: para. 0052; fig. 1; etc.)]; and an interface to couple the compute processor package to one or more additional compute processor packages [the cores may be connected by a bus interface and busses to shared cache and system memory (Henry: para. 0052; fig. 1; etc.)]; at least the first die comprising: a first scalar register file to store data related to control flow operations [each processor chip includes media registers, general purpose registers, and control & status registers (Henry: fig. 1, etc.) which can be used for control flow instruction execution, including branches with target addresses, call or return instructions, etc. (Henry: paras. 0045, 0052, etc.)]; first scalar execution circuitry to execute one or more scalar instructions to perform control flow operations in accordance with the data to control execution of an instruction sequence, the control flow operations including program loops, branches, and address calculations [each processor chip includes a branch unit that can be used for control flow instructions including branches (Henry: paras. 0045, 0052, etc.)], and the instruction sequence including a first matrix multiply instruction [the units execute multiply-accumulate instructions operating on sub-matrices of the DR (Henry: paras. 0187-189; fig. 26A; etc.)]; a first decoder to decode the instructions of the instruction sequence including the first matrix multiply instruction [for each processor chip instructions including the multiply-accumulate instruction are decoded by the sequencer to create microinstructions, which are themselves decoded by the decoder (Henry: paras. 0237, 0277-279, etc.)], the first matrix multiply instruction to specify multiplication of a first matrix by a second matrix, and to indicate a first two-dimensional size associated with the first matrix and a second two-dimensional size associated with the second matrix [using an EXWIDEMATRIX-MULTIPLY instruction which performs matrix-vector or matrix-matrix multiplication, including size and shape parameters from the operand descriptors to indicate the size and shape of the operand matrices (Hansen: paras. 0040-45, 0052-54, 0058; claim 5; etc.) for execution by a matrix execution functional unit (Hansen fig. 7, etc.)]; and a first matrix processing unit comprising a first array of processing elements to perform a plurality of parallel fused multiply-accumulate operations in accordance with the first matrix multiply instruction [each core includes a number of neural processing units (NPUs) that perform the instruction in parallel (Henry: para. 0071, etc.) including the multiply-accumulate instruction (Henry: paras. 0187-189, etc.) for execution by a matrix execution functional unit (Hansen fig. 7, etc.)] to multiply first data elements of the first matrix by corresponding second data elements of the second matrix to generate a corresponding plurality of products, and to add the plurality of products to corresponding accumulated values to generate a result matrix [each NPU includes multipliers, adders, and an accumulator to multiply elements of the matrices and add the plurality of products to corresponding accumulated values to generate the result (Henry: figs. 2, 7, 11; paras. 0112-114, 0146-150, 0187-192; etc.); using an EXWIDEMATRIX-MULTIPLY instruction which performs matrix-vector or matrix-matrix multiplication, including size and shape parameters from the operand descriptors to indicate the size and shape of the operand matrices (Hansen: paras. 0040-45, 0052-54, 0058; claim 5; etc.) for execution by a matrix execution functional unit (Hansen fig. 7, etc.)].
Examiner’s Note: the reasoning and motivation for the combination is provided in claims 1 and 2, above.
As per claim 14, see the rejection of claim 3, above.
As per claim 15, see the rejection of claim 4, above.
As per claim 16, see the rejection of claim 5, above.
As per claim 17, see the rejection of claim 6, above.
As per claim 18, see the rejection of claim 7, above.
As per claim 19, see the rejection of claim 8, above.
As per claim 20, see the rejection of claim 9, above.
As per claim 21, see the rejection of claim 10, above.
As per claim 22, see the rejection of claim 11, above.
As per claim 23, see the rejection of claim 12, above.
Response to Arguments
Applicant's arguments filed 18 June 2025 have been fully considered but they are not persuasive.
Applicant argues that the combination of Henry with Hansen would change the principle of operation of the system taught by Henry, which performs multiply-accumulate operations for a number of instructions.
However, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to utilize a single instruction for performing matrix multiplication, including the sizes of the matrices, and using a matrix functional unit, as taught by Hansen, in the system utilizing multiply-accumulate instructions to perform multiplication and addition for two matrices in the system taught by Henry, as
Hansen teaches that, by allowing the instruction to specify the size and shape of the matrices, efficiency can be increased, while also allowing more flexibility (paras. 0024-27, etc.)]. Replacing an element of Henry with the element of Hansen to perform the same operations (matrix multiply-accumulate operations) would not change the principle of operation of the system taught by Henry.
Additionally, even assuming, arguendo, that modifying the system of Henry to replace the multi-instruction operations of the NPUs with a single controlling instruction would change the principle of operation of the system, Henry also teaches that the instructions being performed in the NPUs are microinstructions translated from a single (macro) instruction, to perform the operations of that instruction (see, e.g., Henry: paras. 0046-47). Therefore, replacing the single instruction being translated (taught by Henry) with the EXWIDEMATRIX-MULTIPLY instruction which performs matrix-vector or matrix-matrix multiplication, using size and shape parameters from the operand descriptors (taught by Hansen), would still be within the broadest reasonable interpretation of the claimed first matrix processing unit comprising an array of processing elements to perform a plurality of parallel fused multiply-accumulate operations (the microinstructions) in accordance with the first matrix multiple instruction (EXWIDEMATRIX-MULTIPLY).
Conclusion
The following is a summary of the treatment and status of all claims in the application as recommended by M.P.E.P. 707.07(i): claims 1-23 are rejected.
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Eichenberger (US 2011/0055517) – discloses fused multiply-add instructions and units for matrix multiplication including complex matrix multiplication.
Brothers (US 2017/0011288) – discloses a matrix multiply unit array for performing convolutions.
Nasiri et al. (Modified Fused Multiply-Accumulate Chained Unit, Sept 2014, pgs. 889-892) – discloses pipelined fused multiply add units forming a multiply-accumulation chained unit.
Abdallah (US 2016/0026486) – discloses decoding macro instructions into micro instructions and optimizing micro instructions.
Hancock (US 2013/0268794) – discloses decoding macro instructions into micro instructions and optimizing micro instructions.
Hansen (US 7,843,459) – discloses a matrix multiplication instruction including indicating the size of the matrix elements to be multiplied.
Ginzburg (US 2011/0153707) – discloses using a single matrix multiply instruction.
Ge (US 2014/0237010) – discloses a matrix multiplication instruction.
Shifer (US 2014/0089635) – discloses a multicore processor including matrix multiplication logic as well as decoding macroinstructions into multiple micro-instructions.
Gopal (US 2014/0006753) – discloses a matrix multiply accumulate instruction, as well as decoding macroinstructions into micro-operations.
Kojima (US 5,038,312) – discloses a data processing system including decoding macro-instructions into micro-instructions for matrix-vector multiplication.
The examiner requests, in response to this Office action, that support be shown for language added to any original claims on amendment and any new claims. That is, indicate support for newly added claim language by specifically pointing to page(s) and line number(s) in the specification and/or drawing figure(s). This will assist the examiner in prosecuting the application.
When responding to this office action, Applicant is advised to clearly point out the patentable novelty which he or she thinks the claims present, in view of the state of the art disclosed by the references cited or the objections made. He or she must also show how the amendments avoid such references or objections. See 37 CFR 1.111(c).
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GEORGE GIROUX whose telephone number is (571)272-9769. The examiner can normally be reached M-F 10am-6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Omar Fernandez Rivas can be reached on 571-272-2589. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GEORGE GIROUX/Primary Examiner, Art Unit 2128