DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 08/23/2023, 02/16/2024, 08/21/2024, 09/02/2025, 02/23/2026, and 05/12/2026 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Specification
The disclosure is objected to because of the following informalities:
The appropriate patent application numbers should be indicated in the blank spaces in [0002]
Appropriate correction is required.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 9, and 15 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Choquette US 2023/0289398.
The applied reference has a common assignee with the instant application. Based upon the earlier effectively filed date of the reference, it constitutes prior art under 35 U.S.C. 102(a)(2). This rejection under 35 U.S.C. 102(a)(2) might be overcome by: (1) a showing under 37 CFR 1.130(a) that the subject matter disclosed in the reference was obtained directly or indirectly from the inventor or a joint inventor of this application and is thus not prior art in accordance with 35 U.S.C. 102(b)(2)(A); (2) a showing under 37 CFR 1.130(b) of a prior public disclosure under 35 U.S.C. 102(b)(2)(B) if the same invention is not being claimed; or (3) a statement pursuant to 35 U.S.C. 102(b)(2)(C) establishing that, not later than the effective filing date of the claimed invention, the subject matter disclosed in the reference and the claimed invention were either owned by the same person or subject to an obligation of assignment to the same person or subject to a joint research agreement.
Regarding claim 1, Choquette teaches:
1. A processor comprising:
one or more circuits to perform an instruction to indicate whether one or more matrix multiply-accumulate (MMA) memory instructions have completed ([0104]: the MMA synchronization instruction (i.e., an instruction) indicates completion of previous MMA instructions (which are one or more MMA memory instructions in that they operate on data in memory, see also [0106] describing a GMMA instruction that reads from memory)).
Claim 9 is directed to a system corresponding to the processor of claim 1 and is rejected for the same reasons as claim 1.
Claim 15 is directed to a method performing steps corresponding to the processor of claim 1 and is rejected for the same reasons as claim 1.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5, 9-12, and 14-18 are rejected under 35 U.S.C. 103 as being unpatentable over Edwards US 2021/0294673 (cited in 02/23/2026 IDS) in view of Sade US 2019/0042254.
Regarding claim 1, Edwards teaches:
1. A processor comprising:
one or more circuits ([0045]: the GPU executing process 102) to perform an instruction to indicate whether copy operations have completed ([0049] discloses that the synchronization 114 may be a function call/instruction and that it may copy data in shared memory to the threads and [0050] discloses that once data values are synchronized from shared memory the threads perform operations on the shared data values, these portions indicate that the synchronization indicates to the threads when the copy has completed for the threads to know when to start performing operations on the shared data values).
Edwards does not teach one or more matrix multiply-accumulate (MMA) memory instructions for copying the data.
However, Sade teaches one or more matrix multiply-accumulate (MMA) memory instructions ([0056] discloses that tile may be loaded from memory using load operations/instructions and [0053] discloses that each tile may be acted upon by tile multiplication and accumulation operations, which indicates that the load operations may be for matrix multiply accumulate (MMA) operations, i.e., the load operations are MMA memory instructions)
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Edwards to copy/load data from memory using load operations as taught by Sade to support matrix multiply and accumulate (MMA) operations such that the combination may copy/load data for matrix multiply-accumulate operations. One of ordinary skill in the art would have been motivated to modify Edwards to support MMA operations to increase the types of operations that may be performed, which would improve flexibility of the system. One of ordinary skill in the art would have been motivated to modify Edwards to load data for the matrix multiply and accumulate operations using load/memory instructions because using load instructions is a known technique on the known device of a processor for loading data from memory and would yield the predictable result of improving control over loading data from memory.
Regarding claim 2, Edwards in view of Sade teaches:
2. The processor of claim 1, wherein the instruction is to cause the one or more circuits to issue one or more MMA operations in response to completion of the one or more MMA memory instructions (Edwards [0049]-[0050]: the synchronization instruction copies data (using MMA memory instructions in the combination) from memory to the threads, which causes the threads to perform/issue operations (MMA operations, in the combination) on the data once the data is synchronized (i.e., in response to completing the memory instructions that copy the data)).
Regarding claim 3, Edwards in view of Sade teaches:
3. The processor of claim 1, wherein the one or more circuits are to issue one or more MMA operations to be performed by one or more accelerators in response to completion of the one or more MMA memory instructions (Edwards [0049]-[0050]: the synchronization instruction copies data (using MMA memory instructions in the combination) from memory to the threads, which causes the threads to perform/issue operations (MMA operations, in the combination) on the data once the data is synchronized (i.e., in response to completing the memory instructions that copy the data), any execution resource of the GPU that executes the operations of the thread after the synchronization is an accelerator).
Regarding claim 4, Edwards in view of Sade teaches:
4. The processor of claim 1, wherein the one or more MMA memory instructions are to cause the one or more circuits to copy one or more operands in memory to be used by one or more MMA operations (Edwards [0049]: synchronization 114 copies (using load/MMA memory instructions, in the combination) data in shared memory to the threads, and the data may be operands as Edwards [0050] discloses that the threads may perform operations (i.e., MMA operations, in the combination) on the data).
Regarding claim 5, Edwards in view of Sade teaches:
5. The processor of claim 1, wherein the one or more MMA memory instructions are to cause the one or more circuits to copy one or more fragments of one or more operands in memory to be used by one or more MMA operations (Edwards [0049]: synchronization 114 copies (using load/MMA memory instructions, in the combination) data in shared memory to the threads, and the data may be operands (which would include fragments of the operands under BRI) as Edwards [0050] discloses that the threads may perform operations (i.e., MMA operations, in the combination) on the data).
Regarding claim 9, Edwards teaches:
9. A system comprising:
one or more processors ([0045]: the GPU executing process 102) to perform an instruction to indicate whether copy operations have completed ([0049] discloses that the synchronization 114 may be a function call/instruction and that it may copy data in shared memory to the threads and [0050] discloses that once data values are synchronized from shared memory the threads perform operations on the shared data values, these portions indicate that the synchronization indicates to the threads when the copy has completed for the threads to know when to start performing operations on the shared data values).
Edwards does not teach one or more matrix multiply-accumulate (MMA) memory instructions for copying the data.
However, Sade teaches one or more matrix multiply-accumulate (MMA) memory instructions ([0056] discloses that tile may be loaded from memory using load operations/instructions and [0053] discloses that each tile may be acted upon by tile multiplication and accumulation operations, which indicates that the load operations may be for matrix multiply accumulate (MMA) operations, i.e., the load operations are MMA memory instructions)
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Edwards to copy/load data from memory using load operations as taught by Sade to support matrix multiply and accumulate (MMA) operations such that the combination may copy/load data for matrix multiply-accumulate operations. One of ordinary skill in the art would have been motivated to modify Edwards to support MMA operations to increase the types of operations that may be performed, which would improve flexibility of the system. One of ordinary skill in the art would have been motivated to modify Edwards to load data for the matrix multiply and accumulate operations using load/memory instructions because using load instructions is a known technique on the known device of a processor for loading data from memory and would yield the predictable result of improving control over loading data from memory.
Regarding claim 10, Edwards in view of Sade teaches:
10. The system of claim 9, wherein the one or more processors are to issue, in response to the instruction, an MMA operation (Edwards [0049]-[0050]: the synchronization instruction copies data (using MMA memory instructions in the combination) from memory to the threads, which causes the threads to perform/issue operations (MMA operations, in the combination) on the data once the data is synchronized (i.e., in response to completing the memory instructions that copy the data))
Edwards in view of Sade does not teach:
the MMA operation to be performed by one or more streaming multiprocessors (SMs).
However, a further embodiment of Edwards teaching streaming multiprocessors including cores configured to perform matrix multiply and accumulate operations, see [0230]-[0231] and Figs. 23-24.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Edwards in view of Sade to perform the MMA operation by one or more streaming multiprocessors as taught by the further embodiment of Edwards. One of ordinary skill in the art would have been motivated to make this modification improve performance since by providing more processing resources to process MMA operations.
Regarding claim 11, Edwards in view of Sade teaches:
11. The system of claim 9, wherein the instruction is to cause the one or more processors to issue one or more MMA operations in response to completion of the one or more MMA memory instructions (Edwards [0049]-[0050]: the synchronization instruction copies data (using MMA memory instructions in the combination) from memory to the threads, which causes the threads to perform/issue operations (MMA operations, in the combination) on the data once the data is synchronized (i.e., in response to completing the memory instructions that copy the data)).
Regarding claim 12, Edwards in view of Sade teaches:
12. The system of claim 9, wherein the one or more MMA memory instructions are to cause the one or more processors to copy one or more operands in memory to be used by one or more MMA operations (Edwards [0049]: synchronization 114 copies (using load/MMA memory instructions, in the combination) data in shared memory to the threads, and the data may be operands as Edwards [0050] discloses that the threads may perform operations (i.e., MMA operations, in the combination) on the data).
Regarding claim 14, Edwards in view of Sade teaches:
14. The system of claim 9, wherein the one or more processors are graphics processing units (GPUs) (Edwards [0045]: the GPU executing process 102).
Regarding claim 15, Edwards teaches:
15. A method comprising:
performing an instruction to indicate whether copy operations have completed ([0049] discloses that the synchronization 114 may be a function call/instruction and that it may copy data in shared memory to the threads and [0050] discloses that once data values are synchronized from shared memory the threads perform operations on the shared data values, these portions indicate that the synchronization indicates to the threads when the copy has completed for the threads to know when to start performing operations on the shared data values).
Edwards does not teach one or more matrix multiply-accumulate (MMA) memory instructions for copying the data.
However, Sade teaches one or more matrix multiply-accumulate (MMA) memory instructions ([0056] discloses that tile may be loaded from memory using load operations/instructions and [0053] discloses that each tile may be acted upon by tile multiplication and accumulation operations, which indicates that the load operations may be for matrix multiply accumulate (MMA) operations, i.e., the load operations are MMA memory instructions)
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Edwards to copy/load data from memory using load operations as taught by Sade to support matrix multiply and accumulate (MMA) operations such that the combination may copy/load data for matrix multiply-accumulate operations. One of ordinary skill in the art would have been motivated to modify Edwards to support MMA operations to increase the types of operations that may be performed, which would improve flexibility of the system. One of ordinary skill in the art would have been motivated to modify Edwards to load data for the matrix multiply and accumulate operations using load/memory instructions because using load instructions is a known technique on the known device of a processor for loading data from memory and would yield the predictable result of improving control over loading data from memory.
Regarding claim 16, Edwards in view of Sade teaches:
16. The method of claim 15, further comprising issuing one or more MMA operations in response to the instruction (Edwards [0049]-[0050]: the synchronization instruction copies data (using MMA memory instructions in the combination) from memory to the threads, which causes the threads to perform/issue operations (MMA operations, in the combination) on the data once the data is synchronized (i.e., in response to completing the memory instructions that copy the data)).
Regarding claim 17, Edwards in view of Sade teaches:
17. The method of claim 15, further comprising issuing, in response to the instruction, an MMA operation (Edwards [0049]-[0050]: the synchronization instruction copies data (using MMA memory instructions in the combination) from memory to the threads, which causes the threads to perform/issue operations (MMA operations, in the combination) on the data once the data is synchronized (i.e., in response to completing the memory instructions that copy the data))
Edwards in view of Sade does not teach:
the MMA operation to be performed by one or more streaming multiprocessors (SMs).
However, a further embodiment of Edwards teaching streaming multiprocessors including cores configured to perform matrix multiply and accumulate operations, see [0230]-[0231] and Figs. 23-24.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Edwards in view of Sade to perform the MMA operation by one or more streaming multiprocessors as taught by the further embodiment of Edwards. One of ordinary skill in the art would have been motivated to make this modification improv
Regarding claim 18, Edwards in view of Sade teaches:
18. The method of claim 15, further comprising causing one or more processors (Edwards [0045]: the GPU executing process 102) to copy, in response to the instruction, one or more operands in memory to be used by one or more MMA operations (Edwards [0049]: synchronization 114 copies (using load/MMA memory instructions, in the combination) data in shared memory to the threads, and the data may be operands as Edwards [0050] discloses that the threads may perform operations (i.e., MMA operations, in the combination) on the data).
Claims 6-7 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Edwards US 2021/0294673 in view of Sade US 2019/0042254 and Giroux US 2021/0124627.
Regarding claim 6, Edwards in view of Sade teaches:
6. The processor of claim 1, wherein the one or more MMA memory instructions are to cause the one or more circuits to copy one or more fragments of one or more operands in memory to be used by one or more MMA operations (Edwards [0049]: synchronization 114 copies (using load/MMA memory instructions, in the combination) data in shared memory to the threads, and the data may be operands (which would include fragments of the operands under BRI) as Edwards [0050] discloses that the threads may perform operations (i.e., MMA operations, in the combination) on the data).
Edwards in view of Sade does not teach one or more asynchronous MMA operations.
However, Giroux teaches asynchronous compute operations, see [0010].
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the MMA operations of Edwards in view of Sade to be asynchronous MMA operations as suggested by Giroux. One of ordinary skill in the art would have been motivated to make this modification to increase GPU utilization (Giroux [0010]).
Regarding claim 7, Edwards in view of Sade teaches:
7. The processor of claim 1, wherein the one or more MMA memory instructions are to cause the one or more circuits to copy one or more fragments of one or more operands in memory to be used by one or more MMA operations to be performed by one or more accelerators (Edwards [0049]-[0050]: the synchronization instruction copies data (using MMA memory instructions in the combination) from memory to the threads, which causes the threads to perform/issue operations (MMA operations, in the combination) on the data once the data is synchronized (i.e., in response to completing the memory instructions that copy the data), any execution resource of the GPU that executes the operations of the thread after the synchronization is an accelerator).
Edwards in view of Sade does not teach one or more asynchronous MMA operations.
However, Giroux teaches asynchronous compute operations, see [0010].
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the MMA operations of Edwards in view of Sade to be asynchronous MMA operations as suggested by Giroux. One of ordinary skill in the art would have been motivated to make this modification to increase GPU utilization (Giroux [0010]).
Regarding claim 20, Edwards in view of Sade teaches:
20. The method of claim 15, further comprising copying, in response to the instruction, one or more fragments of one or more operands in memory to be used by one or more MMA operations (Edwards [0049]: synchronization 114 copies (using load/MMA memory instructions, in the combination) data in shared memory to the threads, and the data may be operands (which would include fragments of the operands under BRI) as Edwards [0050] discloses that the threads may perform operations (i.e., MMA operations, in the combination) on the data).
Edwards in view of Sade does not teach one or more asynchronous MMA operations.
However, Giroux teaches asynchronous compute operations, see [0010].
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the MMA operations of Edwards in view of Sade to be asynchronous MMA operations as suggested by Giroux. One of ordinary skill in the art would have been motivated to make this modification to increase GPU utilization (Giroux [0010]).
Claims 8, 13, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Edwards US 2021/0294673 in view of Sade US 2019/0042254 and Hughes US 2022/0100513.
Regarding claim 8, Edwards in view of Sade teaches:
8. The processor of claim 1, wherein completion of the one or more MMA memory instructions is to cause the one or more circuits to issue one or more MMA operations to be performed on input operands (Edwards [0049]-[0050]: the synchronization instruction copies data (using MMA memory instructions in the combination) from memory to the threads, which causes the threads to perform/issue operations (MMA operations, in the combination) on the data/input operands once the data is synchronized (i.e., in response to completing the memory instructions that copy the data))
Edwards in view of Sade does not teach:
input operands with a shape and a type indicated to the instruction.
However, Hughes teaches a determining the amount of data to load by a load instruction using a configuration that indicates the number of rows and columns (i.e., a shape) and the number of bytes per row/column (i.e., a type), see [0174].
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the synchronization of Edwards to receive indication of the numbers of rows/columns and number of bytes per row/column (i.e., a shape and type) to load/copy from memory as suggested by Hughes. One of ordinary skill in the art would have been motivated to make this modification to allow for more precious loading of data.
Regarding claim 13, Edwards in view of Sade teaches:
13. The system of claim 9, wherein completion of the one or more MMA memory instructions is to cause the one or more circuits to issue one or more MMA operations to be performed on input operands (Edwards [0049]-[0050]: the synchronization instruction copies data (using MMA memory instructions in the combination) from memory to the threads, which causes the threads to perform/issue operations (MMA operations, in the combination) on the data/input operands once the data is synchronized (i.e., in response to completing the memory instructions that copy the data))
Edwards in view of Sade does not teach:
input operands with a shape and a type indicated to the instruction.
However, Hughes teaches a determining the amount of data to load by a load instruction using a configuration that indicates the number of rows and columns (i.e., a shape) and the number of bytes per row/column (i.e., a type), see [0174].
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the synchronization of Edwards to receive indication of the numbers of rows/columns and number of bytes per row/column (i.e., a shape and type) to load/copy from memory as suggested by Hughes. One of ordinary skill in the art would have been motivated to make this modification to allow for more precious loading of data.
Regarding claim 19, Edwards in view of Sade teaches:
19. The method of claim 15, wherein completion of the one or more MMA memory instructions is to cause one or more MMA operations to be issued based, at least in part, on operands (Edwards [0049]-[0050]: the synchronization instruction copies data (using MMA memory instructions in the combination) from memory to the threads, which causes the threads to perform/issue operations (MMA operations, in the combination) on/based on the data/input operands once the data is synchronized (i.e., in response to completing the memory instructions that copy the data))
Edwards in view of Sade does not teach:
operands with a shape and a type indicated to the instruction.
However, Hughes teaches a determining the amount of data to load by a load instruction using a configuration that indicates the number of rows and columns (i.e., a shape) and the number of bytes per row/column (i.e., a type), see [0174].
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the synchronization of Edwards to receive indication of the numbers of rows/columns and number of bytes per row/column (i.e., a shape and type) to load/copy from memory as suggested by Hughes. One of ordinary skill in the art would have been motivated to make this modification to allow for more precious loading of data.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 2006/0101062 teaches executing commands asynchronously as resources become available, see [0030]
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KASIM ALLI whose telephone number is (571)270-1476. The examiner can normally be reached Monday - Friday 9am 5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jyoti Mehta can be reached at (571) 270-3995. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KASIM ALLI/Examiner, Art Unit 2183 /JYOTI MEHTA/Supervisory Patent Examiner, Art Unit 2183
Prosecution Insights