DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This Action is non-final and is in response to the claims filed 12/09/2022. Claims 1-20 are currently pending, of which claims 1-20 are currently rejected.
Drawings
The drawings are objected to because Fig. 14 has the arrow going through the text on step 1410. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 7, 14, and 20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 7 recites the limitation “The system of claim 1, wherein generating the recommendation comprises: generating an image using the MMA computations; generating text using the MMA computations; or generating software code using the MMA computations.” It is unclear if applicant intends the recommendation to be one of the listed elements, or more than one of the listed elements. Appropriate correction is required. For purposes of prior art rejections, it will be interpreted as only one.
Claims 14 ad 20 recite the same limitations, and are rejected for the same reasons as claim 7.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Ware (U.S. Patent No.: US 11960856 B1), hereinafter “Ware”, in view of Gore et al. (U.S. Patent Application Publication No.: US 20220107805 A1), hereinafter “Gore”, further in view of Shivam et al. (U.S. Patent Application Publication No.: US 20230297643 A1), hereinafter “Shivam”.
Regarding Claim 1, Ware teaches:
A system comprising:
a processor (Column 7 Lines 13-31, e.g., processor generates filter and weights, and converts data format of weights to be processed by multiplier-accumulator processing circuitry); and
a computer-readable medium storing instructions that are operative upon execution by the processor (Column 35 Lines 59-65, e.g., conversion circuitry may receive instruction signals from one or more data storage elements) to:
copy data values in a first floating point format … to shared memory (Fig. 5B, e.g., weights/coefficients are sent (copied) to L1 memory (shared memory); Column 11 Lines 40-44, e.g., L1 memory is shared by each of the MAC execution pipelines);
load [data] of the first floating point data values from the shared memory into a set of registers (Fig. 5B, e.g., Data is inputted (loaded) from L1 memory to Data Format Conversion Circuitry; Fig. 6A, e.g., shows Data Format Conversion Circuitry including top registers storing input values (first floating point data values));
convert the first floating point data values in the set of registers to second floating point data values in a second floating point format (Fig. 6A, e.g., shows Data Format Conversion Circuitry that receives GFP8 (first floating point data values) in top registers, then converts them to FP16 (second floating point data values in second floating point format));
submit the second floating point data values to a [plurality of multiplier-accumulator circuits] (Fig. 5B, e.g., Data Format Conversion Circuitry outputs data to L0 memory, then to the plurality of multiplier-accumulator circuits; Column 12 Line 46 – Column 13 Line 4);
perform, with the [plurality of multiplier-accumulator circuits], matrix multiply accumulate (MMA) computations (Column 12 Line 46 – Column 13 Line 4, e.g., plurality of multiplier-accumulator circuits perform multiply and accumulate operations);
and generate a recommendation using the MMA computations (Column 18 Lines 11-16, e.g., multiply-accumulator processes filtering of image data).
Ware does not teach:
copy data values in a first floating point format from global memory to shared memory;
load thread block tiles of the first floating point data values from the shared memory into a set of registers;
submit the second floating point data values to a tensor core;
perform, with the tensor core, matrix multiply accumulate (MMA) computations;
However, Gore teaches:
copy data values in a first floating point format from global memory to shared memory (Fig. 1, e.g., shows load/store unit reading and writing data from data memory 122 (global memory) to registers; Abstract, e.g., computations are performed on floating point values in a floating point format);
Ware does not show where the input data to L1 memories comes from. Therefore, it would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to which said subject matter pertains to combine the data memory as taught by Gore with the multiplier-accumulator circuit (MAC) execution processing pipelines as taught by Ware. One would have been motivated to combine these references because both references disclose multiply-accumulate operations using floating point values of a first and second formats, and Gore enhances the model of Ware by allowing for input data to L1 memory to be stored and provided from a data memory.
Ware in view of Gore do not teach:
load thread block tiles of the first floating point data values from the shared memory into a set of registers;
submit the second floating point data values to a tensor core;
perform, with the tensor core, matrix multiply accumulate (MMA) computations;
However, in the same field of endeavor, Shivam teaches using tensor cores to receive thread block tiles to perform multiply accumulate operations. Shivam explains “The kernel may define a number of thread blocks configured to execute a number of threads in parallel to process different data using a similar set of instructions. In addition, the threads can include instructions configured for specialized logic units such as a load/store unit or a tensor core” (Shivam: 0034), and “The threads may cause the parallel processor 204 to launch a matrix multiplication operation to be executed by a tensor core” (Shivam: ¶0059).
Therefore, it would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to which said subject matter pertains to substitute the plurality of multiplier-accumulator circuits to perform multiply-accumulate operations as taught by Ware in view of Gore with the tensor core receiving thread tiles as taught by Shivam. One would have been motivated to combine these references because both references disclose floating-point multiply-accumulate operations on matrices, and Shivam enhances the model of Ware in view of Gore because “tensor cores extend the idea of a vector unit to matrices enabling a number of dot products to be calculated in parallel. These units dramatically speed up these types of matrix calculations compared to implementing the operations on a conventional floating-point arithmetic logic unit.” (Shivam: ¶0002)
Regarding Claim 2, Ware in view of Gore in view of Shivam teach:
The system of claim 1, wherein the first floating point format comprises 8-bit floating point (FP8) format (Ware: Column 13 Lines 5-12, e.g., Data Format Conversion Circuitry receives Gaussian FP8 floating point format);
wherein the second floating point format comprises 16-bit floating point (FP 16) format (Ware: Column 13 Lines 5-12, e.g., Data Format Conversion Circuitry outputs FP16); and
wherein results of the MMA computations comprise 16-bit FP16 data values or 32-bit floating point (FP32) data values (Ware: Column 13 Lines 5-12, e.g., Data Format Conversion Circuitry outputs FP16; Fig. 5B, e.g., plurality of multiplier-accumulator circuits perform multiply and accumulate operations using data output from data format conversion circuitry).
Regarding Claim 3, Ware in view of Gore in view of Shivam teach:
The system of claim 1, wherein loading the thread block tiles of the first floating point data values from the shared memory into the set of registers comprises loading four data values in the first floating point format from the shared memory into a single register of the set of registers (Fig. 5B, e.g., L1 memory (shared memory) outputs data to Data Format Conversion Circuitry; Fig. 6A, e.g., shows Data Format Conversion Circuitry receiving 4 fractional bits (data values) in top register F[1:4]).
Regarding Claim 4, Ware in view of Gore in view of Shivam teach:
The system of claim 3, wherein the instructions are further operative to:
load the thread block tiles of the first floating point data values from the shared memory into the set of registers while performing MMA computations on prior converted data values (Ware: Column 3 Lines 30-46, e.g., conversion of filter weights and multiplication-accumulation operations are performed in a pipeline. Therefore, pipelining causes for multiplier-accumulator operations to be performed using prior converted data values while data of the next cycle is being converted; Shivam: ¶0034, e.g., threads cause the parallel processor 204 to launch a matrix multiplication operation to be executed by a tensor core).
The motivation to combine provided with respect to claim 1 applies equally to claim 4.
Regarding Claim 5, Ware in view of Gore in view of Shivam teach the system of claim 1. They do not teach:
wherein the instructions are further operative to:
prior to submitting data values to the tensor core, shift data positions of the second floating point data values to a layout accepted by the tensor core concurrently with converting the first floating point data values to the second floating point data values.
However, in the same field of endeavor, Gore further teaches floating-point logic that shifts the mantissa of operands in order form the to be represented on the same scale. Gore explains “the floating point logic 112 bit-shifts the mantissa of one or both of the source operands left or right so as to represent them on the same scale (i.e. with the same exponent), and then adds the mantissas.” (Gore: ¶0067)
Therefore, it would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to which said subject matter pertains to combine the floating-point logic to shift mantissas on operands as taught by Gore with the integrated circuit including MAC processing circuitry as taught by Ware in view of Gore in view of Shivam. One would have been motivated to combine these references because both references disclose multiply-accumulate operations using floating point values of a first and second formats, and Gore enhances the model of Ware in view of Gore in view of Shivam by shifting mantissas of operands “so as to represent them on the same scale”. (Gore: ¶0067)
Regarding Claim 6, Ware in view of Gore in view of Shivam teach:
The system of claim 1, wherein the instructions are further operative to:
… copy data values from the global memory to the shared memory, load data values into the set of registers, convert data value format, and submit data values to the tensor core, … (See claim mapping of claim 1, where these steps are taught by the combination of Ware in view of Gore in view of Shivam).
Shivam further teaches:
continue to copy data values from the global memory to the shared memory, load data values into the set of registers, convert data value format, and submit data values to the tensor core, until MMA computations for matrices in the global memory are complete (Shivam: ¶0040, e.g., process is repeated until all k elements of matrices A and B are completed).
It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to which said subject matter pertains to modify the process of copying data values from data memory, performing floating point conversion, and performing multiply-accumulate operations as taught by Ware in view of Gore in view of Shivam to finish computations when the processing of both input and weight matrices are complete as taught by Shivam. One would have been motivated to combine these references because both references disclose floating-point multiply-accumulate operations on matrices, and Shivam enhances the model of Ware in view of Gore in view of Shivam by allowing for the completion of matrix data processing by repeating floating-point conversions and multiply-accumulate operations.
Regarding Claim 7, Ware in view of Gore in view of Shivam teach:
The system of claim 1, wherein generating the recommendation comprises:
generating an image using the MMA computations (Column 29 Lines 48-58, e.g., image data is processed);
generating text using the MMA computations; or
generating software code using the MMA computations.
Regarding Claims 8-14, they are method claims practiced by the apparatus of claims 1-7. They are rejected for the same reasons as claims 1-7.
Regarding Claims 15-20, they are similar claims to the apparatus of claims 1-7, wherein all claim limitations also have been addressed and/or covered in cited areas. They are rejected for the same reasons as claims 1-7.
Prior Art Made of Record
US 20200341759 A1 – teaches converting half precision floating point values to single precision floating point values. See Fig. 12 and corresponding description. This teaching is pertinent to the conversion from fp8 to fp16 of the instant application disclosed in Fig. 3 and corresponding description.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CARLOS H DE LA GARZA whose telephone number is (571)272-0474. The examiner can normally be reached Monday-Friday 9:30AM-6PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Caldwell can be reached at (571) 272-3702. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/C.H.D./
Carlos H. De La GarzaExaminer, Art Unit 2182
/EMILY E LAROCQUE/Primary Examiner, Art Unit 2182
Prosecution Insights