DETAILED ACTION
Claims 1-20 are pending.
The office acknowledges the following papers:
Power of attorney filed on 3/25/2025,
International application status report filed on 5/28/2026.
Allowable Subject Matter
Claim 12 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Priority
The effective filing date for the subject matter defined in the pending claims in this application is 9/14/2022.
Drawings
The Examiner contends that the drawings submitted on 3/11/2025 are acceptable for examination proceedings.
Specification
The disclosure is objected to because of the following informalities:
The lengthy specification has not been checked to the extent necessary to determine the presence of all possible minor errors. The Applicant’s cooperation is requested in correcting any errors of which the Applicant may become aware.
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed. The following title is suggested: “Multiple-Outer-Product Instruction using compressed and uncompressed source vector operands”.
Appropriate correction is required.
Claim Objections
Claim 10 is objected to because of the following informalities:
Claim 10 is objected to under 37 CFR 1.75(c) as being in improper form because a multiple dependent claim should refer to other claims in the alternative only. See MPEP § 608.01(n). Accordingly, the claim has not been further treated on the merits.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 20 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without either reciting additional elements that integrate the judicial exception into a practical application or reciting additional elements that amount to significantly more than the judicial exception. Independent claim 20 recites a mental process of storing a computer program. All of the claims are directed towards a process, machine manufacture, or a composition of matter.
The storing a computer program is a process that, under its broadest reasonable interpretation, covers a mental process of the mind with the aid of pen and paper but for the recitation of generic computer components (e.g. storage medium).
This judicial exception is not integrated into a practical application. In particular, the claim only recites “a computer-readable storage medium” to store the “computer program”. The “computer-readable storage medium” is recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computer component. Therefore, this additional element doesn’t integrate the abstract idea into a practical application because it doesn’t impose any meaningful limits on practicing the abstract idea.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. Thus, the claims are directed towards an abstract idea and aren’t patent eligible.
As per claim 20, they are rejected because the applicant has provided evidence that the applicant intends the term "computer-readable storage medium” to include non-statutory matter. The applicant describes a computer-readable storage medium as including open ended language and thus it is reasonable to interpret it to include all possible mediums, including non-statutory mediums (see pages 3, 14, and 28). The words "storage" and/or "recording" are insufficient to convey only statutory embodiments to one of ordinary skill in the art absent an explicit and deliberate limiting definition or clear differentiation between storage media and transitory media in the disclosure. As such, the claim(s) is/are drawn to a form of energy. Energy is not one of the four categories of invention and therefore this/these claim(s) is/are not statutory. Energy is not a series of steps or acts and thus is not a process. Energy is not a physical article or object and as such is not a machine or manufacture. Energy is not a combination of substances and therefore not a composition of matter. The Examiner suggests amending the claim(s) to read as a “non-transitory computer-readable storage medium”.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 4-9, 11, 13-15, and 18-20 are rejected under 35 U.S.C. 102(a)(1 & 2) as being anticipated by Anders et al. (U.S. 2020/0334038).
As per claim 1:
Anders disclosed an apparatus comprising:
processing circuitry to perform vector operations (Anders: Figures 1-3 elements 119, 208-210, 302-304, and 308, paragraphs 44-45 and 67)(Individual MAC units perform operations on vectors of the input matrices.); and
instruction decoder circuitry to decode instructions from a set of instructions to control the processing circuitry to perform the vector operations specified by the instructions (Anders: Figure 1 elements 107-109, paragraph 41), wherein:
the set of instructions comprises a multiple-outer-product instruction specifying a plurality of first source vector operands, at least one second source vector operand and correlation information associated with the at least one second source vector operand, wherein each vector operand comprises a plurality of data elements (Anders: Figures 1 and 8 elements 107 and 800-804, paragraphs 36, 40-41, 55, 57, 59, and 101)(The VFVSMM instruction (i.e. MOP instruction) specifies source matrices A and B, both of which include vector operands that comprise data elements. For sparse matrices, each data element includes a field to indicate its row and column location within the matrix. The collection of this sparsity information reads upon the correlation information.) and, for a given second source vector operand, the correlation information is arranged to indicate, for each data element of the given second source vector operand, a corresponding first source vector operand (Anders: Figures 1 and 8 elements 107 and 800-804, paragraphs 36, 40-41, 55, 57, 59, and 101)(For sparse matrices, each data element includes a field to indicate its row and column location within the matrix. This information allows for indicating which element of the source dense matrix corresponds to the packed element of the sparse source matrix.);
the instruction decoder circuitry is responsive to the multiple-outer-product instruction to control the processing circuitry to perform a plurality of computations to implement a plurality of outer product operations (Anders: Figures 1-2 elements 107-109 and 208-210, paragraphs 41)(The decoder controls execution of the VFVSMM instruction (i.e. MOP instruction). Each MAC unit performs an outer product operation.), wherein the plurality of outer product operations comprise, for a given first source vector operand, performing an associated outer product operation to calculate an outer product of the given first source vector operand with a subset of data elements of the at least one second source vector operand (Anders: Figures 2-5 elements 202-204, 208-214, 302-304, 308, 402-404, 550A-D, and 608-610, paragraphs 45, 55, 57, 59, 67-72, 81, 85-86, and 97)(Each MAC unit performs an outer product operation of a vector from matrix A and a vector from matrix B. For VFVSMM instructions in a dense-sparse mode, a subset of data elements of a vector from a sparse matrix A/B is calculated with the vector from a dense matrix B/A.); and
the processing circuitry is configured to select, for each data element of the at least one second source vector operand, a corresponding first source vector operand to be used when performing the associated outer product operation, in dependence on the correlation information (Anders: Figures 2-6 elements 212 and 550A-D, paragraphs 55-60, 69, and 85-97)(For VFVSMM instructions in a dense-sparse mode, a subset of data elements of a vector from a sparse matrix A/B is calculated with the vector from a dense matrix B/A. The sparse encoding includes information (i.e. correlation information) for each data element that specifies “its logical row and column address” of the matrix. The information is used to match with dense matrix data elements to perform MAC operations.).
As per claim 4:
Anders disclosed the apparatus of claim 1, wherein:
the correlation information comprises at least one set of indices (Anders: Figure 2 elements 202-204, paragraphs 55-60 and 218)(Sparse matrix encoding includes additional information for each data element “to specify its logical row and column addresses” (i.e. set of indices).); and
the at least one set of indices comprises an index associating each data element of the given second source vector operand with the corresponding first source vector operand (Anders: Figure 2 elements 202-204, paragraphs 55-60 and 218)(Sparse matrix encoding includes additional information for each data element “to specify its logical row and column addresses” (i.e. set of indices). This information associates a given sparse element with its corresponding dense element in the other source matrix in a dense-sparse mode.).
As per claim 5:
Anders disclosed the apparatus of claim 4, wherein:
the correlation information is provided by at least one correlation source operand specified by the multiple-outer-product instruction (Anders: Figures 2 and 8 elements 202-204, 800, and 803-804, paragraphs 55-60, 97 and 218)(Sparse matrix encoding includes additional information for each data element “to specify its logical row and column addresses” (i.e. set of indices). The additional information is provided as part of the source matrix encoding of the VFVSMM instruction.); and
a given correlation source operand comprises, for each data element of the given second source vector operand, a corresponding element comprising the index associating that data element of the given second source vector operand with the corresponding first source vector operand (Anders: Figures 2-6 elements 212 and 550A-D, paragraphs 55-60, 69, and 85-97)(For VFVSMM instructions in a dense-sparse mode, a subset of data elements of a vector from a sparse matrix A/B is calculated with the vector from a dense matrix B/A. The sparse encoding includes information (i.e. correlation information) for each data element that specifies “its logical row and column address” of the matrix. The information is used to match with dense matrix data elements to perform MAC operations.).
As per claim 6:
Anders disclosed the apparatus of claim 4, wherein the correlation information comprises a set of indices for each second source vector operand (Anders: Figure 2 elements 202-204, paragraphs 55-60 and 218)(Sparse matrix encoding includes additional information for each data element “to specify its logical row and column addresses” (i.e. set of indices). The additional information for the same row/column of the sparse source matrix reads upon a set of indices.).
As per claim 7:
Anders disclosed the apparatus of claim 4, wherein the at least one set of indices comprises a set of indices providing the correlation information for a plurality of second source vector operands (Anders: Figure 2 elements 202-204, paragraphs 55-60 and 218)(Sparse matrix encoding includes additional information for each data element “to specify its logical row and column addresses” (i.e. set of indices). The additional information for the same row/column of the sparse source matrix reads upon a set of indices.).
As per claim 8:
Anders disclosed the apparatus of claim 7, wherein:
the correlation information is provided by at least one correlation source operand specified by the multiple-outer-product instruction (Anders: Figures 2 and 8 elements 202-204, 800, and 803-804, paragraphs 55-60, 97 and 218)(Sparse matrix encoding includes additional information for each data element “to specify its logical row and column addresses” (i.e. set of indices). The additional information is provided as part of the source matrix encoding of the VFVSMM instruction.); and
each element of a given correlation source operand comprises a plurality of indices, the plurality of indices comprising an index for a corresponding data element of each of the plurality of second source vector operands (Anders: Figure 2 elements 202-204, paragraphs 55-60 and 218)(Sparse matrix encoding includes additional information for each data element “to specify its logical row and column addresses” (i.e. set of indices). The additional information for the same row/column of the sparse source matrix reads upon a set of indices.).
As per claim 9:
Anders disclosed the apparatus of claim 1, wherein each outer product operation performed by the processing circuitry is based on a different subset of data elements of the at least one second source vector operand (Anders: Figures 2-5 elements 202-204, 208-214, 302-304, 308, 402-404, 550A-D, and 608-610, paragraphs 45, 55, 57, 59, 67-72, 81, 85-86, and 97)(Each MAC unit performs an outer product operation of a vector from matrix A and a vector from matrix B. For VFVSMM instructions in a dense-sparse mode, a subset of data elements of a vector from a sparse matrix A/B is calculated with the vector from a dense matrix B/A. MAC units in different rows/columns receive different subsets of the sparse matrix.).
As per claim 11:
Anders disclosed the apparatus of claim 1, comprising:
a set of vector registers accessible to the processing circuitry (Anders: Figure 1 element 115, paragraph 43), wherein:
each vector register is arranged to store a vector comprising a plurality of data elements; and the plurality of first source vector operands and the at least one second source vector operand comprise vectors contained within vector registers of the set of vector registers (Anders: Figures 1 and 10 elements 115 and 1010, paragraphs 43 and 166)(Vector registers store the matrix operands.).
As per claim 13:
Anders disclosed the apparatus of claim 1, wherein the data elements in each second source vector operand represent data values from a plurality of rows or a plurality of columns of a source matrix (Anders: Figures 2-3 and 8 elements 202-204, 302-304, and 803-804, paragraphs 45, 55, 67, and 97).
As per claim 14:
Anders disclosed the apparatus of claim 13, wherein each element of the given correlation source operand is arranged to enable reconstruction of the source matrix from the at least one second source vector operand (Anders: Figures 2-3 and 8 elements 202-204, 308, 800, and 803-804, paragraphs 55-60, 67, 97 and 218)(Sparse matrix encoding includes additional information for each data element “to specify its logical row and column addresses” (i.e. set of indices). This additional information allows for reconstructing the original non-compressed source matrix for execution by the grid of MAC units.).
As per claim 15:
Anders disclosed the apparatus of claim 13, wherein:
each data element in each second source vector operand is associated with a corresponding first dimension in the source matrix, wherein the corresponding first dimension comprises a corresponding row or a corresponding column in the source matrix (Anders: Figures 2-3 and 8 elements 202-204, 308, 800, and 803-804, paragraphs 55-60, 67, 97 and 218)(Sparse matrix encoding includes additional information for each data element “to specify its logical row and column addresses” (i.e. set of indices).); and
each data element in each second source vector operand provides a data value selected from among the data values in the corresponding first dimension in the source matrix (Anders: Figures 2-3 and 8 elements 202-204, 308, 800, and 803-804, paragraphs 55-60, 67, 97 and 218)(Sparse matrix encoding includes additional information for each data element “to specify its logical row and column addresses” (i.e. set of indices). The sparse matrix encoding also includes the data element value that corresponds to the “logical row and column address” location within the matrix.).
As per claim 18:
Claim 18 essentially recites the same limitations of claim 1. Therefore, claim 18 is rejected for the same reasons as claim 1.
As per claim 19:
Claim 19 essentially recites the same limitations of claim 1. Therefore, claim 19 is rejected for the same reasons as claim 1.
As per claim 20:
Claim 20 essentially recites the same limitations of claim 19. Claim 20 additionally recites the following limitations:
a computer readable storage medium storing the computer program (Anders: Figure 1 elements 101-103, paragraph 40).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 16-17 are rejected under 35 U.S.C. 103 as being unpatentable over Anders et al. (U.S. 2020/0334038), in view of Official Notice.
As per claim 16:
Anders disclosed the apparatus of claim 13, wherein the source matrix comprises a matrix with N:M structured sparsity, wherein each defined group of M data values in the source matrix comprises at most N non-zero data values (Anders: Paragraphs 55-60)(Official notice is given that sparse matrices can include structured sparsity rates for the advantage of simplified data compression and decompression for processing. Thus, it would have been obvious to one of ordinary skill in the art to implement structured sparsity rates in the sparse matrices of Anders.).
As per claim 17:
Anders disclosed the apparatus of claim 13, wherein the source matrix comprises a matrix of weights or a matrix of activations for use in execution of an artificial neural network (Anders: Figures 2-3 elements 202-204 and 302-304, paragraphs 3-4, 45, and 67)(Anders disclosed machine learning and deep neural networks, but doesn’t explicitly stated that the matrix operations are performed using input feature and weight matrices. Official notice is given that neural network matrix convolution operations can be performed on input feature and weight matrices for the advantage of detecting features or patterns from input sets. Thus, it would have been obvious to one of ordinary skill in the art to implement matrix convolutions in Anders using input feature and weight matrices.).
Claims 2 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Anders et al. (U.S. 2020/0334038), in view of Bainville et al. (U.S. 2020/0034145).
As per claim 2:
Anders disclosed the apparatus of claim 1.
Anders failed to teach array storage circuitry comprising storage elements to store data elements, the array storage circuitry being arranged to store at least one two dimensional array of data elements accessible to the processing circuitry when performing the vector operations, wherein: the multiple-outer-product instruction specifies a given two dimensional array of data elements within the array storage forming a destination operand; and the processing circuitry is configured to perform the associated outer product operation for a given first source vector operand by multiplying each data element of the given first source vector operand by each data element in the subset of data elements of the at least one second source vector operand in order to generate a plurality of outer product result elements, and using each outer product result element to update a value held in an associated storage element within the given two dimensional array of storage elements.
However, Bainville combined with Anders disclosed array storage circuitry comprising storage elements to store data elements, the array storage circuitry being arranged to store at least one two dimensional array of data elements accessible to the processing circuitry when performing the vector operations (Bainville: Figure 5 elements 24-26, paragraphs 22, 26, 37, and 49)(Anders: Figures 1 and 10 elements 115 and 1010, paragraphs 43 and 166)(Bainville disclosed storing matrices in a two-dimensional array of registers. The combination allows for storing matrices in a two-dimensional register file array.), wherein:
the multiple-outer-product instruction specifies a given two dimensional array of data elements within the array storage forming a destination operand (Bainville: Figure 5 element 28, paragraph 52)(Anders: Figure 8 element 802, paragraph 97)(The combination allows for storing matrices in a two-dimensional register file array. The VFVSMM instruction (i.e. MOP instruction) specifies a destination matrix in the two-dimensional register file array.); and
the processing circuitry is configured to perform the associated outer product operation for a given first source vector operand by multiplying each data element of the given first source vector operand by each data element in the subset of data elements of the at least one second source vector operand in order to generate a plurality of outer product result elements (Anders: Figures 2-5 elements 202-204, 208-214, 302-304, 308, 402-404, 550A-D, and 608-610, paragraphs 45, 55, 57, 59, 67-72, 81, 85-86, and 97)(Each MAC unit performs an outer product operation of a vector from matrix A and a vector from matrix B. For VFVSMM instructions in a dense-sparse mode, a subset of data elements of a vector from a sparse matrix A/B is calculated with the vector from a dense matrix B/A.), and using each outer product result element to update a value held in an associated storage element within the given two dimensional array of storage elements (Bainville: Figure 5 element 28, paragraph 52)(Anders: Figure 8 element 802, paragraphs 44 and 97)(The combination allows for storing matrices in a two-dimensional register file array. The VFVSMM instruction (i.e. MOP instruction) specifies a destination matrix in the two-dimensional register file array. The result is written back to the two-dimensional register file array as indicated by the destination of the instruction encoding.).
The advantage of the two-dimensional register file array structure of Bainville is that a single register address of an instruction can be used to locate an entire matrix to be used for processing that is stored across multiple registers. Thus, it would have been obvious to one of ordinary skill in the art at the time of the effective filing date to implement the two-dimensional register file array of Bainville within Anders for the above advantage.
As per claim 10:
Anders and Bainville disclosed the apparatus of any preceding claim when dependent on claim 2, wherein:
the multiple-outer-product instruction comprises a sum-of-outer-products instruction (Anders: Figures 1-3 and 8 elements 107, 210, 214, 308, and 800, paragraphs 36, 41, 45, 67, and 97)(The VFVSMM instruction performs a matrix convolution (i.e. sum-of-outer-products), where each MAC unit produces a vector of products from source vectors and accumulates them.); and
multiple outer product results have the same associated storage element within the given two dimensional array of storage elements, and the processing circuitry is configured to combine those multiple outer product results in order to update the value held in the associated storage element (Bainville: Figure 5 element 28, paragraph 52)(Anders: Figure 8 element 802, paragraphs 44 and 97)(The combination allows for storing matrices in a two-dimensional register file array. The VFVSMM instruction (i.e. MOP instruction) specifies a destination matrix in the two-dimensional register file array. The result is written back to the two-dimensional register file array as indicated by the destination of the instruction encoding.).
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Anders et al. (U.S. 2020/0334038), in view of Bainville et al. (U.S. 2020/0034145), in view of Official Notice.
As per claim 3:
Anders and Bainville disclosed the apparatus of claim 2, wherein the processing circuitry comprises:
multiplication circuitry to generate each outer product result when performing the plurality of outer product operations (Anders: Figures 2-3 elements 210, 214, and 308, paragraphs 45 and 67); and
multiplexer circuitry associated with the multiplication circuitry, the multiplexer circuitry to select, under control of the correlation information, a selected data element of the plurality of first source vector operands and a selected data element of the at least one second source vector operand to be multiplied in order to generate an associated outer product result element (Anders: Figures 2-6 elements 212 and 550A-D, paragraphs 55-60, 69, and 85-97)(For VFVSMM instructions in a dense-sparse mode, a subset of data elements of a vector from a sparse matrix A/B is calculated with the vector from a dense matrix B/A. The sparse encoding includes information (i.e. correlation information) for each data element that specifies “its logical row and column address” of the matrix. The information is used to match with dense matrix data elements to perform MAC operations. Visually, in figure 5 multiplexers appear to be present to perform selecting operations, but aren’t explicitly described as such. Official notice is given that multiplexers can be used to perform selection operations for the advantage of outputting one of many inputs. Thus, it would have been obvious to one of ordinary skill in the art to implement multiplexers in the MAC circuitry to select an incoming data element or a registered (i.e. held) data element waiting for a match.).
Conclusion
The following is text cited from 37 CFR 1.111(c): In amending in reply to a rejection of claims in an application or patent under reexamination, the applicant or patent owner must clearly point out the patentable novelty which he or she thinks the claims present in view of the state of the art disclosed by the references cited or the objections made. The applicant or patent owner must also show how the amendments avoid such references or objections.
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure.
Valliappan et al. (U.S. 2022/0058026), taught efficient MAC ops with sparse matrices.
Baum et al. (U.S. 2020/0210517), taught multiplication of sparse matrices.
Hansen et al. (U.S. 2019/0065149), taught outer product accumulate operations.
Narayanamoorthy (U.S. 2019/0042542), taught sparse-dense matrix multiplication.
Nurvitadhi et al. (U.S. 2018/0189234), taught very-sparse and hyper-sparse matrix processing.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JACOB A. PETRANEK whose telephone number is (571)272-5988. The examiner can normally be reached on M-F 8:00-4:30.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jyoti Mehta can be reached on (571) 270-3995. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JACOB PETRANEK/Primary Examiner, Art Unit 2183
