DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claims 18-20 are objected to under 37 C.F.R. 1.71(a) which requires “full, clear, concise, and exact terms” as to enable any person skilled in the art or science to which the invention or discovery appertains, or with which it is most nearly connected, to make and use the same. The following should be corrected.
A. In claim 18 line 12, “a post processed operation” should read “a post processing operation” instead for better clarity. Claims 19-20 inherit the same deficiency as claim 18 by reason of dependence.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-6, 8-15 and 17-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Under Step 1, claims 1-6 and 8-9 recite a series of steps and, therefore, is a process. Claims 10-15 and 17 recite a system and, therefore, is a machine. Claims 18-20 recite a circuit and, therefore, is a machine.
Under Step 2A prong 1, claim 1 recites
A method comprising, by a processing circuit including a set of multipliers:
receiving, from a first register, input feature values;
receiving, from a second register, weight values;
receiving an indication of output registers;
performing, in parallel by the set of multipliers, a matrix multiplication of the input feature values and the weight values to obtain matrix multiplication results;
receiving, from a third register, an indication of at least one of a clamp range or a shift value;
performing, based on the indication, a post processing operation on the matrix multiplication results to generate a post processed result; and
providing the post processed result to the output registers based on the received indication of the output registers.
The above underlined limitations of performing matrix multiplication to obtain results and performing a post processing operation on the results to generate a post processed result amounts to processing mathematical calculations and falls within the “Mathematical Concepts” grouping of abstract ideas. See at least paragraph [0042] for the equation for the post processing operation. Accordingly, the claim is directed to recite an abstract idea.
Under step 2A prong 2, the claim recites the following additional elements: by a processing circuit including a set of multipliers: receiving, from a first register, input feature values; receiving, from a second register, weight values; receiving an indication of output registers; in parallel by the set of multipliers, receiving, from a third register, an indication of at least one of a clamp range or a shift value, and providing the post processed result to the output registers based on the received indication of the output registers. However, the additional elements of “processing circuit”, “set of multipliers”, “a first register”, “a second register”, a third register” and “output registers” are recited at a high-level of generality (i.e., as a generic processing circuit; as generic multipliers for multiplying; and as generic registers for storing data) such that they amount to no more than mere instructions using a generic computer component or merely as tools to implement the abstract idea. Use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. See MPEP 2106.05(f)(2) for more information. The additional elements of “receiving input feature values”, “receiving weight values”, “receiving an indication of output registers”, “receiving an indication of at least one of a clamp range or a shift value”, “providing the post processed result to the output registers based on the received indication of the output registers” and “in parallel” are merely adding insignificant extra-solution activities. The additional elements do not, individually or in combination, integrate the exception into a practical application. Accordingly, the claim is not integrated into a practical application.
Under step 2B, claim 1 does not include additional elements that, individually or in combination, are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of “processing circuit”, “set of multipliers”, “a first register”, “a second register”, a third register” and “output registers” are recited at a high-level of generality (i.e., as a generic processing circuit; as generic multipliers for multiplying; and as generic registers for storing data) such that they amount to no more than mere instructions using a generic computer component or merely as tools to implement the abstract idea. Use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. See MPEP 2106.05(f)(2) for more information. The additional elements of “receiving input feature values”, “receiving weight values”, “receiving an indication of output registers”, “receiving an indication of at least one of a clamp range or a shift value”, “providing the post processed result to the output registers based on the received indication of the output registers” and “in parallel” are merely adding insignificant extra-solution activities. See MPEP 2106.05(d)(II) which states that the courts have recognized computer functions such as “Receiving or transmitting data over a network” and “Storing and retrieving information in memory” as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity. See also Hennessy et al. “Computer Organization and Design: The Hardware/Software Interface”, Fifth Edition (2014) chapter 2.2 page 63-64 which discloses arithmetic instructions being performed by a computer including specifying the source and destination registers of the input operands and the result, and chapter 6.3 page 509-510 and Fig. 6.3 which discloses performing calculation in parallel. The claim does not recite additional elements that alone or in combination amount to an inventive concept. Accordingly, the claim does not amount to significantly more than the abstract idea.
Under step 2A prong 1, claims 2-6 and 8-9 recite the same abstract idea as claim 1 by reason of dependence. Further, claim 2 recites further details of the abstract idea of “perform the post processing operation on the matrix multiplication results; wherein the post processing operation includes clamping the matrix multiplication results to limit the matrix multiplication results to the clamp range” Claim 3 recites further details of the abstract idea of “wherein the post processing operation further includes a bit shift operation”.
Claim 4 recites further details of the abstract idea “wherein: the indication indicates both the clamp range and the shift value; and the bit shift operation is performed based on the shift value”. Claim 6 recites further details of the abstract idea of the post processing operation of “wherein the post processing operation includes multiplying the matrix multiplication results with the scaling values”. Claim 8 recites further abstract idea of “wherein a bias is applied to the matrix multiplication results before the post processing operation”. Claim 9 recites further details of the input to the matrix multiplication “wherein values of the weights include one of binary values or ternary values” which falls within the “Mathematical Concepts” and/or “Mental Processes” grouping of abstract ideas. In particular claims 3-4, 6 and 8-9 do not include additional elements that would require further analysis under step 2A prong 2 and step 2B. Accordingly, the claims are directed to recite an abstract idea.
Under step 2A prong 2, claim 2 recites the following additional elements: receiving an indication. Claim 5 recites the following additional elements: receiving scaling values. However, the additional elements of “receiving an indication” to perform the post processing operation” in claim 2; and “receiving scaling values” in claim 5 are merely adding insignificant extra-solution activities. The additional elements do not, individually or in combination, integrate the exception into a practical application. Accordingly, the claims are not integrated into a practical application.
Under step 2B, claims 2 and 5 do not include additional elements that, individually or in combination, are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of “receiving an indication” to perform the post processing operation” in claim 2; and “receiving scaling values” in claim 5 are merely adding insignificant extra-solution activities. See MPEP 2106.05(d)(II) which states that the courts have recognized computer functions such as “Receiving or transmitting data over a network” and “Storing and retrieving information in memory” as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity. See also Hennessy et al. “Computer Organization and Design: The Hardware/Software Interface”, Fifth Edition (2014) chapter 2.2 page 63-64 which discloses instructions being performed by a computer including specifying which operation is to be performed including values to be used in the operation. The claims do not recite additional elements that alone or in combination amount to an inventive concept. Accordingly, the claims do not amount to significantly more than the abstract idea.
Under Step 2A prong 1, claim 10 recites
A system, comprising:
a first register configured to receive input feature values;
a second register configured to receive weights;
a third register configured to receive at least one of a clamp range or a shift value;
output registers;
a processor including:
a set of multipliers, and
a series of adders,
wherein the processor is configurable to:
receive the input feature values from the first register;
receive the weights from the second register;
receive an indication of the output registers;
process, in parallel by the set of multipliers, the input feature values and the weights to obtain intermediate results;
process, by the series of adders, the intermediate results to obtain a matrix multiplication output value;
receive at least one of the clamp range or the shift value from the third register;
perform, based on at least one of the clamp range or the shift value, a post processing operation on the matrix multiplication output value to generate a post processed result; and
output the post processed result value to the output registers based on the received indication.
The above underlined limitations of performing matrix multiplication to obtain results and performing a post processing operation on the results to generate a post processed result amounts to processing mathematical calculations and falls within the “Mathematical Concepts” grouping of abstract ideas. See at least paragraph [0042] for the equation for the post processing operation. Accordingly, the claim is directed to recite an abstract idea.
Under step 2A prong 2, the claim recites the following additional elements:
a first register configured to receive input feature values; a second register configured to receive weights; a third register configured to receive at least one of a clamp range or a shift value; output registers; a processor including: a set of multipliers, and a series of adders, wherein the processor is configurable to: receive the input feature values from the first register; receive the weights from the second register; receive an indication of the output registers; in parallel by the set of multipliers; receive at least one of the clamp range or the shift value from the third register; and output the post processed result value to the output registers based on the received indication. However, the additional elements of “a first register”, “a second register”, “a third register “, “output registers”, “a processor”, “a set of multipliers”, and “a series of adders” are recited at a high-level of generality (i.e., as generic computer components for storing data; and as a generic computer component for processing input data including a set of generic multipliers for multiplying; and a set of generic adders for adding) such that they amount to no more than mere instructions using a generic computer component or merely as tools to implement the abstract idea or merely reciting the words “apply it” (or an equivalent) with the abstract idea. Use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. See MPEP 2106.05(f)(2) for more information. The additional elements of “receive the input feature values”, “receive the weights”, “receive at least one of a clamp range or a shift value”, “receive an indication of the output registers”, “output the matrix multiplication output value to the output registers based on the received indication”, and “in parallel” are merely adding insignificant extra-solution activities. The additional elements do not, individually or in combination, integrate the exception into a practical application. Accordingly, the claim is not integrated into a practical application.
Under step 2B, claim 10 does not include additional elements that, individually or in combination, are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of “a first register”, “a second register”, “a third register “, “output registers”, “a processor”, “a set of multipliers”, and “a series of adders” are recited at a high-level of generality (i.e., as generic computer components for storing data; and as a generic computer component for processing input data including a set of generic multipliers for multiplying; and a set of generic adders for adding) such that they amount to no more than mere instructions using a generic computer component or merely as tools to implement the abstract idea or merely reciting the words “apply it” (or an equivalent) with the abstract idea. Use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. See MPEP 2106.05(f)(2) for more information. The additional elements of “receive the input feature values”, “receive the weights”, “receive at least one of a clamp range or a shift value”, “receive an indication of the output registers”, “output the matrix multiplication output value to the output registers based on the received indication”, and “in parallel” are merely adding insignificant extra-solution activities. See MPEP 2106.05(d)(II) which states that the courts have recognized computer functions such as “Receiving or transmitting data over a network” and “Storing and retrieving information in memory” as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity. See also Hennessy et al. “Computer Organization and Design: The Hardware/Software Interface”, Fifth Edition (2014) chapter 2.2 page 63-64 which discloses arithmetic instructions being performed by a computer including specifying the source and destination registers of the input operands and the result, and chapter 6.3 page 509-510 and Fig. 6.3 which discloses performing calculation in parallel. The claim does not recite additional elements that alone or in combination amount to an inventive concept. Accordingly, the claim does not amount to significantly more than the abstract idea.
Under step 2A prong 1, claims 11-15 and 17 recite the same abstract idea as claim 10 by reason of dependence. Further, claim 11 recites further details of the weight data “wherein values of the weights include binary values or ternary values”. Claim 12 recites further details of the abstract idea of “wherein the post processing operation includes: generate the post processed result by clamping the matrix multiplication output value to limit the matrix multiplication output value to a range”. Claim 13 recites further details of the abstract idea of “wherein the post processing operation further includes a bit shift operation”. Claim 14 recites further details of the abstract idea of the clamping and bit shift operation “wherein the clamping is performed based on the clamp range, and the bit shift operation is performed based on the shift value”. Claim 15 recites further abstract idea of “multiply the matrix multiplication output value with scaling values of the set of scaling values”. Claim 17 recites further abstract idea of “wherein a bias is applied to the matrix multiplication output value before the post processing operation” which falls within the “Mathematical Concepts” and/or “Mental Processes” grouping of abstract ideas. In particular claims 11-13 and 17 do not include additional elements that would require further analysis under step 2A prong 2 and step 2B. Accordingly, the claims are directed to recite an abstract idea.
Under step 2A prong 2, claim 14 recites the following additional elements: receive both the clamp range and the shift value. Claim 15 recites the following additional elements: receive a set of scaling values. However, the additional elements of “receive both the clamp range and the shift value” in claim 14; and “receive a set of scaling values” in claim 15 are merely adding insignificant extra-solution activities. The additional elements do not, individually or in combination, integrate the exception into a practical application. Accordingly, the claims are not integrated into a practical application.
Under step 2B, claims 14-15 do not include additional elements that, individually or in combination, are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of “receive both the clamp range and the shift value” in claim 14; and “receive a set of scaling values” in claim 15 are merely adding insignificant extra-solution activities. See MPEP 2106.05(d)(II) which states that the courts have recognized computer functions such as “Receiving or transmitting data over a network” and “Storing and retrieving information in memory” as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity. See also Hennessy et al. “Computer Organization and Design: The Hardware/Software Interface”, Fifth Edition (2014) chapter 2.2 page 63-64 which discloses instructions being performed by a computer including specifying which operation is to be performed including values to be used in the operation. The claims do not recite additional elements that alone or in combination amount to an inventive concept. Accordingly, the claims do not amount to significantly more than the abstract idea.
Under Step 2A prong 1, claim 18 recites
An electronic circuit comprising:
a first register configured to store input feature values;
a second register configured to store weight values;
a third register configured to store at least one of a clamp range or a shift value;
output registers configured to provide a matrix multiplication output value; and
a processor coupled to the first register, the second register, and the output registers, the processor comprising:
a set of multipliers configured to process the input feature values and the weight values to obtain intermediate results;
a series of adders configured to process the intermediate results to obtain the matrix multiplication output value; and
a post processing circuit configured to perform a post processed operation on the matrix multiplication output value based on at least one of the clamp range or the shift value.
The above underlined limitations of performing matrix multiplication to obtain matrix multiplication output value and performing a post processing operation on the matrix multiplication output value to generate a post processed result amounts to processing mathematical calculations and falls within the “Mathematical Concepts” grouping of abstract ideas. See at least paragraph [0042] for the equation for the post processing operation. Accordingly, the claim is directed to recite an abstract idea.
Under step 2A prong 2, the claim recites the following additional elements: a first register configured to store input feature values; a second register configured to store weight values; a third register configured to store at least one of a clamp range or a shift value; output registers configured to provide a matrix multiplication output value; a processor coupled to the first register, the second register, and the output registers, the processor comprising: a set of multipliers; a series of adders; and a post processing circuit. However, the additional elements of “a first register”, “a second register”, “a third register “, “output registers”, “a processor”, “a set of multipliers”, “a series of adders”, and “post processing circuit” are recited at a high-level of generality (i.e., as generic computer components for storing data; and as a generic computer component for processing input data including a set of generic multipliers for multiplying; a set of generic adders for adding; and a post processing circuit for performing a post-processing operation) such that they amount to no more than mere instructions using a generic computer component or merely as tools to implement the abstract idea or merely reciting the words “apply it” (or an equivalent) with the abstract idea. Use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. See MPEP 2106.05(f)(2) for more information. The additional elements do not, individually or in combination, integrate the exception into a practical application. Accordingly, the claim is not integrated into a practical application.
Under step 2B, claim 18 does not include additional elements that, individually or in combination, are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of “a first register”, “a second register”, “a third register “, “output registers”, “a processor”, “a set of multipliers”, “a series of adders”, and “post processing circuit” are recited at a high-level of generality (i.e., as generic computer components for storing data; and as a generic computer component for processing input data including a set of generic multipliers for multiplying; a set of generic adders for adding; and a post processing circuit for performing a post-processing operation) such that they amount to no more than mere instructions using a generic computer component or merely as tools to implement the abstract idea or merely reciting the words “apply it” (or an equivalent) with the abstract idea. Use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. See MPEP 2106.05(f)(2) for more information. The claim does not recite additional elements that alone or in combination amount to an inventive concept. Accordingly, the claim does not amount to significantly more than the abstract idea.
Under step 2A prong 1, claims 19-20 recite the same abstract idea as claim 18 by reason of dependence. Further, claim 19 recites further details of the weight data “wherein values of the weights include one of binary values or ternary values”. Claim 19 recites further abstract idea of “perform a clamping operation on the matrix multiplication output value to limit matrix multiplication results to a range to generate a post processed result” which falls within the “Mathematical Concepts” and/or “Mental Processes” grouping of abstract ideas. In particular claim 19 does not include additional elements that would require further analysis under step 2A prong 2 and step 2B. Accordingly, the claims are directed to recite an abstract idea.
Under step 2A prong 2, claim 20 recites the following additional elements: a clamping circuit and provide the post processed result. However, the additional element of “a clamping circuit” is recited at a high-level of generality (i.e., as a generic clamping circuit for clamping) such that it amounts to no more than mere instructions using a generic computer component or merely as a tool to implement the abstract idea or merely reciting the words “apply it” (or an equivalent) with the abstract idea. The additional elements of “provide the post processed result” is merely adding insignificant extra-solution activity. The additional elements do not, individually or in combination, integrate the exception into a practical application. Accordingly, the claim is not integrated into a practical application.
Under step 2B, claim 20 does not include additional elements that, individually or in combination, are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of “a clamping circuit” is recited at a high-level of generality (i.e., as a generic clamping circuit for clamping) such that it amounts to no more than mere instructions using a generic computer component or merely as a tool to implement the abstract idea or merely reciting the words “apply it” (or an equivalent) with the abstract idea. The additional elements of “provide the post processed result” is merely adding insignificant extra-solution activity. See MPEP 2106.05(d)(II) which states that the courts have recognized computer functions such as “Receiving or transmitting data over a network” and “Storing and retrieving information in memory” as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity. The claim does not recite additional elements that alone or in combination amount to an inventive concept. Accordingly, the claim does not amount to significantly more than the abstract idea.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-6 and 8 are rejected under 35 U.S.C. 103 as being unpatentable over Mansell et al. (US 20190369989 A1), hereinafter Mansell, in view of Emberling et al. (US 20230097279 A1), hereinafter Emberling and Sodani et al. (US 20190244141 A1), hereinafter Sodani.
Regarding claim 1, Mansell teaches a method comprising, by a processing circuit including a set of multipliers (Mansell Fig. 1 and claim 1; processor - processing circuitry; 11-13 set of multipliers – multipliers 400-406):
receiving, from a first register, input (Mansell Fig. 15A-16 and paragraphs [0064-0065] first register – 510/520 or v0/v2; input values – contents of the register);
receiving, from a second register, (Mansell Fig. 15A-16 and paragraphs [0064-0065] second register – other of 510/520 or v0/v2);
receiving an indication of output registers (Mansell Fig. 15A-16 and paragraphs [0064-0065] indication – 508; output registers – 522-528 or v4-v7);
performing, in parallel by the set of multipliers, a matrix multiplication of the input (Mansell Fig. 15A-16 and paragraphs [0064-0065] matrix multiplication results – result matrix C); and
providing the (Mansell Fig. 15A-16 and paragraphs [0064-0065] “The output of the FMA units is applied to a respective register of the set of accumulation registers specified in the instruction (see item 508 in FIG. 15B) … The accumulators for the result matrix C are stored in the registers v4-v7”).
Mansell does not explicitly teach input feature values and weight values; receiving, from a third register, an indication of at least one of a clamp range or a shift value; performing, based on the indication, a post processing operation on the matrix multiplication results to generate a post processed result; and providing the post processed result to the output registers based on the received indication of the output registers.
However, on the same field of endeavor, Emberling discloses receiving input feature values and weight values and performing matrix multiplication of the input feature values and the weight values to obtain matrix multiplication results (Emberling Fig.4 and paragraphs [0026, 0030]; input feature values – pixel values I1-I34; weight values – weight values w11-w33; matrix multiplication results – convolution result). Further, Emberling discloses receiving an indication to perform a post processing operation including shifting and clamping operations along with a clamp range on the matrix multiplication results; generating a post processed result by shifting and clamping the matrix multiplication results; and providing the post processed result to an output register (Emberling Figs. 3 and 5; paragraphs [0028 and 0031]; claim 10; indication to perform a post processing operation – instruction to perform at least one of transitional operations 350-370 or 515-530; post processed result – output pixels; clamping – clamping).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention, to modify Mansell using Emberling and configure the data stored in registers 510/520 (v0/v2) as input feature values and weight values. Further, configure the apparatus to receive an indication to perform a post processing operation including shifting and clamping operations on the matrix multiplication results; perform the post processing operation to generate a post processed result; and store the post processed result to the output registers in order to implement a system for processing layers of a neural network (Emberling paragraph [0028]).
Therefore, the combination of Mansell as modified in view of Emberling teaches input feature values and weight values; receiving an indication of at least one of a clamp range or a shift value; performing, based on the indication, a post processing operation on the matrix multiplication results to generate a post processed result; and providing the post processed result to the output registers based on the received indication of the output registers.
The combination of Mansell as modified in view of Emberling does not explicitly teach receiving, from a third register, an indication of at least one of a clamp range or a shift value.
However, on the same field of endeavor, Sodani discloses storing values corresponding to post processing operations such as scale, shift, and/or offset values into appropriate registers (Sodani paragraph [0051]).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention, to modify Mansell in view of Emberling and generalize the teaching of Sodani by storing the values used to implement the post processing operations such as the bias, scaling, shift and clamp in a third register in order to implement a more flexible system that can be programmable for different machine learning models (Sodani paragraph [0051]).
Therefore, the combination of Mansell as modified in view of Emberling and Sodani teaches receiving, from a third register, an indication of at least one of a clamp range or a shift value.
Regarding claim 2, Mansell as modified in view of Emberling and Sodani teaches all the limitations of claim 1 as stated above. Further, Mansell as modified in view of Emberling and Sodani teaches further comprising: receiving an indication to perform the post processing operation on the output matrix multiplication results (Emberling Figs. 3 and 5; paragraphs [0028 and 0031]; claim 10; indication to perform a post processing operation – instruction to perform at least one of transitional operations 350-370 or 515-530),
wherein the post processing operation includes clamping the matrix multiplication results to limit the matrix multiplication results to the clamp range (Emberling Figs. 3 and 5; paragraphs [0028 and 0031]).
Regarding claim 3, Mansell as modified in view of Emberling and Sodani teaches all the limitations of claim 2 as stated above. Further, Mansell as modified in view of Emberling and Sodani teaches wherein the post processing operation further includes a bit shift operation (Emberling Fig. 5 and paragraph [0031] bit shift operation – right shift by 14 bits or shift operation 522).
Regarding claim 4, Mansell as modified in view of Emberling and Sodani teaches all the limitations of claim 3 as stated above. Further, Mansell as modified in view of Emberling and Sodani teaches wherein the indication indicates of the clamp range and the shift value; and the bit shift operation is performed based on the shift value (Emberling paragraphs [0028-0029 and 0031]; claim 10; clamp range - the -127 to 128 range of an 8-bit signed integer; shift value - 14 bits).
Regarding claim 5, Mansell as modified in view of Emberling and Sodani teaches all the limitations of claim 4 as stated above. Further, Mansell as modified in view of Emberling and Sodani teaches further comprising receiving scaling values (Emberling paragraphs [0028-0029 and 0031]; scaling values – scaling values to scale each convolution result value).
Regarding claim 6, Mansell as modified in view of Emberling and Sodani teaches all the limitations of claim 5 as stated above. Further, Mansell as modified in view of Emberling and Sodani teaches wherein the post processing operation includes multiplying the matrix multiplication results with the scaling values (Emberling paragraphs [0028 and 0031]).
Regarding claim 8, Mansell as modified in view of Emberling and Sodani teaches all the limitations of claim 1 as stated above. Further, Mansell as modified in view of Emberling and Sodani teaches wherein a bias is applied to the matrix multiplication results before the post processing operation (Emberling Fig. 5 and paragraphs [0028 and 0031] bias – bias value 505/535). The motivation to combine is the same as claim 2.
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Mansell in view of Emberling and Sodani as applied to claim 1 above, and further in view of Andri et al. (NPL – “YodaNN: An Architecture for Ultralow Power Binary-Weight CNN Acceleration”), hereinafter Andri.
Regarding claim 9, Mansell as modified in view of Emberling and Sodani teaches all the limitations of claim 1 as stated above.
Mansell does not explicitly teach wherein the weight values include binary values or ternary values.
However, on the same field of endeavor, Andri discloses using weights that has binary values (Andri abstract and page 52-53 section B “ the weights which are reduced to a binary value”).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention, to modify Mansell in view of Emberling and Sodani using Andri and configure the weight values as binary weight values in order to reduce the weight storage requirement of the system (Andri abstract and Introduction page 48 right col last full paragraph).
Therefore, the combination of Mansell as modified in view of Emberling, Sodani and Andri teaches wherein the weight values include binary values or ternary values.
Claims 10, 12-15, 17-18 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Mansell in view of Emberling, Sodani and Nair et al. (US 11138292 B1), hereinafter Nair.
Regarding claim 10, Mansell teaches a system, comprising:
a first register configured to receive input (Mansell Fig. 15A-16 and paragraphs [0064-0065] first register – 510/520 or v0/v2; input values – contents of the register);
a second register configured to receive values (Mansell Fig. 15A-16 and paragraphs [0064-0065] second register – other of 510/520 or v0/v2);
output registers (Mansell Fig. 15A-16 and paragraphs [0064-0065] output registers – 522-528 or v4-v7);
a processor including (Mansell Fig. 1 and claim 1; processor - processing circuitry):
a set of multipliers (Mansell Figs. 11-13 set of multipliers – multipliers 400-406), and
(Mansell Figs. 11-13 adders – adders 408-414),
wherein the processor is configurable to:
receive the input (Mansell Fig. 15A-16 and paragraphs [0064-0065]);
receive the values from the second register (Mansell Fig. 15A-16 and paragraphs [0064-0065]);
receive an indication of the output registers (Mansell Fig. 15A-16 and paragraphs [0064-0065] indication - 508);
process, in parallel by the set of multipliers, the input values to obtain intermediate results (Mansell Fig. 15A-16 and paragraphs [0064-0069] intermediate results – result of the multiply operations);
process, by the (Mansell Fig. 15A-16 and paragraphs [0064-0065] matrix multiplication output value – matrix result C); and
output (Mansell Fig. 15A-16 and paragraphs [0064-0065] “The output of the FMA units is applied to a respective register of the set of accumulation registers specified in the instruction (see item 508 in FIG. 15B) … The accumulators for the result matrix C are stored in the registers v4-v7”).
Mansell does not explicitly teach input feature values; weights; a third register configured to receive at least one of a clamp range or a shift value; a series of adders; receive at least one of the clamp range or the shift value from the third register; perform, based on at least one of the clamp range or the shift value, a post processing operation on the matrix multiplication output value to generate a post processed result; and output the post processed result to the output registers based on the received indication.
However, on the same field of endeavor, Emberling discloses receiving input feature values and weight values and performing matrix multiplication of the input feature values and the weight values to obtain matrix multiplication results (Emberling Fig.4 and paragraphs [0026, 0030]; input feature values – pixel values I1-I34; weight values – weight values w11-w33; matrix multiplication results – convolution result). Further, Emberling discloses receiving an indication to perform a post processing operation including shifting and clamping operations along with a clamp range on the matrix multiplication results; generating a post processed result by shifting and clamping the matrix multiplication results; and providing the post processed result to an output register (Emberling Figs. 3 and 5; paragraphs [0028 and 0031]; claim 10; indication to perform a post processing operation – instruction to perform at least one of transitional operations 350-370 or 515-530; post processed result – output pixels; clamping – clamping).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention, to modify Mansell using Emberling and configure the data stored in registers 510/520 (v0/v2) as input feature values and weight values. Further, configure the apparatus to receive an indication to perform a post processing operation including shifting and clamping operations on the matrix multiplication results; perform the post processing operation to generate a post processed result; and store the post processed result to the output registers in order to implement a system for processing layers of a neural network (Emberling paragraph [0028]).
Therefore, the combination of Mansell as modified in view of Emberling teaches input feature values; weights; receive at least one of the clamp range or the shift value; perform, based on at least one of the clamp range or the shift value, a post processing operation on the matrix multiplication output value to generate a post processed result; and output the post processed result to the output registers based on the received indication.
Mansell as modified in view of Emberling does not teach a third register configured to receive at least one of a clamp range or a shift value; a series of adders; and receive at least one of the clamp range or the shift value from the third register.
However, on the same field of endeavor, Sodani discloses storing values corresponding to post processing operations such as scale, shift, and/or offset values into appropriate registers (Sodani paragraph [0051]).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention, to modify Mansell in view of Emberling and generalize the teaching of Sodani by storing the values used to implement the post processing operations such as the bias, scaling, shift and clamp in a third register in order to implement a more flexible system that can be programmable for different machine learning models (Sodani paragraph [0051]).
Therefore, the combination of Mansell as modified in view of Emberling and Sodani teaches a third register configured to receive at least one of a clamp range or a shift value; and receive at least one of the clamp range or the shift value from the third register.
Mansell as modified in view of Emberling and Sodani does not explicitly teach a series of adders.
However, on the same field of endeavor, Nair discloses performing matrix multiplication of input feature values and weights and using a series of adders to sum outputs of a plurality of multiplier circuits (Nair Figs. 2A and 3A, col 6 lines 20-57 and col 8 lines 42-44; input feature values – input image/matrix 102; weights – weights; a series of adders – adder tree 322).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention, to modify Mansell using Nair and configure each of the adders 408-414 as an adder tree for summing outputs of respective plurality multipliers 400-406 in order to implement a system for neural network computations (Nair col 1 line 33-40 and col 5 lines 25-37). Using an tree configuration as disclosed by Nair would still correctly add the output of the respective plurality multipliers 400-406.
Therefore, the combination of Mansell as modified in view of Emberling, Sodani and Nair a series of adders.
Regarding claim 12, Mansell as modified in view of Emberling, Sodani and Nair teaches all the limitations of claim 10 as stated above. Further, Mansell as modified in view of Emberling, Sodani and Nair teaches wherein the post processing operation includes: generate the post processed result by clamping the matrix multiplication output value to limit the matrix multiplication output value to a range (Emberling Figs. 3 and 5; paragraphs [0028 and 0031]).
Regarding claim 13, Mansell as modified in view of Emberling, Sodani and Nair teaches all the limitations of claim 12 as stated above. Further, Mansell as modified in view of Emberling, Sodani and Nair teaches wherein the post processing operation further includes a bit shift operation (Emberling Fig. 5 and paragraph [0031] bit shift operation – right shift by 14 bits).
Regarding claim 14, Mansell as modified in view of Emberling, Sodani and Nair teaches all the limitations of claim 13 as stated above. Further, Mansell as modified in view of Emberling, Sodani and Nair teaches wherein the processor is configured to: receive both the clamp range and the shift value from the third register, wherein the clamping is performed based on the clamp range, and the bit shift operation is performed based on the shift value (Emberling paragraphs [0028-0029 and 0031]).
Regarding claim 15, Mansell as modified in view of Emberling, Sodani and Nair teaches all the limitations of claim 14 as stated above. Further, Mansell as modified in view of Emberling, Sodani and Nair teaches wherein the processor is configured to: receive a set of scaling values; and multiply the matrix multiplication output value with scaling values of the set of scaling values (Emberling paragraphs [0028-0029 and 0031]; set of scaling values – scaling values to scale each convolution result value).
Regarding claim 17, Mansell as modified in view of Emberling, Sodani and Nair teaches all the limitations of claim 10 as stated above. Further, Mansell as modified in view of Emberling, Sodani and Nair teaches wherein a bias is applied to the matrix multiplication output value before the post processing operation (Emberling Fig. 5 and paragraphs [0028 and 0031] bias – bias value 505/535). The motivation to combine is the same as claim 12.
Regarding claim 18, Mansell teaches an electronic circuit comprising:
a first register configured to store input (Mansell Fig. 15A-16 and paragraphs [0064-0065] first register – 510/520 or v0/v2; input values – contents of the register);
a second register configured to store (Mansell Fig. 15A-16 and paragraphs [0064-0065] second register – other of 510/520 or v0/v2);
output registers configured to provide a matrix multiplication output value (Mansell Fig. 15A-16 and paragraphs [0064-0065] output registers – 522-528 or v4-v7; “The output of the FMA units is applied to a respective register of the set of accumulation registers specified in the instruction (see item 508 in FIG. 15B) … The accumulators for the result matrix C are stored in the registers v4-v7”); and
a processor coupled to the first register, the second register, and the output registers, the processor comprising (Mansell Fig. 1 and claim 1; processor - processing circuitry):
a set of multipliers configured to process the input (Mansell Figs. 11-13 set of multipliers – multipliers 400-406; Fig. 15A-16 and paragraphs [0064-0069] intermediate results – result of the multiply operations); and
(Mansell Figs. 11-13 adders – adders 408-414; Fig. 15A-16 and paragraphs [0064-0065]).
Mansell does not explicitly teach input feature values; weight values; a third register configured to store at least one of a clamp range or a shift value; a series of adders; and a post processing circuit configured to perform a post processed operation on the matrix multiplication output value based on at least one of the clamp range or the shift value.
However, on the same field of endeavor, Emberling discloses receiving input feature values and weight values and performing matrix multiplication of the input feature values and the weight values to obtain matrix multiplication results (Emberl