Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Remarks
The Examiner acknowledges amendments made to applicant’s claims. The Examiner notes that the applicant states, in remarks, that paragraph [0173] of the specification has been deleted, but the applicant’s specification only goes up to paragraph [00114].
Claim Objections
The Examiner withdraws the objections made regarding claims 12, and 22 minor informalities due to amended claims. The Examiner notes that while the amended claim 22 corrects the minor informality, it is not remarked upon in the applicant’s remarks for Claim Objections. Furthermore, the Examiner withdraws the objections made regarding claim 24 of it being misnumbered as claim 25 along with improper dependence. The Examiner notes that the applicant’s remarks include claim 23 in the section for response to Claim Objections, however Claim 23 was not objected to in the Examiner’s non-final rejection.
35 U.S.C. §112(f)
The Examiner acknowledges the applicant agrees to the interpretation of claim limitations that use the word “means”, claims 22-25, as being interpreted under 35 U.S.C. 112(f).
35 U.S.C. §112(b)
The Examiner withdraws the 35 U.S.C. §112(b) rejections made in regards to claims 1-25 being indefinite due to use of the relative term “portion”, due to amendments to the claims.
Rejections under 35 U.S.C. §103
Independent Claim 1
Applicant asserts that Baum in view of additional prior art, PredictiveNet, fails to teach “send the predicted sign of the partial convolution value as a control signal to selectively bypass future convolution computation involving the remaining bits of the mantissa fields,” as set forth in claim 1, the Examiner respectfully disagrees. Applicant seemingly acknowledged that PredictiveNet describes a method for reducing computation by evaluating the most significant bits of convolution output to decide whether to skip evaluation of least significant bits (Applicant’s arguments page 16 paragraph 2). This would satisfy the limitation of “bypass future convolution computation involving the remaining bits of the mantissa fields.” The applicant fails to further acknowledge that PredictiveNet determines whether to continue with convolution of the remaining bits by predicting the sign and using that to determine further computation, PredictiveNet uses the predicted sign, and sends it as a signal to determine whether to complete the convolution of the least significant bits or not [PredictiveNet: page 2, section III, and Figure 1 regarding sign(Ymsb) as a signal]. Furthermore, Baum teaches a zero_skip signal to notify processing elements whether or not to continue MAC operations on the data [Baum: [00236], Figure 9 item 217]. Thus, the combination of Baum in view of PredictiveNet teaches “send the predicted sign of the partial convolution value as a control signal to selectively bypass future convolution computation involving the remaining bits of the mantissa fields,”.
Independent Claims 12, and 22
The Examiner acknowledges that the applicant asserts that Baum nor PredictiveNet teaches the limitation from the same argument as for independent claim 1 listed above, for claims 12 and 22. The Examiner respectfully disagrees for at least the same reasons with the same arguments regarding claim 1.
Claim Objections
Claim 22 is objected to because of the following informalities:
With regards to claim 22, claim 22 appears to have a grammatical error and the following limitation should be changed to: “means for populating an input buffer circuitry with a portion from an input mantissa field of a floating point input data,”
Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-10, 12-20, 22-25 are rejected under 35 U.S.C. 103 as being unpatentable over Baum (U.S. Patent Application Publication No. 2022/0100601 A1), hereinafter “Baum”, in view of PredictiveNet (Y. Lin, C. Sakr, Y. Kim and N. Shanbhag, "PredictiveNet: An energy-efficient convolutional neural network via zero prediction," 2017 IEEE International Symposium on Circuits and Systems (ISCAS), Baltimore, MD, USA, 2017, pp. 1-4, doi: 10.1109/ISCAS.2017.8050797), hereinafter “PredictiveNet”.
With regards to claim 1, Baum teaches:
An apparatus, comprising: (Fig. 1);
memory; (Fig. 1 item 24 (main memory); Fig. 5 items 64 (L4 Memory), 68 (Memory I/F ), 98 (L5 Memory), 78 (L1 Memory), 74 (L2 Memory), 72 (L3 Memory), 92 (Instruction Memory), and 94 (Data Memory));
machine-readable instructions: and programmable circuitry to at least one of instantiate or execute the machine-readable instructions to at least: (¶0135 regarding program instructions executed by a processor to implement functions);
populate an input buffer circuitry with an input from an input mantissa field of a floating point input data (Fig. 11 Item 284 (input buffers); ¶0248 regarding input data and weights are stored in input buffers (Fig. 11 Item 284); Fig. 5 item 80; Fig. 6 items 162 (To Activation Function), 162 (To L2/L3 memory), 175 (Destination Control); ¶0192 regarding a dual buffer between interconnected layers where one layer output writes to a buffer and the next layer reads from the buffer as an input ; ¶0248 regarding input data and weight data written to or read from L3 memory; ¶0166 regarding Processing Element (PE) being on layer 1. As interpreted by the examiner, data between layers goes through a buffer. L3 memory is on layer 3, thus in the instance of retrieving input and weight data from L3 memory to the Processing Element (PE) it would be passed from the memory circuit, to an input buffer; ¶0223 regarding an input data representation circuit which converts from floating point to integer depending on an INT/FP signal. As interpreted by the Examiner, this indicates that the data from the input may be floating point);
floating point input data; (¶0223 regarding an input data representation circuit which converts from floating point to integer depending on an INT/FP signal. As interpreted by the Examiner, this indicates that the data from the input may be floating point; ¶0224 regarding the circuit making computations on floating point input and weight data);
populate a kernel weight buffer with a weight from a weight mantissa field of a floating point weight data, (Fig. 11 Item 284 (input buffers); ¶0248 regarding input data and weights are stored in input buffers (Fig. 11 Item 284); Fig. 5 item 80; Fig. 6 items 162 (To Activation Function), 162 (To L2/L3 memory), 175 (Destination Control); ¶0192 regarding a dual buffer between interconnected layers where one layer output writes to a buffer and the next layer reads from the buffer as an input; ¶0248 regarding input data and weight data written to or read from L3 memory; ¶0166 regarding Processing Element (PE) being on layer 1. As interpreted by the examiner, data between layers goes through a buffer. L3 memory is on layer 3, thus in the instance of retrieving input and weight data from L3 memory to the Processing Element (PE) it would be passed from the memory circuit, to an input buffer; ¶0223 regarding an input data representation circuit which converts from floating point to integer depending on an INT/FP signal. As interpreted by the Examiner, this indicates that the data from the input may be floating point; ¶0224 regarding the circuit making computations on floating point input and weight data);
floating point weight data; (¶0223 regarding an input data representation circuit which converts from floating point to integer depending on an INT/FP signal. As interpreted by the Examiner, this indicates that the data from the weight may be floating point; ¶0224 regarding the circuit making computations on floating point input and weight data);
calculate a convolution value of the input and the weight; (Fig. 6 items 158 (input memory), 161 (input), 163 (weight), and 160(weight memory); ¶0210 regarding input data and weight data input into multiplier(s) in a Processing Element (PE); ¶0297 regarding convolution performed by the device);
and send a signal as a control signal to selectively bypass future convolution computation (Fig. 9 item 217 (Zero_skip); [0236] regarding a signal used to notify the Processing elements whether to skip an operation or not);
Baum does not explicitly teach:
an input portion
the input portion corresponding to less than all of the input mantissa field of the;
a weight portion
the weight portion corresponding to less than all of the weight mantissa field of the;
calculate a partial convolution value of the input portion and the weight portion to determine a predicted sign of the partial convolution value;
send the predicted sign of the partial convolution value as a control signal to selectively bypass future convolution computation involving the remaining bits of the mantissa fields
However, PredictiveNet teaches:
an input portion (Section I. 3rd para. Regarding evaluating the most significant bit part of the convolution rather than the entire set of bits; Section III. A. regarding computing using only the most significant bits)
the input portion corresponding to less than all of the input mantissa field of the; (Section I. 3rd para. Regarding evaluating the most significant bit part of the convolution rather than the entire set of bits; Section III. A. regarding computing using only the most significant bits)
a weight portion (Section I. Regarding evaluating the most significant bit part of the convolution rather than the entire set of bits; Section III. A. regarding computing using only the most significant bits; Fig. 1 regarding use of only the most significant bits of the weight data input (Wmsb) and most significant bits of the input data (Xmsb))
the weight portion corresponding to less than all of the weight mantissa field of the; (Section I. Regarding evaluating the most significant bit part of the convolution rather than the entire set of bits; Section III. A. regarding computing using only the most significant bits; Fig. 1 regarding use of only the most significant bits of the weight data input (Wmsb) and most significant bits of the input data (Xmsb))
calculate a partial convolution value of the input portion and the weight portion to determine a predicted sign of the partial convolution value; (Section III. A. Regarding using only the most significant bits in a convolution computation to predict the sign of the outcome; Fig. 1 regarding input data most significant bits (Xmsb) and weight data most significant bits (Wmsb) being used for the partial convolution computation)
send the predicted sign of the partial convolution value as a control signal to selectively bypass future convolution computation involving the remaining bits of the mantissa fields (section III, and Fig. 1 regarding sign (Ymsb) used as a signal to determine further computation or not)
Therefore, it would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to which said subject matter pertains to combine Baum with the partial convolution and sign prediction of PredictiveNet to obtain an even greater reduction in computational cost (PredictiveNet: Section I).
With regards to claim 2, Baum in view of PredictiveNet teaches the apparatus of claim 1, as referenced above.
Baum further teaches:
wherein the programmable circuitry is to store the convolution value; (Fig. 6 items 146(destination mux), and 175(destination control); ¶0220 regarding data flow in the Processing Element (PE) being flexible, and that the destination control could send the result to the activation function or to L2/L3 memory or back to L1 memory);
calculate a full convolution value of the floating point input data and the floating point weight data; (Fig. 6 items 146(destination mux), 175(destination control), 148 (Source mux), 154 (write mux), 152(L1 Memory), 156(Read mux), and 144(Adder); ¶0220 regarding data flow in the Processing Element (PE) sending the result back to L1 memory where it would then be used as an input for the adder; ¶0297 regarding convolution performed by the device; ¶0231 regarding decode block circuitry; Fig. 8 item 192; ¶0224 regarding the circuit making computations on floating point input and weight data);
and calculate the full convolution value from the input data and weight data (Fig. 6 items 158 (input memory), 161 (input), 163 (weight), and 160(weight memory); ¶0210 regarding input data and weight data input into multiplier(s) in a Processing Element (PE); ¶0297 regarding convolution performed by the device).
Baum does not explicitly teach:
partial convolution
in response to the predicted sign of the partial convolution value being non-negative;
and calculate the full convolution value from the partial convolution value and a remaining subset of bits of the input data and weight data not used to determine the predicted sign of the partial convolution value, the partial convolution value.
However, PredictiveNet teaches:
partial convolution (Section I. Regarding evaluating the most significant bit part of the convolution rather than the entire set of bits, and depending on the predicted sign, continuing the least significant bit’s computation)
in response to the predicted sign of the partial convolution value being non-negative; (Section I. Regarding evaluating the most significant bit part of the convolution rather than the entire set of bits, and depending on the predicted sign, continuing the least significant bit’s computation)
and calculate the full convolution value from the partial convolution value and a remaining subset of bits of the input data and weight data not used to determine the predicted sign of the partial convolution value, the partial convolution value. (Section I. Regarding evaluating the most significant bit part of the convolution rather than the entire set of bits, and depending on the predicted sign, continuing the least significant bit’s computation; Section III. A. regarding an equation showing a full convolution through combining the partial convolution of the most significant bits and least significant bits)
Therefore, it would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to which said subject matter pertains to combine Baum with the partial convolution and sign prediction of PredictiveNet to obtain an even greater reduction in computational cost (PredictiveNet: Section I).
With regards to claim 3, Baum in view of PredictiveNet teaches the apparatus of claim 2, as referenced above.
Baum further teaches:
wherein the convolution value is a first convolution value, (Fig. 6 items 158 (input memory), 161 (input), 163 (weight), and 160(weight memory); ¶0210 regarding input data and weight data input into multiplier(s) in a Processing Element (PE); ¶0261 regarding Processing Element (PE) receives input data and generates output which serve as input for other subclusters for computations; ¶0297 regarding convolution performed by the device; As interpreted by the examiner, in applicant’s specification ¶0018 applicant states: descriptors such as "first,""second,""third," etc., are used herein without imputing or otherwise indicating any meaning of priority, physical order, arrangement in a list, and/or ordering in any way, but are merely used as labels and/or arbitrary names to distinguish elements for ease of understanding the disclosed examples. Therefore, as interpreted by the examiner, Baum teaches a first convolution value);
the input portion is a first input, the weight is a first weight, (Fig. 6 items 158 (input memory), 161 (input), 163 (weight), and 160(weight memory); ¶0210 regarding input data and weight data input into multiplier(s) in a Processing Element (PE);¶0261 regarding Processing Element (PE) receives input data and generates output which serve as input for other subclusters for computations; ¶0231 regarding a plurality of Processing Elements (PE), each with their own local L1 memory. As interpreted by the examiner, in applicant’s specification ¶0018 applicant states: descriptors such as "first,""second,""third," etc., are used herein without imputing or otherwise indicating any meaning of priority, physical order, arrangement in a list, and/or ordering in any way, but are merely used as labels and/or arbitrary names to distinguish elements for ease of understanding the disclosed examples. Therefore, as interpreted by the examiner, Baum teaches a first input and a first weight);
and the programmable circuitry is to calculate at least a second convolution value of a second input from the input mantissa field of the floating point input data and a second weight of the weight mantissa field of the floating point weight data. (Fig. 6 items 158 (input memory), 161 (input), 163 (weight), and 160(weight memory); ¶0210 regarding input data and weight data input into multiplier(s) in a Processing Element (PE);¶0261 regarding Processing Element (PE) receives input data and generates output which serve as input for other subclusters for computations; ¶0231 regarding a plurality of Processing Elements (PE), each with their own local L1 memory. As interpreted by the examiner, in applicant’s specification ¶0018 applicant states: descriptors such as "first,""second,""third," etc., are used herein without imputing or otherwise indicating any meaning of priority, physical order, arrangement in a list, and/or ordering in any way, but are merely used as labels and/or arbitrary names to distinguish elements for ease of understanding the disclosed examples. Therefore, as interpreted by the examiner, Baum teaches a plurality of Processing Elements (PE), each has an input data and weight data, here a “second” convolution is computed using a “second input data and the weight data” in one of the plurality of other Processing Elements (PE); ¶0224 regarding the circuit making computations on floating point input and weight data);
Baum does not explicitly teach:
partial convolution
input portion
weight portion
However, PredictiveNet teaches:
partial convolution as referenced above
input portion as referenced above
weight portion as referenced above
Therefore, it would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to which said subject matter pertains to combine Baum with the partial convolution and sign prediction of PredictiveNet to obtain an even greater reduction in computational cost (PredictiveNet: Section I).
With regards to claim 4, Baum in view of PredictiveNet teaches the apparatus of claim 2, as referenced above.
Baum further teaches:
wherein the input data is a first input data, (Fig. 6 items 158 (input memory), 161 (input), 163 (weight), and 160(weight memory); ¶0210 regarding input data and weight data input into multiplier(s) in a Processing Element (PE); As interpreted by the examiner, in applicant’s specification ¶0018 applicant states: descriptors such as "first,""second,""third," etc., are used herein without imputing or otherwise indicating any meaning of priority, physical order, arrangement in a list, and/or ordering in any way, but are merely used as labels and/or arbitrary names to distinguish elements for ease of understanding the disclosed examples. Therefore, as interpreted by the examiner, Baum teaches a first input data);
and wherein the memory is to include a plurality of banks to store a plurality of input data elements including an input data tile, (Fig. 11 Item 284 (input buffers); ¶0248 regarding input data and weights are stored in input buffers (Fig. 11 Item 284); ¶0258 regarding a memory window scheme where the computing element is given access to a subset of available memory as a data tile);
the input data tile including the first floating point input data (¶0258 regarding a memory window scheme; ¶0263 regarding each subcluster only seeing a relatively small window of memory, just enough for the Processing Elements (PE) to perform their function; Fig. 6 items 158 (input memory), 161 (input), 163 (weight), and 160(weight memory); ¶0210 regarding input data and weight data input into multiplier(s) in a Processing Element (PE); ¶0224 regarding the circuit making computations on floating point input and weight data).
With regards to claim 5, Baum in view of PredictiveNet teaches the apparatus of claim 4, as referenced above.
Baum further teaches:
wherein the convolution value is a first convolution value, (Fig. 6 items 158 (input memory), 161 (input), 163 (weight), and 160(weight memory); ¶0210 regarding input data and weight data input into multiplier(s) in a Processing Element (PE); ¶0261 regarding Processing Element (PE) receives input data and generates output which serve as input for other subclusters for computations; ¶0297 regarding convolution performed by the device; As interpreted by the examiner, in applicant’s specification ¶0018 applicant states: descriptors such as "first,""second," and "third," etc., are used herein without imputing or otherwise indicating any meaning of priority, physical order, arrangement in a list, and/or ordering in any way, but are merely used as labels and/or arbitrary names to distinguish elements for ease of understanding the disclosed examples. Therefore, as interpreted by the examiner, Baum teaches a first convolution value);
and wherein the programmable circuitry is to calculate at least one of a plurality of convolution values, (Fig. 6 items 158 (input memory), 161 (input), 163 (weight), and 160(weight memory); ¶0210 regarding input data and weight data input into multiplier(s) in a Processing Element (PE); ¶0261 regarding Processing Element (PE) receives input data and generates output which serve as input for other subclusters for computations; (¶0231 regarding a plurality of Processing Elements (PE), each with their own local L1 memory; ¶0297 regarding convolution performed by the device);
the plurality of convolution values calculated from at least a of each of the plurality of input data elements in the input data tile. (¶0258 regarding a memory window scheme; ¶0263 regarding each subcluster only seeing a relatively small window of memory, just enough for the Processing Elements (PE) to perform their function; Fig. 6 items 158 (input memory), 161 (input), 163 (weight), and 160(weight memory); ¶0210 regarding input data and weight data input into multiplier(s) in a Processing Element (PE)).
Baum does not explicitly teach:
partial convolution
portion
However, PredictiveNet teaches:
partial convolution as referenced above
portion Fig. 1 regarding input data most significant bits (Xmsb) and weight data most significant bits (Wmsb) being used for the partial convolution computation
Therefore, it would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to which said subject matter pertains to combine Baum with the partial convolution and sign prediction of PredictiveNet to obtain an even greater reduction in computational cost (PredictiveNet: Section I).
With regards to claim 6, Baum in view of PredictiveNet teaches the apparatus of claim 2, as referenced above.
Baum further teaches:
and wherein the programmable circuitry is to calculate a second convolution value of a second floating point input data and the floating point weight data (Fig. 6 items 158 (input memory), 161 (input), 163 (weight), and 160(weight memory); ¶0210 regarding input data and weight data input into multiplier(s) in a Processing Element (PE);¶0261 regarding Processing Element (PE) receives input data and generates output which serve as input for other subclusters for computations; ¶0231 regarding a plurality of Processing Elements (PE), each with their own local L1 memory. As interpreted by the examiner, in applicant’s specification ¶0018 applicant states: descriptors such as "first,""second," and "third," etc., are used herein without imputing or otherwise indicating any meaning of priority, physical order, arrangement in a list, and/or ordering in any way, but are merely used as labels and/or arbitrary names to distinguish elements for ease of understanding the disclosed examples. Therefore, as interpreted by the examiner, Baum teaches a plurality of Processing Elements (PE), each has an input data and weight data, here a “second” convolution is computed using a “second input data and the weight data” in one of the plurality of other Processing Elements (PE); ¶0224 regarding the circuit making computations on floating point input and weight data);
while during calculation of the full convolution value of the first floating point input data and the floating point weight data. (Fig. 6 items 158 (input memory), 161 (input), 163 (weight), and 160(weight memory); ¶0210 regarding input data and weight data input into multiplier(s) in a Processing Element(PE); ¶0261 regarding Processing Element (PE) receives input data and generates output which serve as input for other subclusters for computations; ¶0231 regarding a plurality of Processing Elements (PE), each with their own local L1 memory; ¶0297 regarding convolution performed by the device; ¶0224 regarding the circuit making computations on floating point input and weight data).
Baum does not explicitly teach:
partial convolution
However, PredictiveNet teaches:
partial convolution as referenced above
Therefore, it would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to which said subject matter pertains to combine Baum with the partial convolution and sign prediction of PredictiveNet to obtain an even greater reduction in computational cost (PredictiveNet: Section I).
With regards to claim 7, Baum in view of PredictiveNet teaches the apparatus of claim 1, as referenced above.
Baum further teaches:
wherein the control signal is provided to a processing circuit. (Fig. 9 item 217 (Zero_skip); [0236] regarding a signal used to notify the Processing elements whether to skip an operation or not).
Baum does not explicitly teach:
a rectified linear unit (ReLu) function
However, PredictiveNet teaches:
a rectified linear unit (ReLu) function (Section III. A. regarding use of ReLU activation function; Section III. A. regarding passing convolution outputs through the ReLu function)
Therefore, it would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to which said subject matter pertains to combine Baum with the partial convolution, sign prediction and ReLu function of PredictiveNet to obtain an even greater reduction in computational cost (PredictiveNet: Section I).
With regards to claim 8, Baum in view of PredictiveNet teaches the apparatus of claim 1 as referenced above.
Baum further teaches:
wherein the input data and the weight data are a 32-bit floating point data type. (¶0176 regarding the ability to compute in floating point representation; ¶0332 regarding an example of 32-bit size data used in the device).
With regards to claim 9, Baum in view of PredictiveNet teaches the apparatus of claim 8, as referenced above.
Baum further teaches:
wherein the programmable circuitry is to calculate the convolution value using one or more exponent bits of the input data and weight data. (Fig. 6 items 158 (input memory), 161 (input), 163 (weight), and 160 (weight memory); ¶0210 regarding input data and weight data input into multiplier(s) in a Processing Element (PE); ¶0261 regarding Processing Element (PE) receives input data and generates output which serve as input for other subclusters for computations; ¶0231 regarding a plurality of Processing Elements (PE), each with their own local L1 memory; ¶0297 regarding convolution performed by the device; ¶0224 regarding the number of significant bits for input and weight data can vary; ¶0219 regarding the number of bits being equal to m-bit mantissa bits plus e-bit exponent bits).
Baum does not explicitly teach:
partial convolution
using a sign bit and
However, PredictiveNet teaches:
partial convolution as referenced above
using a sign bit and (Section I. Regarding evaluating the most significant bit part of the convolution rather than the entire set of bits; Section III. A. regarding computing using only the most significant bits; Fig. 2 regarding the various number of bits used in the example starting at two bits. As interpreted by the examiner, PredictiveNet teaches using the most significant bits for sign prediction, in Floating Point standard format the most significant bit is the sign bit with one or more bits that follow being exponent bits, thus using at least two bits most significant bits taught in PredictiveNet, would indicate use of a sign bit and one or more exponent bits)
Therefore, it would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to which said subject matter pertains to combine Baum with the partial convolution and sign prediction of PredictiveNet to obtain an even greater reduction in computational cost (PredictiveNet: Section I).
With regards to claim 10, Baum in view of PredictiveNet teaches the apparatus of claim 8, as referenced above.
Baum further teaches:
wherein the programmable circuitry is to calculate the convolution value using, one or more exponent bits, and one or more upper mantissa bits of the input data and weight data. (Fig. 6 items 158 (input memory), 161 (input), 163 (weight), and 160(weight memory); ¶0210 regarding input data and weight data input into multiplier(s) in a Processing Element (PE); ¶0261 regarding Processing Element (PE) receives input data and generates output which serve as input for other subclusters for computations; ¶0231 regarding a plurality of Processing Elements (PE), each with their own local L1 memory; ¶0297 regarding convolution performed by the device; ¶0224 regarding the number of significant bits for input and weight data can vary; ¶0219 regarding the number of bits being equal to m-bit mantissa bits plus e-bit exponent bits).
Baum does not explicitly teach:
partial convolution
using a sign bit
However, PredictiveNet teaches:
partial convolution as referenced above
using a sign bit as referenced above
Therefore, it would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to which said subject matter pertains to combine Baum with the partial convolution and sign prediction of PredictiveNet to obtain an even greater reduction in computational cost (PredictiveNet: Section I).
With regards to claim 12, Baum teaches:
A non-transitory computer-readable storage medium comprising instructions that, when executed, cause one or more processors of a machine to at least: (Fig. 1; ¶0135 regarding program instructions executed by a processor to implement functions; Fig. 8 item 182 (Processing Elements); ¶0231 regarding a plurality of Processing Elements (PE));
populate an input buffer circuitry with an input from an input mantissa field of a floating point input data, (Fig. 11 Item 284 (input buffers); ¶0248 regarding input data and weights are stored in input buffers (Fig. 11 Item 284); Fig. 5 item 80; Fig. 6 items 162 (To Activation Function), 162 (To L2/L3 memory), 175 (Destination Control); ¶0192 regarding a dual buffer between interconnected layers where one layer output writes to a buffer and the next layer reads from the buffer as an input ; ¶0248 regarding input data and weight data written to or read from L3 memory; ¶0166 regarding Processing Element (PE) being on layer 1. As interpreted by the examiner, data between layers goes through a buffer. L3 memory is on layer 3, thus in the instance of retrieving input and weight data from L3 memory to the Processing Element (PE) it would be passed from the memory circuit, to an input buffer; ¶0224 regarding the circuit making computations on floating point input and weight data);
the input corresponding to the floating point input data; (Fig. 6 items 158 (input memory), 161 (input), 163 (weight), and 160(weight memory); ¶0210 regarding input data and weight data input into multiplier(s) in a Processing Element (PE); ¶0224 regarding the circuit making computations on floating point input and weight data);
the weight corresponding to the floating point weight data; (Fig. 6 items 158 (input memory), 161 (input), 163 (weight), and 160(weight memory); ¶0210 regarding input data and weight data input into multiplier(s) in a Processing Element (PE); ¶0224 regarding the circuit making computations on floating point input and weight data);
calculate a convolution value of the input and the weight; (Fig. 6 items 158 (input memory), 161 (input), 163 (weight), and 160(weight memory); ¶0210 regarding input data and weight data input into multiplier(s) in a Processing Element (PE); ¶0297 regarding convolution performed by the device);
and send a signal as a control signal to selectively bypass future convolution computation. (Fig. 9 item 217 (Zero_skip); [0236] regarding a signal used to notify the Processing elements whether to skip an operation or not).
Baum does not explicitly teach:
an input portion
the input portion corresponding to less than all of the input mantissa field of;
a weight portion
the weight portion corresponding to less than all of the weight mantissa field of;
calculate a partial convolution value of the input portion and the weight portion to determine a predicted sign of the partial convolution value;
send the predicted sign of the partial convolution value as a control signal to selectively bypass future convolution computation involving the remaining bits of the mantissa fields
However, PredictiveNet teaches:
an input portion (Section I. 3rd para. Regarding evaluating the most significant bit part of the convolution rather than the entire set of bits; Section III. A. regarding computing using only the most significant bits)
the input portion corresponding to less than all of the input mantissa field of; (Section I. 3rd para. Regarding evaluating the most significant bit part of the convolution rather than the entire set of bits; Section III. A. regarding computing using only the most significant bits)
a weight portion (Section I. Regarding evaluating the most significant bit part of the convolution rather than the entire set of bits; Section III. A. regarding computing using only the most significant bits; Fig. 1 regarding use of only the most significant bits of the weight data input (Wmsb) and most significant bits of the input data (Xmsb))
the weight portion corresponding to less than all of the weight mantissa field of; (Section I. Regarding evaluating the most significant bit part of the convolution rather than the entire set of bits; Section III. A. regarding computing using only the most significant bits; Fig. 1 regarding use of only the most significant bits of the weight data input (Wmsb) and most significant bits of the input data (Xmsb))
calculate a partial convolution value of the input portion and the weight portion to determine a predicted sign of the partial convolution value; (Section III. A. Regarding using only the most significant bits in a convolution computation to predict the sign of the outcome; Fig. 1 regarding input data most significant bits (Xmsb) and weight data most significant bits (Wmsb) being used for the partial convolution computation)
send the predicted sign of the partial convolution value as a control signal to selectively bypass future convolution computation involving the remaining bits of the mantissa fields as referenced above
Therefore, it would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to which said subject matter pertains to combine Baum with the partial convolution and sign prediction of PredictiveNet to obtain an even greater reduction in computational cost (PredictiveNet: Section I).
With regards to claim 13, Baum in view of PredictiveNet teaches the non-transitory computer-readable storage medium of claim 12, as referenced above.
Baum further teaches:
wherein the instructions, when executed, cause the one or more processors of the machine to at least: (¶0135 regarding program instructions executed by a processor to implement functions);
store the convolution value; (Fig. 6 items 146 (destination mux), and 175(destination control); ¶0220 regarding data flow in the Processing Element (PE) being flexible, and that the destination control could send the result to the activation function or to L2/L3 memory or back to L1 memory; ¶0192 regarding a dual buffer between interconnected layers where one layer output writes to a buffer and the next layer reads from the buffer as an input);
calculate a full convolution value of the input data and the weight data; (Fig. 6 items 146(destination mux), 175(destination control), 148 (Source mux), 154 (write mux), 152(L1 Memory), 156(Read mux), and 144(Adder); ¶0220 regarding data flow in the Processing Element (PE) sending the result back to L1 memory where it would then be used as an input for the adder; ¶0297 regarding convolution performed by the device; ¶0231 regarding decode block circuitry; Fig. 8 item 192);
and calculate the full convolution. (Fig. 6 items 146(destination mux), 175(destination control), 148 (Source mux), 154 (write mux), 152(L1 Memory), 156(Read mux), and 144(Adder); ¶0220 regarding data flow in the Processing Element (PE) sending the result back to L1 memory where it would then be used as an input for the adder; ¶0297 regarding convolution performed by the device; ¶0231 regarding decode block circuitry; Fig. 8 item 192).
Baum does not explicitly teach:
partial convolution
in response to the predicted sign of the partial convolution value being non-negative;
and calculate the full convolution value from the partial convolution value and a remaining subset of bits of the input data and weight data not used to determine the predicted sign of the partial value, the partial convolution value.
However, PredictiveNet teaches:
partial convolution as referenced above
in response to the predicted sign of the partial convolution value being non-negative; as referenced above
and calculate the full convolution value from the partial convolution value and a remaining subset of bits of the input data and weight data not used to determine the predicted sign of the partial value, the partial convolution value. (Section I. Regarding evaluating the most significant bit part of the convolution rather than the entire set of bits, and depending on the predicted sign, continuing the least significant bit’s computation; Section III. A. regarding an equation showing a full convolution through combining the partial convolution of the most significant bits and least significant bits)
Therefore, it would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to which said subject matter pertains to combine Baum with the partial convolution and sign prediction of PredictiveNet to obtain an even greater reduction in computational cost (PredictiveNet: Section I).
With regards to claim 14, Baum in view of PredictiveNet teaches the non-transitory computer-readable storage medium of claim 13, as referenced above.
Baum further teaches:
wherein the convolution value is a first convolution value, (Fig. 6 items 158 (input memory), 161 (input), 163 (weight), and 160(weight memory); ¶0210 regarding input data and weight data input into multiplier(s) in a Processing Element (PE); ¶0261 regarding Processing Element (PE) receives input data and generates output which serve as input for other subclusters for computations; ¶0297 regarding convolution performed by the device; As interpreted by the examiner, in applicant’s specification ¶0018 applicant states: descriptors such as "first,""second," and "third," etc., are used herein without imputing or otherwise indicating any meaning of priority, physical order, arrangement in a list, and/or ordering in any way, but are merely used as labels and/or arbitrary names to distinguish elements for ease of understanding the disclosed examples. Therefore, as interpreted by the examiner, Baum teaches a first convolution value);
the input is a first input, (Fig. 6 items 158 (input memory), 161 (input); ¶0210 regarding input data and weight data input into multiplier(s) in a Processing Element (PE); ¶0261 regarding Processing Element (PE) receives input data and generates output which serve as input for other subclusters for computations; As interpreted by the examiner, in applicant’s specification ¶0018 applicant states: descriptors such as "first,""second," and "third," etc., are used herein without imputing or otherwise indicating any meaning of priority, physical order, arrangement in a list, and/or ordering in any way, but are merely used as labels and/or arbitrary names to distinguish elements for ease of understanding the disclosed examples. Therefore, as interpreted by the examiner, Baum teaches a first input);
the weight is a first weight, (Fig. 6 163 (weight), and 160(weight memory); ¶0210 regarding input data and weight data input into multiplier(s) in a Processing Element (PE); ¶0261 regarding Processing Element (PE) receives input data and generates output which serve as input for other subclusters for computations; As interpreted by the examiner, in applicant’s specification ¶0018 applicant states: descriptors such as "first,""second," and "third," etc., are used herein without imputing or otherwise indicating any meaning of priority, physical order, arrangement in a list, and/or ordering in any way, but are merely used as labels and/or arbitrary names to distinguish elements for ease of understanding the disclosed examples. Therefore, as interpreted by the examiner, Baum teaches a first weight);
and the instructions, when executed, cause the one or more processors of the machine to: (¶0135 regarding program instructions executed by a processor to implement functions);
calculate at least a second convolution value of a second input from the input mantissa field of the floating point input data and a second weight of the weight mantissa field of the floating point weight data. (Fig. 6 items 158 (input memory), 161 (input), 163
(weight), and 160(weight memory); ¶0210 regarding input data and weight data input into multiplier(s) in a Processing Element (PE);¶0261 regarding Processing Element (PE) receives input data and generates output which serve as input for other subclusters for computations; ¶0231 regarding a plurality of Processing Elements (PE), each with their own local L1 memory. As interpreted by the examiner, in applicant’s specification ¶0018 applicant states: descriptors such as "first,""second," and "third," etc., are used herein without imputing or otherwise indicating any meaning of priority, physical order, arrangement in a list, and/or ordering in any way, but are merely used as labels and/or arbitrary names to distinguish elements for ease of understanding the disclosed examples. Therefore, as interpreted by the examiner, Baum teaches a plurality of Processing Elements (PE), each has an input data and weight data, here a “second” convolution is computed using a “second input data and the weight data” in one of the plurality of other Processing Elements (PE); ¶0224 regarding the circuit making computations on floating point input and weight data).
Baum does not explicitly teach:
partial convolution
input portion
weight portion
However, PredictiveNet teaches:
partial convolution as referenced above
input portion as referenced above
weight portion as referenced above
Therefore, it would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to which said subject matter pertains to combine Baum with the partial convolution and sign prediction of PredictiveNet to obtain an even greater reduction in computational cost (PredictiveNet: Section I).
With regards to claim 15, Baum in view of PredictiveNet teaches the non-transitory computer-readable storage medium of claim 12, as referenced above.
Baum further teaches:
wherein the input data is a first input data, (Fig. 6 items 158 (input memory), 161 (input), 163 (weight), and 160(weight memory); ¶0210 regarding input data and weight data input into multiplier(s) in a Processing Element (PE); As interpreted by the examiner, in applicant’s specification ¶0018 applicant states: descriptors such as "first,""second," and "third," etc., are used herein without imputing or otherwise indicating any meaning of priority, physical order, arrangement in a list, and/or ordering in any way, but are merely used as labels and/or arbitrary names to distinguish elements for ease of understanding the disclosed examples. Therefore, as interpreted by the examiner, Baum teaches a first input);
and wherein the instructions, when executed, cause the one or more processors of the machine to: (¶0135 regarding program instructions executed by a processor to implement functions);
store a plurality of input data elements including an input data tile, the input data tile including the first floating point input data. (Fig. 11 Item 284 (input buffers); ¶0248 regarding input data and weights are stored in input buffers (Fig. 11 Item 284); ¶0258 regarding a memory window scheme where the computing element is given access to a subset of available memory; ¶0224 regarding the circuit making computations on floating point input and weight data).
With regards to claim 16, Baum in view of PredictiveNet teaches the non-transitory computer-readable medium of claim 15, as referenced above.
Baum further teaches:
wherein the convolution value is a first convolution value, (Fig. 6 items 158 (input memory), 161 (input), 163 (weight), and 160(weight memory); ¶0210 regarding input data and weight data input into multiplier(s) in a Processing Element (PE); ¶0261 regarding Processing Element (PE) receives input data and generates output which serve as input for other subclusters for computations; ¶0297 regarding convolution performed by the device; As interpreted by the examiner, in applicant’s specification ¶0018 applicant states: descriptors such as "first,""second," and "third," etc., are used herein without imputing or otherwise indicating any meaning of priority, physical order, arrangement in a list, and/or ordering in any way, but are merely used as labels and/or arbitrary names to distinguish elements for ease of understanding the disclosed examples. Therefore, as interpreted by the examiner, Baum teaches a first convolution value);
and wherein the instructions, when executed, cause the one or more processors of the machine to: (¶0135 regarding program instructions executed by a processor to implement functions);
calculate at least one of a plurality of convolution values, the plurality of convolution values calculated from at least a of each of the plurality of input data elements in the input tile. (¶0135 regarding program instructions executed by a processor to implement functions; ¶0231 regarding a plurality of Processing Elements (PE), each with their own local L1 memory; ¶0297 regarding convolution performed by the device; ¶0231 regarding a plurality of Processing Elements (PE), each with their own local L1 memory; ¶0297 regarding convolution performed by the device; ¶0258 regarding a memory window scheme; ¶0263 regarding each subcluster only seeing a relatively small window of memory, just enough for the Processing Elements (PE) to perform their function).
Baum does not explicitly teach:
partial convolution
portion
However, PredictiveNet teaches:
partial convolution
portion (Section I. Regarding evaluating the most significant bit part of the convolution rather than the entire set of bits; Section III. A. regarding computing using only the most significant bits; Fig. 1 regarding input data most significant bits (Xmsb) and weight data most significant bits (Wmsb) being used for the partial convolution computation)
Therefore, it would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to which said subject matter pertains to combine Baum with the partial convolution and sign prediction of PredictiveNet to obtain an even greater reduction in computational cost (PredictiveNet: Section I).
With regards to claim 17, Baum in view of PredictiveNet teaches the non-transitory computer-readable storage medium of claim 13, as referenced above.
Baum further teaches:
wherein the input data is a first input data, (Fig. 6 items 158