DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This Action is non-final and is in response to the claims filed July 25th, 2022. Claims 1-13 are pending, of which claims 1-13 are currently rejected.
Drawings
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5) because they do not include the following reference signs mentioned in the description (at ¶ 0048 - ¶ 0051): 61, 62, 63, 64.
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
In addition to Replacement Sheets containing the corrected drawing figure(s), applicant is required to submit a marked-up copy of each Replacement Sheet including annotations indicating the changes made to the previous version. The marked-up copy must be clearly labeled as “Annotated Sheets” and must be presented in the amendment or remarks section that explains the change(s) to the drawings. See 37 CFR 1.121(d)(1). Failure to timely submit the proposed drawing and marked-up copy will result in the abandonment of the application
Specification
The abstract of the disclosure is objected to because of the use of the acronym “SONA” without a proper definition of the acronym. A corrected abstract of the disclosure is required and must be presented on a separate sheet, apart from any other text. See MPEP § 608.01(b).
The disclosure is objected to because of the following informalities:
[0018] paragraph is incomplete
Claim Objections
Claims 8 and 12-13 are objected to because of the following informalities:
Claim 8 line 4 “cannonical” should be “canonical”, and on line 3 “by sign of each non-zero digit” should be “by a sign of each non-zero digit”, and on line 6 “based on position of non-zero digits” should be “based on a position of each of the two non-zero digits”.
Claim 12 lines 10-11 “subtracting sum of the negative adder tree circuit from sum of the positive adder tree circuit thereby yielding a final product” should be subtracting a sum of the negative adder tree circuit from a sum of the positive adder tree circuit thereby yielding a final product”.
Claim 13 is objected to based on its dependence on claim 12.
Appropriate correction required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-11 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claim 1, at Step 1, the claim is directed to a statutory category of invention (method).
At Step 2A, Prong 1, Examiner notes that the claim recites an abstract idea. Claim language recites performing computation on an input patch represented by a numeral system, and retrieving a kernel in order to carry out a multiplication operation between the input patch and the kernel. Below are the limitations of claim 1 that recite an abstract idea under mathematical concepts:
for performing computations (mathematical concepts)
the input patch is a vector or a matrix extracted from an input and each element of the vector or matrix is represented with M digits in accordance with a numeral system (mathematical concepts: mathematical relationships);
where each weight of the kernel is represented with N digits in accordance with a numeral system; and
computing a multiplication between elements of the input patch and elements of the kernel of the neural network, (mathematical concepts)
where non-zero digits of at least one of the elements of the input patch or kernel is constrained to less than M or less than N, respectively (mathematical concepts: mathematical relationships).
All limitations as indicated describe “mathematical concepts”.
At Step 2A Prong 2, these are the additional elements recited in claim 1:
Neural network
Receiving, by a computer processor, an input patch
A computer processor
Retrieving, by the computer processor, a kernel.
These additional elements do not integrate the judicial exception into a practical application of the exception. See MPEP 2106.05(f). These additional elements represent no more than mere instructions to apply the judicial exception on a computer. Even when viewed in combination, these additional elements do not integrate the recited judicial exception into a practical application and the claim recites a judicial exception.
There are insignificant extra-solution activities which must be made of note:
Receiving, by a computer processor, an input patch (data gathering falling under insignificant extra-solution activity)
Retrieving, by the computer processor, a kernel (data gathering falling under insignificant extra-solution activity).
At Step 2B, there are no additional elements claimed that amount to significantly more than the recited judicial exception, all of which at best are the equivalent of merely adding the words “apply it” to the judicial exception. Mere instructions to apply an exception cannot provide an inventive concept.
In regards to the insignificant extra-solution activity found in this limitation “receiving, by a computer processor, an input patch”, this action describes mere data gathering that is recited at a high level of generality. Per MPEP 2106.05(d)(II), the courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity: i. Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network). This limitation therefore remains insignificant extra-solution activity even upon reconsideration. Thus, this limitation does not amount to significantly more.
In regards to the insignificant extra-solution activity found in this limitation “retrieving, by the computer processor, a kernel”, this action describes mere data gathering that is recited at a high level of generality. Per MPEP 2106.05(d)(II), the courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity: i. Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network). This limitation therefore remains insignificant extra-solution activity even upon reconsideration. Thus, this limitation does not amount to significantly more.
Even when considered in combination, these additional elements represent mere instructions to apply an exception, which do not provide an inventive concept. The claim is not eligible.
Regarding claim 2, at Step 1, the claim is directed to a statutory category of invention (method).
At Step 2A, Prong 1, Examiner notes that the claim recites an abstract idea. Below are the limitations of claim 2 that recite an abstract idea under mathematical concepts:
wherein each element of the vector or the matrix is a two's complement representation having M bits and each weight of the kernel is quantized as a canonical signed digit with N digits, such that the non-zero digits of the canonical signed digit are constrained to less than N (mathematical concepts: mathematical relationships).
All limitations as indicated describe “mathematical concepts”.
At Step 2A Prong 2, there are the no additional elements beyond those recited in claim 1
Even when considered in combination, these additional elements represent mere instructions to apply an exception, which do not provide an inventive concept. The claim is not eligible.
Regarding claim 3, at Step 1, the claim is directed to a statutory category of invention (method).
At Step 2A, Prong 1, Examiner notes that the claim recites an abstract idea. Below are the limitations of claim 3 that recite an abstract idea under mathematical concepts:
wherein each element of the vector or the matrix is represented by a sign and magnitude representation with M bits; (mathematical concepts: mathematical relationships)
each weight of the kernel is represented by a sign and magnitude representation with N bits, such that non-zero bits of at least one of the elements of the input patch or the element of the kernel is constrained to less than M or less than N, respectively (mathematical concepts: mathematical relationships).
All limitations as indicated describe “mathematical concepts”.
At Step 2A Prong 2, there are the no additional elements beyond those recited in claim 1
Even when considered in combination, these additional elements represent mere instructions to apply an exception, which do not provide an inventive concept. The claim is not eligible.
Regarding claim 4, at Step 1, the claim is directed to a statutory category of invention (method).
At Step 2A, Prong 1, Examiner notes that the claim recites an abstract idea. Claim language recites performing computation on an input patch represented by a numeral system, more specifically a binary number, and retrieving a kernel in canonical signed digit representation in order to carry out a multiplication operation between the input patch and the kernel. Below are the limitations of claim 4 that recite an abstract idea under mathematical concepts:
for performing computations (mathematical concepts)
the input patch is a vector or a matrix extracted from an input and each element of the vector or matrix is represented by a binary number (mathematical concepts: mathematical relationships);
where each weight of the kernel is quantized as a canonical signed digit with Nd digits and non-zero digits of the canonical signed digit are constrained to less than N digits; and
computing a multiplication between elements of the input patch and elements of the kernel of the neural network, (mathematical concepts).
All limitations as indicated describe “mathematical concepts”.
At Step 2A Prong 2, these are the additional elements recited in claim 1:
Neural network
Receiving, by a computer processor, an input patch
Computer processor
Retrieving, by the computer processor, a kernel.
These additional elements do not integrate the judicial exception into a practical application of the exception. See MPEP 2106.05(f). These additional elements represent no more than mere instructions to apply the judicial exception on a computer. Even when viewed in combination, these additional elements do not integrate the recited judicial exception into a practical application and the claim recites a judicial exception.
There are insignificant extra-solution activities which must be made of note:
Receiving, by a computer processor, an input patch (data gathering falling under insignificant extra-solution activity)
Retrieving, by the computer processor, a kernel (data gathering falling under insignificant extra-solution activity).
At Step 2B, there are no additional elements claimed that amount to significantly more than the recited judicial exception, all of which at best are the equivalent of merely adding the words “apply it” to the judicial exception. Mere instructions to apply an exception cannot provide an inventive concept.
In regards to the insignificant extra-solution activity found in this limitation “receiving, by a computer processor, an input patch”, this action describes mere data gathering that is recited at a high level of generality. Per MPEP 2106.05(d)(II), the courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity: i. Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network). This limitation therefore remains insignificant extra-solution activity even upon reconsideration. Thus, this limitation does not amount to significantly more.
In regards to the insignificant extra-solution activity found in this limitation “retrieving, by the computer processor, a kernel”, this action describes mere data gathering that is recited at a high level of generality. Per MPEP 2106.05(d)(II), the courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity: i. Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network). This limitation therefore remains insignificant extra-solution activity even upon reconsideration. Thus, this limitation does not amount to significantly more.
Even when considered in combination, these additional elements represent mere instructions to apply an exception, which do not provide an inventive concept. The claim is not eligible.
Regarding claim 5, at Step 1, the claim is directed to a statutory category of invention (method).
At Step 2A, Prong 1, Examiner notes that the claim recites an abstract idea. Below are the limitations of claim 5 that recite an abstract idea under mathematical concepts:
wherein each element of the vector or the matrix is a two’s complement representation having M bits (mathematical concepts).
All limitations as indicated describe “mathematical concepts”.
At Step 2A Prong 2, there are the no additional elements beyond those recited in claim 4.
Even when considered in combination, these additional elements represent mere instructions to apply an exception, which do not provide an inventive concept. The claim is not eligible.
Regarding claim 6, at Step 1, the claim is directed to a statutory category of invention (method).
At Step 2A, Prong 1, Examiner notes that the claim recites an abstract idea. Below are the limitations of claim 6 that recite an abstract idea under mathematical concepts:
wherein each kernel weight is further defined as a canonical signed digit with 8 bits and no more than two non-zero digits and each element of the matrix is a two’s complement representation with 8 bits (mathematical concepts).
All limitations as indicated describe “mathematical concepts”.
At Step 2A Prong 2, there are the no additional elements beyond those recited in claim 4.
Even when considered in combination, these additional elements represent mere instructions to apply an exception, which do not provide an inventive concept. The claim is not eligible.
Regarding claim 7, at Step 1, the claim is directed to a statutory category of invention (method).
At Step 2A, Prong 1, Examiner notes that the claim recites an abstract idea. Below are the limitations of claim 7 that recite an abstract idea under mathematical concepts:
wherein each multiplication is implemented by a bit shift operation for each of the two non-zero digits followed by a 16 bit addition operation (mathematical concepts).
All limitations as indicated describe “mathematical concepts”.
At Step 2A Prong 2, there are the no additional elements beyond those recited in claim 6.
Even when considered in combination, these additional elements represent mere instructions to apply an exception, which do not provide an inventive concept. The claim is not eligible.
Regarding claim 8, at Step 1, the claim is directed to a statutory category of invention (method).
At Step 2A, Prong 1, Examiner notes that the claim recites an abstract idea. Below are the limitations of claim 8 that recite an abstract idea under mathematical concepts:
multiplying a given element of the input patch by sign of each non-zero digit of the cannonical signed digit to yield two products from a first stage (mathematical concepts);
bit shifting products from the first stage in a second stage, where the shifting amount is based on position of non-zero digits in the canonical signed digit (mathematical concepts: aligning of operands and multiplying by shifting); and
adding products from the second stage together (mathematical concepts).
All limitations as indicated describe “mathematical concepts”.
At Step 2A Prong 2, there are no additional elements beyond those recited in claim 6.
Even when considered in combination, these additional elements represent mere instructions to apply an exception, which do not provide an inventive concept. The claim is not eligible.
Regarding claim 9, at Step 1, the claim is directed to a statutory category of invention (method).
At Step 2A, Prong 1, Examiner notes that the claim recites an abstract idea. Below are the limitations of claim 9 that recite an abstract idea under mathematical concepts:
further comprises accumulating partial results from multiplying the elements of the input patch by elements of the kernel (mathematical concepts)
All limitations as indicated describe “mathematical concepts”.
At Step 2A Prong 2, these are the additional elements recited in claim 9:
In a register
And feeding the accumulated results to a next layer of the neural network.
These additional elements do not integrate the judicial exception into a practical application of the exception. See MPEP 2106.05(f). These additional elements merely generally link the recited abstract idea to the field of neural networks. Even when viewed in combination, these additional elements do not integrate the recited judicial exception into a practical application and the claim recites a judicial exception.
At Step 2B, there are no additional elements claimed that amount to significantly more than the recited judicial exception, all of which at best are the equivalent of merely adding the words “apply it” to the judicial exception. Mere instructions to apply an exception cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception, which do not provide an inventive concept. The claim is not eligible.
Regarding claim 10, at Step 1, the claim is directed to a statutory category of invention (method).
At Step 2A, Prong 1, Examiner notes that the claim recites an abstract idea. Claim language recites performing computation on an input patch represented by a sign and magnitude representation, and retrieving a kernel in sign magnitude representation in order to carry out a multiplication operation between the input patch and the kernel. Below are the limitations of claim 10 that recite an abstract idea under mathematical concepts:
performing computations (mathematical concepts)
an input patch of data, where the input patch is a vector or a matrix extracted from an input and each element of the vector or the matrix is represented by a sign and magnitude representation with M bits (mathematical concepts);
a kernel of the neural network, where each weight of the kernel is represented by a sign and magnitude representation with N bits (mathematical concepts); and
computing a multiplication between elements of the input patch and weights of the kernel of the neural network (mathematical concepts), where non-zero bits of at least one of the elements of the input patch or the element of the kernel is constrained to less than M or less than N, respectively (mathematical concepts: mathematical relationships).
All limitations as indicated describe “mathematical concepts”.
At Step 2A Prong 2, these are the additional elements recited in claim 10:
Neural network
Receiving, by a computer processor, an input patch
Computer processor
Retrieving, by the computer processor, a kernel.
These additional elements do not integrate the judicial exception into a practical application of the exception. See MPEP 2106.05(f). These additional elements represent no more than mere instructions to apply the judicial exception on a computer. Even when viewed in combination, these additional elements do not integrate the recited judicial exception into a practical application and the claim recites a judicial exception.
There are insignificant extra-solution activities which must be made of note:
Receiving, by a computer processor, an input patch (data gathering falling under insignificant extra-solution activity)
Retrieving, by the computer processor, a kernel (data gathering falling under insignificant extra-solution activity).
At Step 2B, there are no additional elements claimed that amount to significantly more than the recited judicial exception, all of which at best are the equivalent of merely adding the words “apply it” to the judicial exception. Mere instructions to apply an exception cannot provide an inventive concept.
In regards to the insignificant extra-solution activity found in this limitation “receiving, by a computer processor, an input patch”, this action describes mere data gathering that is recited at a high level of generality. Per MPEP 2106.05(d)(II), the courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity: i. Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network). This limitation therefore remains insignificant extra-solution activity even upon reconsideration. Thus, this limitation does not amount to significantly more.
In regards to the insignificant extra-solution activity found in this limitation “retrieving, by the computer processor, a kernel”, this action describes mere data gathering that is recited at a high level of generality. Per MPEP 2106.05(d)(II), the courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity: i. Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network). This limitation therefore remains insignificant extra-solution activity even upon reconsideration. Thus, this limitation does not amount to significantly more.
Even when considered in combination, these additional elements represent mere instructions to apply an exception, which do not provide an inventive concept. The claim is not eligible.
Regarding claim 11, at Step 1, the claim is directed to a statutory category of invention (method).
At Step 2A, Prong 1, Examiner notes that the claim recites an abstract idea. Below are the limitations of claim 11 that recite an abstract idea under mathematical concepts:
each multiplication (mathematical concepts)
All limitations as indicated describe “mathematical concepts”.
At Step 2A Prong 2, these are the additional elements recited in claim 11:
sign and magnitude multiplier circuit
These additional elements do not integrate the judicial exception into a practical application of the exception. See MPEP 2106.05(f). These additional elements represent no more than mere instructions to apply the judicial exception on a computer. Even when viewed in combination, these additional elements do not integrate the recited judicial exception into a practical application and the claim recites a judicial exception.
At Step 2B, there are no additional elements claimed that amount to significantly more than the recited judicial exception, all of which at best are the equivalent of merely adding the words “apply it” to the judicial exception. Mere instructions to apply an exception cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply an exception, which do not provide an inventive concept. The claim is not eligible.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-3, 8, 10-13 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 recites the limitation “a numeral system” on lines 8-9. It is unclear if this mention of “a numeral system” is the same numeral system as recited on lines 5-6 or another numeral system. For examination purposes, the numeral system of lines 8-9 will be construed to be the same numeral system of line 5-6.
Claim 1 recites the limitation “elements of the kernel” on line 11. It is unclear whether this recitation of “elements of the kernel” are the weights of the kernel as recited on line 8 of the same claim or some other element of the kernel. For examination purposes, elements of the kernel as recited on line 11 of claim 1 will be construed to be the weights of the kernel as recited on line 8 of claim 1.
Because claims 2-3 depend upon claim 1, claims 2-3 are additionally rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite.
Claim 8 recites the limitation “the bit shifting amount” on lines 5-6. There is lack of antecedent basis for this limitation. Appropriate correction is required.
Claim 10 recites the limitation “the element of the kernel” on lines 12-13. It is unclear if this recitation of “the element of the kernel” is the same as the recited limitation “each weight of the kernel” as recited on line 8 of claim 10. Appropriate correction is required. For examination purposes, “the element of the kernel” as recited on lines 12-13 of claim 10 will be construed to be “each weight of the kernel” as recited on line 8 of claim 10.
Because claims 11-13 depend upon claim 10, claims 11-13 are additionally rejected under 35 U.S.C. 112(b), or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite.
Claim 12 recites the limitation “elements of the kernel” on line 2. It is unclear if this recitation of “elements of the kernel” is the same as the recited limitation “each weight of the kernel” as recited on line 8 of claim 10. Appropriate correction is required. For examination purposes, “elements of the kernel” as recited on line 2 of claim 12 will be construed to be “each weight of the kernel” as recited on line 8 of claim 10.
Because claim 13 depends upon claim 12, claims 13 is additionally rejected under 35 U.S.C. 112(b), or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite.
Claim 13 recites the limitation “the output” on line 4. There is lack of antecedent basis fo r this limitation. Appropriate correction is required.
Claim 13 recites the limitation “the result” on line 5. There is lack of antecedent basis for this limitation. Appropriate correction is required.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim 1 is rejected under 35 U.S.C. 102(a)(1) as being anticipated by S. Han et al. ("EIE: Efficient Inference Engine on Compressed Deep Neural Network", 2016) (hereinafter “Han”).
Han teaches:
A computer-implemented method for performing computations in a neural network, comprising (Pg. 10 Col. 1 Section C Line 1, designed for neural networks):
receiving, by a computer processor, an input patch of data (Pg. 4 Fig. 4 shows the internal architecture of each of the processing elements which take as input the vector/matrix to be processed for operations, for computations of the FC layer, also discussed in Pg. 12 Col. 1 Lines 7-10), where the input patch is a vector or a matrix extracted from an input and each element of the vector or the matrix is represented with M digits in accordance with a numeral system (Pg. 4 Col. 1 Section C Lines 10-12 input activation i.e., input patch of length 8 bits; input activations from main input activation vector i.e., patch distributed to the various Pes as discussed in Pg. 5 Col1 Lines 38-39 (starting with "Distributed Leading Non Zero Detection"));
retrieving, by the computer processor, a kernel of the neural network (Pg. 1 Fig. 1 retrieving of virtual weights i.e., kernel from weights memory; further discussed in Pg. 7 Col. 1 Lines 1-6), where each weight of the kernel is represented with N digits in accordance with a numeral system (Pg. 3 Col. 1 Section A Lines 13-14 weights are represented by floating point numeral system, each weight being 4 bits as discussed in Pg. 2 Col. 1 Lines 3-4); and
computing, by the computer processor, a multiplication between elements of the input patch and elements of the kernel of the neural network (Pg. 3 Col. 1 Section A Lines 15-16 and Equation 2 (Wij as kernel weight, aj as input from vector)), where nonzero digits of at least one of the elements of the input patch or the element of the kernel is constrained to less than M or less than N, respectively (Pg. 9 Col. 1 Section A Lines 27-30 70% of vector/activations/input are 0 digits, and so the number of non-zeros would be less than M, the same applies to W which are the weights and so the number of nonzero digits would be less than N).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 2, 4-7 and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Han in view of R. Xiao et al. ("A Low-Power In-Memory Multiplication and Accumulation Array with Modified Radix-4 Input and Canonical Signed Digit Weights", January 2021).
Regarding claim 2, while Han teaches the method of claim 1, Han does not explicitly teach elements of the vector or matrix as a two’s complement representation or weights being quantized as a canonical signed digit.
However, Xiao teaches:
wherein each element of the vector or the matrix is a two’s complement representation having M bits and each weight of the kernel is quantized as a canonical signed digit with N digits (Xiao: Pg. 5 Col. 2 Section B Lines 9-12 twos complement representation for activations/inputs and has M=8 bit length; Pg. 6 Col. 2 Lines 1-5 weights in CSD representation with at most 2 nonzero digits), such that the non-zero digits of the canonical signed digit are constrained to less than N (Pg. 6 Col. 2 Lines 1-5 weights in CSD representation with at most 2 nonzero digits).
It would be obvious to combine the two’s complement representation and canonical signed digit quantization as taught by Xiao with the method as taught by Han as both teachings are directed towards multiplication for neural networks. One with ordinary skill in the art would be motivated to combine the teachings because this would lead to a more balanced operand (Xiao: Pg. 5 Col. 2 Section B Lines 9-12).
Regarding claim 4, Han teaches:
A computer-implemented method for performing computations in a neural network (Han: Pg. 10 Col. 1 Section C Line 1, designed for neural networks), comprising:
receiving, by a computer processor, an input patch of data, (Han: Pg. 4 Fig. 4 shows the internal architecture of each of the processing elements which take as input the vector/matrix to be processed for operations, for computations of the FC layer, also discussed in Pg. 12 Col. 1 Lines 7-10) where the input patch is a vector or a matrix extracted from an input and each element of the vector or the matrix is represented by a binary number (Han: Pg. 4 Col. 1 Section C Lines 10-12 input activation i.e., input patch of length 8 bits; input activations from main input activation vector i.e., patch distributed to the various PEs as discussed in Pg. 5 Col. 1 Lines 38-39 (starting with "Distributed Leading Non Zero Detection"));
retrieving, by the computer processor, a kernel of the neural network (Han: Pg. 1 Fig. 1 retrieving of virtual weights i.e., kernel from weights memory; further discussed in Pg. 7 Col. 1 Lines 1-6),
computing, by the computer processor, a multiplication between elements of the input patch and elements of the kernel of the neural network (Han: Pg. 3 Col. 1 Section A Lines 15-16 and Equation 2 (Wij as kernel weight, aj as input from vector).
Han does not explicitly teach:
where each weight of the kernel is quantized as a canonical signed digit with N digits and non-zero digits of the canonical signed digit are constrained to less than N.
However, Xiao teaches:
where each weight of the kernel is quantized as a canonical signed digit with N digits and non-zero digits of the canonical signed digit are constrained to less than N (Pg. 6 Col. 2 Lines 1-5 weights in CSD representation with at most 2 nonzero digits).
The motivation to combine with respect to claim 2 applies equally to claim 4.
Regarding claim 5, Han in view of Xiao further teaches:
The method of claim 4, wherein each element of the vector or the matrix is a two's complement representation having M bits (Xiao: Pg. 5 Col. 2 Section B Lines 9-12 twos complement representation for activations/inputs and has M=8 bit length).
The motivation to combine with respect to claim 2 applies equally to claim 5.
Regarding claim 6, Han in view of Xiao teaches:
The method of claim 4, wherein each kernel weight is further defined as a canonical signed digit with 8 bits and no more than two non-zero digits and each element of the matrix is a two's complement representation with 8 bits Xiao: Pg. 6 Col. 2 Lines 1-5 weights in CSD representation with at most 2 nonzero digits; Pg. 6 Col. 1 Step 1) weights and inputs are 8-bit binary numbers).
The motivation to combine with respect to claim 2 applies equally to claim 6.
Regarding claim 7, Han further teaches 16 bit precision used for arithmetic operations (Han: Pg. 8 Col. 2 Section “Arithmetic Precision”), arithmetic including addition as shown in Pg. 4 Fig. 4 with multiplication followed by addition in the Arithmetic unit block, and computations taking place with respect to non-zeros (Han: Pg. 9 Col. 1 Lines 7-9). Additionally, Han teaches 4 pipeline stages for operations also shown in Pg. 4 Fig. 4, with a shift and add stage corresponding to the arithmetic unit, with shifting corresponding to the multiplication and 16 bit addition as discussed before corresponding to the adding (Han: Pg. 6 Col. 2 Section IV Lines 3-7).
Han does not explicitly teach the non-zero digits being constrained to two non-zero digits.
However, Xiao teaches non-zero digits being in the quantity of two (Xiao: Pg. 6 Col. 2 Lines 1-5 weights in CSD representation with at most 2 nonzero digits).
The motivation to combine with respect to claim 2 applies equally to claim 7.
Regarding claim 9, Han in view of Xiao teaches:
The method of claim 4, further comprises accumulating partial results from multiplying the elements of the input patch by elements of the kernel in a register and feeding the accumulated results to a next layer of the neural network (Han: Pg. 4 Fig. 4 Act R/W section has registers taking partial products from adder, and sending products to ReLU and next layers, also discussed in Pg. 5 Col. 1 Section Activation Read/Write Lines 4-5).
Claims 3, and 10-11 are rejected under 35 U.S.C. 103 as being unpatentable over Han in view of Gross et al. (US 2021/0256389) (hereinafter “Gross”).
While Han teaches the method of claim 1, Han does not explicitly teach elements of the vector or matrix being represented by a sign and magnitude representation.
However, Gross teaches:
The method of claim 1 wherein each element of the vector or the matrix is represented by a sign and magnitude representation with M bits (Gross: ¶ 0038 input/activation values have sign bit and magnitude, thus they are in sign magnitude format); and
each weight of the kernel is represented by a sign and magnitude representation with N bits, such that non-zero bits of at least one of the elements of the input patch or the element of the kernel is constrained to less than M or less than N, respectively (Gross: ¶ 0036 weights in sign magnitude representation; ¶ 0034 shows representation of bits, nonzero digits being less than M or N).
It would be obvious to combine the sign magnitude representation as taught by Gross with the method as taught by Han as both teachings are directed towards multiplication in neural networks. One with ordinary skill in the art would be motivated to combine the teachings because doing so would enhance the accuracy of computations (Gross: ¶ 0036).
Regarding claim 10, Han teaches:
A computer-implemented method for performing computations in a neural network (Han: Pg. 10 Col. 1 Section C Line 1, designed for neural networks), comprising:
receiving, by a computer processor, an input patch of data (Han: Pg. 4 Fig. 4 shows the internal architecture of each of the processing elements which take as input the vector/matrix to be processed for operations, for computations of the FC layer, also discussed in Pg. 12 Col. 1 Lines 7-10), where the input patch is a vector or a matrix extracted from an input (Han: Pg. 4 Col. 1 Section C Lines 10-12 input activation i.e., input patch of length 8 bits; input activations from main input activation vector i.e., patch distributed to the various PEs as discussed in Pg. 5 Col. 1 Lines 38-39 (starting with "Distributed Leading Non Zero Detection"));
retrieving, by the computer processor, a kernel of the neural network (Han: Pg. 1 Fig. 1 retrieving of virtual weights i.e., kernel from weights memory; further discussed in Pg. 7 Col. 1 Lines 1-6); and
computing, by the computer processor, a multiplication between elements of the input patch and weights of the kernel of the neural network (Han: Pg. 3 Col. 1 Section A Lines 15-16 and Equation 2 (Wij as kernel weight, aj as input from vector)), where nonzero bits of at least one of the elements of the input patch or the element of the kernel is constrained to less than M or less than N, respectively (Han: Pg. 9 Col. 1 Section A Lines 27-30 70% of vector/activations/input are 0 digits, and so the number of non-zeros would be less than M, the same applies to W which are the weights and so the number of nonzero digits would be less than N).
Han does not explicitly teach:
and each element of the vector or the matrix is represented by a sign and magnitude representation with M bits
where each weight of the kernel is represented by a sign and magnitude representation with N bits.
However, Gross teaches:
and each element of the vector or the matrix is represented by a sign and magnitude representation with M bits (Gross: ¶ 0038 input/activation values have sign bit and magnitude; thus, they are in sign magnitude format)
where each weight of the kernel is represented by a sign and magnitude representation with N bits (Gross: ¶ 0036 weights in sign magnitude representation; ¶ 0034 shows representation of bits, non-zero digits being less than M or N).
The motivation to combine with respect to claim 3 applies equally to claim 10.
Regarding claim 11, Han in view of Gross further teaches:
The method of claim 10, wherein each multiplication is implemented by a sign and magnitude multiplier circuits (Gross: ¶ 0038 discusses the use of sign magnitude multipliers).
The motivation to combine with respect to claim 3 applies equally to claim 11.
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Han in view of Gross further in view of Elliott (US 2021/0042086 A1) (hereinafter “Elliott”).
The method of claim 10 wherein computing a multiplication between elements of the input patch and elements of the kernel further comprises multiplying in parallel elements of the input patch by elements of the kernel using a plurality of multiplier circuits (Han: Pg. 6 Col. 2 Section VI Lines 3-7 multiply activations by weights i.e., kernel values in parallel).
Han does not explicitly teach:
inputting, from the plurality of multiplier circuits, products with positive results into a positive adder tree circuit;
inputting, from the plurality of multiplier circuits, products with negative results into a negative adder tree circuit; and
subtracting sum of the negative adder tree circuit from sum of the positive adder tree circuit thereby yielding a final product.
However, Elliott teaches dual adder trees, one adder tree for summing positive values and another adder tree for summing negative values, and combining the results of the two adder trees by subtracting in order to yield a final result (Elliott: Fig. 5 element 320 adder tree for positive values, element 325 adder tree for negative values, and element 332 subtraction operation with the two results from the respective adder trees in order to yield a final result; ¶ 0110 - ¶ 0113 discusses functionality of the respective adder trees and the final subtraction operation).
In combining Elliott with Han in view of Gross, the adder tree configuration followed by subtracting of the corresponding sums of Elliott would take the place of the adder within the arithmetic unit pipeline stage of Han (Han: Pg. 4 Fig. 4) in order to more efficiently handle the weights and inputs in sign magnitude representation as taught by Gross (as previously explained with respect to claim 3 and claim 10).
It would be obvious to combine the dual adder trees and subtractor configuration as taught by Elliott with the method as taught by Han in view of Gross as all teachings are directed towards the implementation of computer-based arithmetic. One with ordinary skill in the art would be motivated to combine the teachings because this would allow for reduced latency, less hardware area, and reduced power consumption (Elliott: ¶ 0103).
Han in view of Gross in view of Elliott therefore teaches:
The method of claim 10 wherein computing a multiplication between elements of the input patch and elements of the kernel further comprises
multiplying in parallel elements of the input patch by elements of the kernel using a plurality of multiplier circuits;
inputting, from the plurality of multiplier circuits, products with positive results into a positive adder tree circuit;
inputting, from the plurality of multiplier circuits, products with negative results into a negative adder tree circuit; and
subtracting sum of the negative adder tree circuit from sum of the positive adder tree circuit thereby yielding a final product.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Han in view of Gross in view of Elliott further in view of Darvish Rouhani et al. (US 2022/0405571 A1) (hereinafter “Darvish”).
Han teaches:
The method of claim 12 further comprises accumulating final products (Han: Accumulation occurs after multiply accumulate operations as discussed in Pg. 5 Col. 1 Section “Arithmetic Unit” Lines 1-6),
computing a non-linear layer on the accumulated final products (Han: Pg. 4 Fig. 4 final products computed in ReLU i.e., nonlinear layer),
and feeding the result to a next layer of the neural network (Han: Pg. 5 Col. 1 “Activation Read/Write” Section Lines 4-5 feeding to next layer).
Han does not explicitly teach:
representing each non-linear layer output in a sign and magnitude form with M bits,
processing the output with a bit sparsification circuit that reduces the number of non-zero bits to less than M.
However, Gross teaches:
representing each non-linear layer output in a sign and magnitude form with M bits (Gross: ¶ 0047 summation after multiplication that is used to be output to next layer i.e., output value of layer has a sign bit as well as magnitude and so the output has a sign and magnitude form).
The motivation to combine with respect to claim 3 applies equally to claim 13.
Han in view of Gross does not explicitly teach:
processing the output with a bit sparsification circuit that reduces the number of non-zero bits to less than M.
However, Darvish teaches:
processing the output with a bit sparsification circuit that reduces the number of non-zero bits to less than M (Darvish: ¶ 0042 after matrix/vector multiplication, output is sent to sparsification layer to sparsify bit values i.e., reduce the number of non-zero bits).
It would be obvious to combine the sparsification as taught by Darvish with the method as taught by Han in view of Gross in view of Elliott as all teachings are directed towards computer-based arithmetic implementation. One with ordinary skill in the art would be motivated to combine the teachings because this would reduce the size of the neural network and would thus decrease the memory or storage required for computations (Darvish: ¶ 0003).
Allowable Subject Matter
Claim 8 would be allowable if rewritten to overcome the rejection(s) under 35 USC 101, and 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
Applicant claims a method for performing computations in a neural network, wherein the method as in claim 4 comprises:
receiving, by a computer processor, an input patch of data, where the input patch is a vector or a matrix extracted from an input and each element of the vector or the matrix is represented by a binary number;
retrieving, by the computer processor, a kernel of the neural network, where each weight of the kernel is quantized as a canonical signed digit with N digits and non-zero digits of the canonical signed digit are constrained to less than N;
computing, by the computer processor, a multiplication between elements of the input patch and elements of the kernel of the neural network.
Wherein claim 6 is dependent on claim 1 further comprising:
The method of claim 4 wherein each kernel weight is further defined as a canonical signed digit with 8 bit and no more than two non-zero digits and each element of the matrix is a two’s complement representation with 8 bits.
Wherein claim 8 is dependent on claim 6 further comprising:
The method of claim 6 wherein computing a multiplication operation includes
multiplying a given element the input patch by sign of each non-zero digit of the cannonical signed digit to yield two products from a first stage;
bit shifting products from the first stage in a second stage, where the bit shifting amount is based on position of non-zero digits in the canonical signed digit; and
adding products from the second stage together.
Han teaches multiply-accumulate operations of vectors taken from an initial input through the use of four pipeline stages, each of the stages implemented within an array of processing elements. Han is silent as to the multiplication of a sign of each of the non-zero digits of the canonical signed digit in order to yield two products from a first stage and bit shifting products based on a position of the non-zero digits.
Xiao teaches low-power multiplication with weights and inputs having a canonical signed digit representation in order to further reduce power consumption during computations. Xiao is silent as to the multiplication of a sign of each of the non-zero digits of the canonical signed digit in order to yield two products from a first stage and bit shifting products based on a position of the non-zero digits.
Prior Art Made of Record
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Chen et al. (“HTNN: Deep Learning in Heterogeneous Transform Domains with Sparse-Orthogonal Weights”, 2021) teaches replacing convolutions with element-wise multiplications between Walsh-Hadamard transformed input patches and kernels, after which the inputs are inverse transformed in order to obtain filtered output for the output feature.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MARIA DE JESUS RIVERA whose telephone number is (571)272-2793. The examiner can normally be reached Monday-Friday 7:30AM-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, James Trujillo can be reached at (571) 272-3677. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/M.D.R./Examiner, Art Unit 2151
/James Trujillo/Supervisory Patent Examiner, Art Unit 2151