DETAILED ACTION
This action is responsive to claims filed on 30 May 2023.
Claims 1-20 are pending for examination.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 3 and analogous claims 13, 19 are objected to because of the following informalities: “each of which” in line 6 should be “the sequence of bits”. Appropriate correction is required.
Claim 3 and analogous claims 13, 19 are objected to because of the following informalities: “each of which” in line 9 should be “the sequence of bits”. Appropriate correction is required.
Claim 5 is objected to because of the following informalities: “each of which” in line 4 should be “the sequence of bits”. Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception, abstract idea, without significantly more.
Step 1: This part of the eligibility analysis evaluates whether the claim(s) falls within any statutory
category. MPEP 2106.03:
According to the first part of the Alice analysis, in the instant case, the claims were determined
to be directed to one of the four statutory categories: an article of manufacture, a method/process (Claims 1-10), a machine/system/product (Claims 11-20), and a composition of matter. Based on the claims being determined to be within of the four categories (i.e., process, machine, manufacture, or composition of matter), (Step 1), it must be determined if the claims are directed to a judicial exception (i.e., law of nature, natural phenomenon, and abstract idea).
Step 2A Prong One: This part of the eligibility analysis evaluates whether the claim(s) recites a
judicial exception.
Regarding independent claims 1, 11, 17, the claims recite a judicial exception (i.e., an abstract idea enumerated in the 2019 PEG) without significantly more (Step-2A: Prong One). The applicant's claim limitations under broadest reasonable interpretation covers activities classified under mental processes - concepts performed in the human mind (including an observation, evaluation, judgment, opinion) (see MPEP § 2106.04(a)(2), subsection Ill) and the 2019 PEG. As evaluated below:
Claims 1, 11, 17:
“wherein the multiplier computes a product by multiplying the first element of the activation operation with the first element of the weight operand” (mental process of evaluation using abstract idea of mathematical function)
“determining whether a second element of the activation operand or a second element of the weight operand is zero valued” (mental process of judgement)
“after determining that the second element of the activation operand or the second element of the weight operand is zero valued” (mental process of judgement)
If the identified limitation(s) falls within at least one of the groupings of abstract ideas, it is
reasonable to conclude that the claim(s) recites an abstract idea in Step 2A Prong One.
Step 2A Prong Two: This part of the eligibility analysis evaluates whether the claim(s) as a whole integrates the recited judicial exception into a practical application of the exception. As evaluated below:
“writing a zero-valued data element into the second storage unit”
These recitations are deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to instructions for mere data gathering or data output, see MPEP 2106.05(g).
“storing a first element of an activation operand of a deep learning operation and a first element of a weight operand of the deep learning operation in one or more first storage units associated with a multiplier”
“storing the product in a second storage unit”
“keeping the first element of the activation operand and the first element of the weight operand in the one or more first storage units”
These recitations are deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to instructions merely indicating a field of use or technological environment in which to apply a judicial exception, see MPEP 2106.05(h).
Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea when considered as an ordered combination and as a whole.
Step 2B: This part of the eligibility analysis evaluates whether the claim, as a whole, amounts to
significantly more than the recited exception, i.e., whether any additional element, or combination of
additional elements, adds an inventive concept to the claim. MPEP 2106.05.
First, the additional elements considered as part of the preamble and the additional elements
directed to the use of computer technology are deemed insufficient to transform the judicial exception
to a patentable invention to a patentable invention because they generally link the judicial exception to
the technology environment, see MPEP 2106.05(h).
Second, the additional elements directed to mere application of the abstract idea or mere instructions to implement an abstract idea on a computer are deemed insufficient to transform the judicial exception to a patentable invention to a patentable invention because the limitations generally apply the use of a generic computer and/or process with the judicial exception, see MPEP 2106.05(f).
Third, the claims are directed to instructions merely indicating a field of use or technological environment in which to apply a judicial exception. The courts have found these types of limitations insufficient to transform the judicial exception to a patentable invention, see MPEP 2106.05(g).
Lastly, the claims directed to data gathering activity as noted above, are deemed directed to an insignificant extra-solution activity. The courts have found these types of limitations insufficient to
qualify as "significantly more", see MPEP 2106.05(g).
Furthermore, when considering evidence in view of Berkheimer v. HP, Inc., 881 F.3d 1360, 1368, 125 USPQ2d 1649, 1654 (Fed. Cir. 2018), see USPTO Berkheimer Memorandum (April 2018). Examiner notes Berkheimer: Option 2 - A citation to one or more of the court decisions discussed in MPEP § 2106.05(d}(II} as noting the well understood, routine, conventional nature of the additional element (s) (e.g., limitations directed to mere data gathering):
The courts have recognized the following computer functions as well understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity, see MPEP 2106.05(d).
The additional limitations, as analyzed, failed to integrate a judicial exception into a practical application at Step 2A and provide an inventive concept in Step 2B, per the analysis above. Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible. Therefore, in examining elements as recited by the limitations individually and as an ordered combination, as a whole, claims 1, 11, 17 do not recite what the courts have identified as "significantly more".
Furthermore, regarding dependent claims 2-10, which depend from claim 1, claims 12-16, which depend from claim 11, claims 18-20, which depend from claim 17, the claims are directed to a judicial exception (i.e., an abstract idea enumerated in the 2019 PEG, a law of nature, or a natural phenomenon) without significantly more as highlighted below in the claim limitations by evaluating the claim limitations under the Step2A and 2B:
Claims 2, 12, 18:
Incorporates the rejections of claims 1, 11, 17, respectively.
“wherein determining whether the second element of the activation operand or the second element of the weight operand is zero valued comprises” (mental process of judgement)
“performing a logical operation on the second element of the activation operand or the second element of the weight operand” (mental process of evaluation)
The recitation is directed to mere instructions to implement an abstract idea on a computer, or
merely uses a computer as a tool to perform an abstract idea and are considered to adding the words "apply it" (or an equivalent) with the judicial exception, See MPEP 2106.05(f).
Limitations directed to mere instructions to implement an abstract idea on a computer/using computer as a tool cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Claims 3, 13, 19:
Incorporates the rejections of claims 1, 11, 17, respectively.
“wherein determining whether the second element of the activation operand or the second element of the weight operand is zero valued comprises” (mental process of judgement)
“determining whether the second element of the activation operand or the second element of the weight operand is zero valued based on an activation bitmap or a weight bitmap” (mental process of judgement)
The recitation is directed to mere instructions to implement an abstract idea on a computer, or
merely uses a computer as a tool to perform an abstract idea and are considered to adding the words "apply it" (or an equivalent) with the judicial exception, See MPEP 2106.05(f).
“the activation bitmap comprises a sequence of bits, each of which corresponds to a respective element of the activation operand and indicates whether the respective element of the activation operand is zero valued”
“the weight bitmap comprises a sequence of bits, each of which corresponds to a respective element of the weight operand and indicates whether the respective element of the weight operand is zero valued”
These recitations are deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to instructions merely indicating a field of use or technological environment in which to apply a judicial exception, see MPEP 2106.05(h).
Limitations directed to mere instructions to implement an abstract idea on a computer/using computer as a tool or directed to mere instructions indicating a field of use or technological environment in which to apply a judicial exception cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Claim 4:
Incorporates the rejection of claim 3.
“wherein determining whether the second element of the activation operand or the second element of the weight operand is zero valued comprises” (mental process of judgement)
“performing a logical operation on a bit in the activation bitmap and a bit in the weight bitmap” (mental process of evaluation)
The recitation is directed to mere instructions to implement an abstract idea on a computer, or
merely uses a computer as a tool to perform an abstract idea and are considered to adding the words "apply it" (or an equivalent) with the judicial exception, See MPEP 2106.05(f).
“the bit in the activation bitmap corresponding to the second element of the activation operand”
“the bit in the weight bitmap corresponding to the second element of the weight operand”
These recitations are deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to instructions merely indicating a field of use or technological environment in which to apply a judicial exception, see MPEP 2106.05(h).
Limitations directed to mere instructions to implement an abstract idea on a computer/using computer as a tool or directed to mere instructions indicating a field of use or technological environment in which to apply a judicial exception cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Claim 5:
Incorporates the rejection of claim 3.
“wherein determining whether the second element of the activation operand or the second element of the weight operand is zero valued comprises” (mental process of judgement)
“computing a combined bitmap based on the activation bitmap and the weight bitmap” (mental process of evaluation)
“determining whether the second element of the activation operand or the second element of the weight operand is zero valued based on the combined bitmap” (mental process of judgement)
The recitation is directed to mere instructions to implement an abstract idea on a computer, or
merely uses a computer as a tool to perform an abstract idea and are considered to adding the words "apply it" (or an equivalent) with the judicial exception, See MPEP 2106.05(f).
“the combined bitmap comprising a sequence of bits, each of which is a product of a bit in the activation bitmap and a bit in the weight bitmap”
These recitations are deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to instructions merely indicating a field of use or technological environment in which to apply a judicial exception, see MPEP 2106.05(h).
Limitations directed to mere instructions to implement an abstract idea on a computer/using computer as a tool or directed to mere instructions indicating a field of use or technological environment in which to apply a judicial exception cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Claims 6, 14:
Incorporates the rejections of claims 1, 11, respectively.
“wherein determining whether the second element of the activation operand or the second element of the weight operand is zero valued comprises” (mental process of judgement)
“determining whether a value of the second element of the activation operand or the second element of the weight operand is no greater than a threshold” (mental process of judgement)
The recitation is directed to mere instructions to implement an abstract idea on a computer, or
merely uses a computer as a tool to perform an abstract idea and are considered to adding the words "apply it" (or an equivalent) with the judicial exception, See MPEP 2106.05(f).
Limitations directed to mere instructions to implement an abstract idea on a computer/using computer as a tool cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Claims 7, 15:
Incorporates the rejections of claims 1, 11, respectively.
“wherein writing the zero-valued data element into the second storage unit comprises: writing the zero-valued data element into the second storage unit after a pipeline in the multiplier is completed”
These recitations are deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to instructions for mere data gathering or data output, see MPEP 2106.05(g).
Limitations directed to instructions for mere data gathering or data output cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Claim 8:
Incorporates the rejection of claim 1.
“wherein after determining that the second element of the activation operand or the second element of the weight operand is zero valued” (mental process of judgement)
“in response to determining that the second element of the activation operand or the second element of the weight operand is zero valued” (mental process of judgement)
The recitation is directed to mere instructions to implement an abstract idea on a computer, or
merely uses a computer as a tool to perform an abstract idea and are considered to adding the words "apply it" (or an equivalent) with the judicial exception, See MPEP 2106.05(f).
“keeping the first element of the activation operand and the first element of the weight operand in the one or more first storage units comprises”
“reducing gate switching for the deep learning operation”
These recitations are deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to instructions merely indicating a field of use or technological environment in which to apply a judicial exception, see MPEP 2106.05(h).
Limitations directed to mere instructions to implement an abstract idea on a computer/using computer as a tool or directed to mere instructions indicating a field of use or technological environment in which to apply a judicial exception cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Claims 9, 16, 20:
Incorporates the rejections of claims 1, 11, 17, respectively.
“after determining that the second element of the activation operand or the second element of the weight operand is zero valued” (mental process of judgement)
The recitation is directed to mere instructions to implement an abstract idea on a computer, or
merely uses a computer as a tool to perform an abstract idea and are considered to adding the words "apply it" (or an equivalent) with the judicial exception, See MPEP 2106.05(f).
“transmitting the product to an accumulator”
These recitations are deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to instructions for mere data gathering or data output, see MPEP 2106.05(g).
“disabling the accumulator”
These recitations are deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to instructions merely indicating a field of use or technological environment in which to apply a judicial exception, see MPEP 2106.05(h).
Limitations directed to mere instructions to implement an abstract idea on a computer/using computer as a tool or directed to instructions for mere data gathering or data output or directed to instructions merely indicating a field of use or technological environment in which to apply a judicial exception cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Claim 10:
Incorporates the rejection of claim 1.
“the multiplier computes the product in a first clock cycle” (mental process of evaluation using abstract idea of mathematical function)
The recitation is directed to mere instructions to implement an abstract idea on a computer, or
merely uses a computer as a tool to perform an abstract idea and are considered to adding the words "apply it" (or an equivalent) with the judicial exception, See MPEP 2106.05(f).
“the zero-valued data element is written into the second storage unit in a second clock cycle after the first clock cycle”
These recitations are deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to instructions for mere data gathering or data output, see MPEP 2106.05(g).
“the first element of the activation operand is arranged before the second element of the activation operand”
“the first element of the activation operand and the first element of the weight operand are stored in the one or more first storage units in the first clock cycle and the second clock cycle”
These recitations are deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to instructions merely indicating a field of use or technological environment in which to apply a judicial exception, see MPEP 2106.05(h).
Limitations directed to mere instructions to implement an abstract idea on a computer/using computer as a tool or directed to instructions for mere data gathering or data output or directed to instructions merely indicating a field of use or technological environment in which to apply a judicial exception cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
The dependent claims as analyzed above, do not recite limitations that integrated the judicial exception into a practical application. In addition, the claim limitations do not include additional elements that are sufficient to amount to significantly more than the judicial exception (Step-2B). Therefore, the claims do not recite any limitations, when considered individually or as a whole, that recite what have the courts have identified as "significantly more", see MPEP 2106.05; and therefore, as a whole the claims are not patent eligible. As shown above, the dependent claims do not provide any additional elements that when considered individually or as an ordered combination, amount to significantly more than the abstract idea identified. Therefore, as a whole, the dependent claims do not recite what have the courts have identified as "significantly more" than the recited judicial exception. Therefore, claims 2-10, 12-16, 18-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception and does not recite, when claim elements are examined individually and as a whole, elements that the courts have identified as "significantly more" than the recited judicial exception.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-5, 7, 10-13, 15, 17-19 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Raha et al. (U.S. Pre-Grant Publication No. 20210326144, hereinafter ‘Raha').
Regarding claim 1 and analogous claims 11, 17, Raha teaches A method for deep learning, comprising: storing a first element of an activation operand of a deep learning operation and a first element of a weight operand of the deep learning operation in one or more first storage units associated with a multiplier ([0081] The example data receiver 604 of FIG. 6 receives the compressed machine learning parameter data 102. Depending on the example data controller used, the compressed activation and weight data 102A may contain both useful and not useful sections. Depending on the example data controller used, the data receiver 604 may not receive compressed activation and weight data every round. The data receiver 604 will receive sparsity bitmap data every round. The data receiver 604 storing a first element of an activation operand of a deep learning operation and a first element of a weight operand of the deep learning operation in one or more first storage units associated with a multiplier stores the compressed machine learning parameter data 102 in local memory 606.),
wherein the multiplier computes a product by multiplying the first element of the activation operation with the first element of the weight operand; storing the product in a second storage unit ([0083] The example MAC operator 610 of FIG. 6 represents one way that decompressed activation and weight data is used within the example processor engine 110. The example MAC operator 610 wherein the multiplier computes a product by multiplying the first element of the activation operation with the first element of the weight operand multiplies different sections of activation and weight data together to form multiple products. The example MAC operator 610 also sums the multiple products. The resulting sum of multiple storing the product in a second storage unit products produced by the example MAC operator 610 is saved in local memory 606.);
determining whether a second element of the activation operand or a second element of the weight operand is zero valued ([0102] The example zero compute load skipper 4B04 determines if the machine learning parameter data 102 from Block 802 will produce a nonzero MAC value (Block 808).);
after determining that the second element of the activation operand or the second element of the weight operand is zero valued, keeping the first element of the activation operand and the first element of the weight operand in the one or more first storage units ([0104] If the example zero compute load skipper 4B04 after determining that the second element of the activation operand or the second element of the weight operand determines the decompressed activation and weight data will produce nonzero MAC results, then the compressed activation and weight data of block 802 is sent to the data provider (Block 810). If the example value is zero valued is equal to 0, keeping the first element of the activation operand and the first element of the weight operand in the one or more first storage units Block 810 is skipped and data is not sent to the data provider in the current round.; [0070] While the known data controller 200 loads both useful and not useful portions of the compressed machine learning parameter data 102 into the example processor engine 110, the example data controller 4B00 leverages the zero compute load skipper 4B04 to only load useful portions of the compressed machine learning parameter data 102 into the example processor engine 110.); and
writing a zero-valued data element into the second storage unit ([0113] FIG. 10 is a flowchart representative of machine readable instructions which may be executed to implement a (MAC) operation using two values, A and B. The process 1010 begins when the example MAC operator 610 acquires two values A and B, both of length n (Block 1000). Generally, MAC operators may compute any two values of equal length. In the example process 900, A and B refer to the decompressed activation and decompressed weight data from the MAC operations 906, 908.; [0117] The example MAC operator 610 adds the multiplication result of 1006 to the current value of the MAC result and writing a zero-valued data element into the second storage unit stores the new MAC result value in local memory 606 (Block 1008).).
Regarding claim 2 and analogous claims 12, 18, Raha teaches The method of claim 1, The one or more non-transitory computer-readable media of claim 11, The apparatus of claim 17, respectively.
Raha teaches wherein determining whether the second element of the activation operand or the second element of the weight operand is zero valued comprises: performing a logical operation on the second element of the activation operand or the second element of the weight operand ([0102] The example zero compute load skipper 4B04 determines if the machine learning parameter data 102 from Block 802 will produce a nonzero MAC value (Block 808). In some examples, the example zero compute load skipper 4B04 makes this determination by performing a bitwise AND function with the sparsity bitmap data 102B from Block 802. In this example, the bitwise AND function is with an activation bitmap and a weight bitmap, which both compose the sparsity bitmap data 102B from Block 802. A performing a logical operation on the second element of the activation operand or the second element of the weight operand bitwise AND function is performed by performing n AND operations over each index of the decompressed activation and weight data and summing the results together. Therefore, for any index i that is less than n, if the ith bit of the activation weight and ith bit of the weight data are both high valued (e.g., a value of 1), the result of the bitwise AND operation will be nonzero. Alternatively, if there is no index i that is less than n where both the ith bit of the activation weight and ith bit of the weight data are both high valued (e.g., a value of 1), then the result of the bitwise AND operation will be zero.).
Regarding claim 3 and analogous claims 13, 19, Raha teaches The method of claim 1, The one or more non-transitory computer-readable media of claim 11, The apparatus of claim 17, respectively.
Raha teaches wherein determining whether the second element of the activation operand or the second element of the weight operand is zero valued comprises: determining whether the second element of the activation operand or the second element of the weight operand is zero valued based on an activation bitmap or a weight bitmap ([0102] The example zero compute load skipper 4B04 determining whether the second element of the activation operand or the second element of the weight operand is zero valued determines if the machine learning parameter data 102 from Block 802 will produce a nonzero MAC value (Block 808). In some examples, the example zero compute load skipper 4B04 makes this determination by performing a bitwise AND function with the sparsity bitmap data 102B from Block 802. In this example, the bitwise AND function based on an activation bitmap is with an activation bitmap or a weight bitmap and a weight bitmap, which both compose the sparsity bitmap data 102B from Block 802. A bitwise AND function is performed by performing n AND operations over each index of the decompressed activation and weight data and summing the results together. Therefore, for any index i that is less than n, if the ith bit of the activation weight and ith bit of the weight data are both high valued (e.g., a value of 1), the result of the bitwise AND operation will be nonzero. Alternatively, if there is no index i that is less than n where both the ith bit of the activation weight and ith bit of the weight data are both high valued (e.g., a value of 1), then the result of the bitwise AND operation will be zero.),
wherein: the activation bitmap comprises a sequence of bits, each of which corresponds to a respective element of the activation operand and indicates whether the respective element of the activation operand is zero valued, and the weight bitmap comprises a sequence of bits, each of which corresponds to a respective element of the weight operand and indicates whether the respective element of the weight operand is zero valued ([0102] In this example, the bitwise AND function is with an activation bitmap and a weight bitmap, which both compose the sparsity bitmap data 102B from Block 802. A bitwise AND function is performed by performing the activation bitmap comprises a sequence of bits, each of which corresponds to a respective element of the activation operand and indicates whether the respective element of the activation operand is zero valued n AND operations over each index of the decompressed activation and the weight bitmap comprises a sequence of bits, each of which corresponds to a respective element of the weight operand and indicates whether the respective element of the weight operand is zero valued weight data and summing the results together. Therefore, for any index i that is less than n, if the ith bit of the activation weight and ith bit of the weight data are both high valued (e.g., a value of 1), the result of the bitwise AND operation will be nonzero. Alternatively, if there is no index i that is less than n where both the ith bit of the activation weight and ith bit of the weight data are both high valued (e.g., a value of 1), then the result of the bitwise AND operation will be zero.).
Regarding claim 4, Raha teaches The method of claim 3.
Raha teaches wherein determining whether the second element of the activation operand or the second element of the weight operand is zero valued comprises: performing a logical operation on a bit in the activation bitmap and a bit in the weight bitmap, the bit in the activation bitmap corresponding to the second element of the activation operand, the bit in the weight bitmap corresponding to the second element of the weight operand ([0102] In this example, the bitwise AND function is with an activation bitmap and a weight bitmap, which both compose the sparsity bitmap data 102B from Block 802. A performing a logical operation on a bit in the activation bitmap and a bit in the weight bitmap bitwise AND function is performed by performing n AND operations over each index of the decompressed the bit in the activation bitmap corresponding to the second element of the activation operand activation and the bit in the weight bitmap corresponding to the second element of the weight operand weight data and summing the results together. Therefore, for any index i that is less than n, if the ith bit of the activation weight and ith bit of the weight data are both high valued (e.g., a value of 1), the result of the bitwise AND operation will be nonzero. Alternatively, if there is no index i that is less than n where both the ith bit of the activation weight and ith bit of the weight data are both high valued (e.g., a value of 1), then the result of the bitwise AND operation will be zero.).
Regarding claim 5, Raha teaches The method of claim 3.
Raha teaches wherein determining whether the second element of the activation operand or the second element of the weight operand is zero valued comprises: computing a combined bitmap based on the activation bitmap and the weight bitmap, the combined bitmap comprising a sequence of bits, each of which is a product of a bit in the activation bitmap and a bit in the weight bitmap ([0102] The example zero compute load skipper 4B04 determines if the machine learning parameter data 102 from Block 802 will produce a nonzero MAC value (Block 808). In some examples, the example zero compute load skipper 4B04 makes this determination by performing a bitwise AND function with the sparsity bitmap data 102B from Block 802. In this example, the bitwise AND function is computing a combined bitmap based on the activation bitmap and the weight bitmap, the combined bitmap comprising a sequence of bits, each of which is a product of a bit in the activation bitmap and a bit in the weight bitmap with an activation bitmap and a weight bitmap, which both compose the sparsity bitmap data 102B from Block 802.); and
determining whether the second element of the activation operand or the second element of the weight operand is zero valued based on the combined bitmap ([0102] A bitwise AND function is performed by performing n AND operations over each index of the decompressed activation and weight data and summing the results together. Therefore, for any index i that is less than n, if the ith bit of the activation weight and ith bit of the weight data are both high valued (e.g., a value of 1), the result of the bitwise AND operation will be nonzero. Alternatively, if there is no index i that is less than n where both the determining whether the second element of the activation operand or the second element of the weight operand is zero valued based on the combined bitmap ith bit of the activation weight and ith bit of the weight data are both high valued (e.g., a value of 1), then the result of the bitwise AND operation will be zero.).
Regarding claim 7 and analogous claim 15, Raha teaches The method of claim 1, The one or more non-transitory computer-readable media of claim 11, respectively.
Raha teaches wherein writing the zero-valued data element into the second storage unit comprises: writing the zero-valued data element into the second storage unit after a pipeline in the multiplier is completed ([0047] In some examples, multiple clock cycles may occur within the time frame defined as a round. The number of clock cycles may change based on the after a pipeline in the multiplier is completed example pipeline architecture employed by the machine learning accelerator.; [0116] The example MAC operator 610 multiplies the ith least significant set of X bytes in A with the ith least significant set of X bytes in B (Block 1006). In some examples, the value of X will change depending on n, the length of A and B. The example multiplication result of 1006 is stored within local memory 606.; [0117] The example MAC operator 610 adds the multiplication result of 1006 to the current value of the MAC result and writing the zero-valued data element into the second storage unit stores the new MAC result value in local memory 606 (Block 1008).).
Regarding claim 10, Raha teaches The method of claim 1.
Raha teaches wherein: the first element of the activation operand is arranged before the second element of the activation operand, the multiplier computes the product in a first clock cycle ([0083] The example MAC operator 610 of FIG. 6 represents one way that decompressed activation and weight data is used within the example processor engine 110. The example the multiplier computes the product in a first clock cycle MAC operator 610 multiplies the first element of the activation operand is arranged before the second element of the activation operand different sections of activation and weight data together to form multiple products. The example MAC operator 610 also sums the multiple products. The resulting sum of multiple products produced by the example MAC operator 610 is saved in local memory 606.),
the zero-valued data element is written into the second storage unit in a second clock cycle after the first clock cycle [0117] The example MAC operator 610 the zero-valued data element is written into the second storage unit in a second clock cycle after the first clock cycle adds the multiplication result of 1006 to the current value of the MAC result and stores the new MAC result value in local memory 606 (Block 1008).), and
the first element of the activation operand and the first element of the weight operand are stored in the one or more first storage units in the first clock cycle and the second clock cycle ([0082] The example local memory 606 of FIG. 6 stores machine readable instructions 608, that may be executed and/or instantiated by processor circuitry to implement the decompression of compressed activation and weight data 102A, the data receiver 604, the MAC operator 610, and the data provider 612. Additionally, the the first element of the activation operand and the first element of the weight operand are stored in the one or more first storage units in the first clock cycle and the second clock cycle local memory 606 stores compressed machine learning parameter data 102 and results from the MAC operator 610.; [0109] In the example MAC operations 906, 908, the MAC operator 610 uses the current round's sparsity bitmap 304 to obtain decompressed activation and weight data. Each bit within the decompressed activation and weight data is used by the MAC operator 610 in both multiplication and summation operations.).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 6, 14 are rejected under 35 U.S.C. 103 as being unpatentable over Raha, in view of Dally et al. (U.S. Pre-Grant Publication No. 20180046900, hereinafter 'Dally').
Regarding claim 6 and analogous claim 14, Raha teaches The method of claim 1, The one or more non-transitory computer-readable media of claim 11, respectively.
Raha fails to teach wherein determining whether the second element of the activation operand or the second element of the weight operand is zero valued comprises: determining whether a value of the second element of the activation operand or the second element of the weight operand is no greater than a threshold.
Dally teaches wherein determining whether the second element of the activation operand or the second element of the weight operand is zero valued comprises: determining whether a value of the second element of the activation operand or the second element of the weight operand is no greater than a threshold ([0052] Sparsity in a layer of a CNN is defined as the fraction of zeros in the layer's weight and input activation matrices. The primary technique for creating weight sparsity is to prune the network during training. In one embodiment, any weight with an absolute value that is close to zero (e.g. below a defined threshold) is set to zero.).
Raha and Dally are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Raha, it would have been obvious for a person of ordinary skill in the art to apply the teachings of Dally to Raha before the effective filing date of the claimed invention in order to reduce energy consumption and improve processing throughput (cf. Dally, [0036] A sparse CNN (SCNN) accelerator architecture described herein, exploits weight and/or activation sparsity to reduce energy consumption and improve processing throughput. The SCNN accelerator architecture couples an algorithmic dataflow that eliminates all multiplications with a zero operand while employing a compressed representation of both weights and activations through almost the entire computation. In one embodiment, each non-zero weight and activation value is represented by a (value, position) pair.).
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Raha, in view of Ye et al. (NPL: "A Zero-Gating Processing Element Design for Low Power Deep Convolutional Neural Networks", hereinafter 'Ye').
Regarding claim 8, Raha teaches The method of claim 1.
Raha fails to teach wherein after determining that the second element of the activation operand or the second element of the weight operand is zero valued, keeping the first element of the activation operand and the first element of the weight operand in the one or more first storage units comprises: in response to determining that the second element of the activation operand or the second element of the weight operand is zero valued, reducing gate switching for the deep learning operation.
Ye teaches wherein after determining that the second element of the activation operand or the second element of the weight operand is zero valued, keeping the first element of the activation operand and the first element of the weight operand in the one or more first storage units comprises: in response to determining that the second element of the activation operand or the second element of the weight operand is zero valued, reducing gate switching for the deep learning operation ([IV. PROPOSED ZERO-GATING PE DESIGN, pg. 319] In our work, the CNN architecture as shown in Fig.2 was used for a general CNN implementation, in which an array of (9×64) PEs is included. The proposed zero-gating PE design is shown in Fig. 5. To explore the possible elimination of redundant computation, in our work the zeros in both the activation maps and the filters’ weights are taken into consideration for the purpose of low power consumption. The zero-gating logic is implemented through clock-gating by exploiting zeros in the inputs. The module (!=) means that each bit of both activations and weights are compared with zero. This module is made up with several connected OR gates. We set two control signals, one for weights and the other for activations in zero checking. in response to determining that the second element of the activation operand or the second element of the weight operand is zero valued If a zero input appears in either activations or weights, a clock gating signal is enabled to disable the update of the registers of activations and weights from reading new data and then reducing gate switching for the deep learning operation prevent the followed multiplier from switching for power saving. One multiplexer is set at the output of the adder to choose the corresponding result from the adder or the input partial sum when the clock gating signal is enabled. Fig. 6 shows an example of how the required computation can be eliminated through the proposed zero-gating method. Original PE computes all pairs of activation and weight. Fig. 6 (a) shows the input sequences of activations and weights. Fig. 6 (b) and (c) shows the required computation with and without the proposed zero-gating. From the figure, it can be clearly observed that there is only one MAC operation in the proposed zero-gating PE while four and two MAC operations are required in the baseline PEs without zero-gating and with zero-gating only in activations, respectively. The proposed zero-gating PE design works in full power only when neither activation nor weight is zero, which is promising for low power deep CNNs with no loss of precision.).
Raha and Ye are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Raha, it would have been obvious for a person of ordinary skill in the art to apply the teachings of Ye to Raha before the effective filing date of the claimed invention in order to eliminate redundant computation for power reduction (cf. Ye, [Abstract, pg. 317] Convolution neural networks (CNNs) have shown great success in many areas such as object detection and pattern recognition. However, the high computational complexity of state-of-the-art deep CNNs makes them extreme difficult to be run on resource-constrained mobile and wearable devices. To address this design challenge, in this paper we first analyzed the filters' weights of pre-trained models from four state-of-the-art CNNs. We found that in all the CNNs that we analyzed, from about 20% (AlexNet) to 43% (VGG-19) of the weights are zeros, which lead to redundant large amounts of computation. Then, based on this observation, a zero-gating processing element (PE) design was proposed for low-power deep CNNs, in which the vast number of zeros in both activation maps and filter weights are explored to eliminate redundant computation for power reduction. We implemented our proposal with VGG-16 using ImageNet dataset. Experiments were conducted for evaluations of area and total power consumption. Compared with the baseline PE design without zero-gating, overall the proposed zero-gating PE can achieve 37% power saving while the corresponding area overhead is less than 8%.).
Claims 9, 16, 20 are rejected under 35 U.S.C. 103 as being unpatentable over Raha, in view of Dally, and further in view of Ye.
Regarding claim 9 and analogous claims 16, 20, Raha teaches The method of claim 1, The one or more non-transitory computer-readable media of claim 11, The apparatus of claim 17, respectively.
Raha fails to teach further comprising: transmitting the product to an accumulator; and after determining that the second element of the activation operand or the second element of the weight operand is zero valued, disabling the accumulator.
Dally teaches further comprising: transmitting the product to an accumulator ([0041] At step 115, each one of the non-zero weight values is multiplied with every one of the non-zero input activation values, within a multiplier array, to produce a third vector of products.; [0042] At step 125, the transmitting the product to an accumulator third vector is transmitted to an accumulator array, where each one of the products in the third vector is transmitted to an adder in the accumulator array that is configured to generate an output activation value at the position associated with the product. In one embodiment, the third vector is transmitted through an array of buffers in the accumulator array, where each one of the buffers is coupled to an input of one of the adders in the accumulator array.); and
Raha and Dally are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Raha, it would have been obvious for a person of ordinary skill in the art to apply the teachings of Dally to Raha before the effective filing date of the claimed invention in order to reduce energy consumption and improve processing throughput (cf. Dally, [0036] A sparse CNN (SCNN) accelerator architecture described herein, exploits weight and/or activation sparsity to reduce energy consumption and improve processing throughput. The SCNN accelerator architecture couples an algorithmic dataflow that eliminates all multiplications with a zero operand while employing a compressed representation of both weights and activations through almost the entire computation. In one embodiment, each non-zero weight and activation value is represented by a (value, position) pair.).
Ye teaches after determining that the second element of the activation operand or the second element of the weight operand is zero valued, disabling the accumulator ([IV. PROPOSED ZERO-GATING PE DESIGN, pg. 319] In our work, the CNN architecture as shown in Fig.2 was used for a general CNN implementation, in which an array of (9×64) PEs is included. The proposed zero-gating PE design is shown in Fig. 5. To explore the possible elimination of redundant computation, in our work the zeros in both the activation maps and the filters’ weights are taken into consideration for the purpose of low power consumption. The zero-gating logic is implemented through clock-gating by exploiting zeros in the inputs. The module (!=) means that each bit of both activations and weights are compared with zero. This module is made up with several connected OR gates. We set two control signals, one for weights and the other for activations in zero checking. after determining that the second element of the activation operand or the second element of the weight operand is zero valued If a zero input appears in either activations or weights, a clock gating signal is enabled to disabling the accumulator disable the update of the registers of activations and weights from reading new data and then prevent the followed multiplier from switching for power saving. One multiplexer is set at the output of the adder to choose the corresponding result from the adder or the input partial sum when the clock gating signal is enabled. Fig. 6 shows an example of how the required computation can be eliminated through the proposed zero-gating method. Original PE computes all pairs of activation and weight. Fig. 6 (a) shows the input sequences of activations and weights. Fig. 6 (b) and (c) shows the required computation with and without the proposed zero-gating. From the figure, it can be clearly observed that there is only one MAC operation in the proposed zero-gating PE while four and two MAC operations are required in the baseline PEs without zero-gating and with zero-gating only in activations, respectively. The proposed zero-gating PE design works in full power only when neither activation nor weight is zero, which is promising for low power deep CNNs with no loss of precision.).
Raha, Dally, and Ye are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Raha and Dally, it would have been obvious for a person of ordinary skill in the art to apply the teachings of Ye to Raha before the effective filing date of the claimed invention in order to eliminate redundant computation for power reduction (cf. Ye, [Abstract, pg. 317] Convolution neural networks (CNNs) have shown great success in many areas such as object detection and pattern recognition. However, the high computational complexity of state-of-the-art deep CNNs makes them extreme difficult to be run on resource-constrained mobile and wearable devices. To address this design challenge, in this paper we first analyzed the filters' weights of pre-trained models from four state-of-the-art CNNs. We found that in all the CNNs that we analyzed, from about 20% (AlexNet) to 43% (VGG-19) of the weights are zeros, which lead to redundant large amounts of computation. Then, based on this observation, a zero-gating processing element (PE) design was proposed for low-power deep CNNs, in which the vast number of zeros in both activation maps and filter weights are explored to eliminate redundant computation for power reduction. We implemented our proposal with VGG-16 using ImageNet dataset. Experiments were conducted for evaluations of area and total power consumption. Compared with the baseline PE design without zero-gating, overall the proposed zero-gating PE can achieve 37% power saving while the corresponding area overhead is less than 8%.).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MAGGIE MAIDO whose telephone number is (703) 756-1953. The examiner can normally be reached M-Th: 6am - 4pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael Huntley can be reached on (303) 297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MM/Examiner, Art Unit 2129
/MICHAEL J HUNTLEY/Supervisory Patent Examiner, Art Unit 2129
Prosecution Insights