DETAILED ACTION
This action is in response to the Applicant Response filed 06 July 2023 for application 18/135,958 filed 18 April 2023.
Claim(s) 24-43 is/are new.
Claim(s) 1-23 is/are cancelled.
Claim(s) 24-43 is/are pending.
Claim(s) 24-43 is/are rejected.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim(s) 31-43 is/are objected to because of the following informalities:
Claim 31, line 5, “and” should be deleted
Claim 38, line 6, “and” should be deleted
Claim 41, line 2, further comprise, further comprising: should read “further comprise:”
Claim 42, line 2, further comprise, further comprising: should read “further comprise:”
Claims 32-37, 39-43 are objected to due to their dependence, either directly or indirectly, on claims 31, 37, 41-42
Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
mixed precision unit configured to decompose (claim 24) … decompose (claim 24) … decompose (claim 28) …
multiply-accumulation unit configured to perform (claim 24) … generate (claim 27) …
sparsity unit configured to skip (claim 24) …
compression unit configured to compress (claim 25) … compress (claim 26) … compress (claim 29) …
addition unit configured to compute (claim 30) …
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112(b)
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 24-30 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claims 24-30, various claim limitations reciting mixed precision unit configured to decompose (claim 24) … decompose (claim 24) … decompose (claim 28) …; multiply-accumulation unit configured to perform (claim 24) … generate (claim 27) …; sparsity unit configured to skip (claim 24) …; compression unit configured to compress (claim 25) … compress (claim 26) … compress (claim 29) …; addition unit configured to compute (claim 30) … invoke 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. However, the written description fails to disclose the corresponding structure, material, or acts for performing the entire claimed functions and to clearly link the structure, material, or acts to the functions. The specification is devoid of adequate structure to perform the claimed functions. There is no clear disclosure of the particular structure, either explicitly or inherently, to perform the functions of the claims. As would be recognized by those of ordinary skill in the art, the functions can be performed in any number of ways including in hardware, in software, or a combination of the two. The specification does not provide sufficient details such that one of ordinary skill in the art would understand which structure or structures perform(s) the claimed functions.
Therefore, the claim is indefinite and is rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph.
Applicant may:
(a) Amend the claim so that the claim limitation will no longer be interpreted as a limitation under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph;
(b) Amend the written description of the specification such that it expressly recites what structure, material, or acts perform the entire claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(c) Amend the written description of the specification such that it clearly links the structure, material, or acts disclosed therein to the function recited in the claim, without introducing any new matter (35 U.S.C. 132(a)).
If applicant is of the opinion that the written description of the specification already implicitly or inherently discloses the corresponding structure, material, or acts and clearly links them to the function so that one of ordinary skill in the art would recognize what structure, material, or acts perform the claimed function, applicant should clarify the record by either:
(a) Amending the written description of the specification such that it expressly recites the corresponding structure, material, or acts for performing the claimed function and clearly links or associates the structure, material, or acts to the claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(b) Stating on the record what the corresponding structure, material, or acts, which are implicitly or inherently set forth in the written description of the specification, perform the claimed function. For more information, see 37 CFR 1.75(d) and MPEP §§ 608.01(o) and 2181.
Claims 25-30 are rejected under 35 U.S.C. 112(b) due to their dependence, either directly or indirectly, on claims 24-30.
Claim Rejections - 35 USC § 112(a)
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 24-30 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement.
Claims 24-30 contain subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for pre AIA the inventor(s), at the time the application was filed, had possession of the claimed invention. As discussed above, the disclosure does not provide adequate structure to perform the claimed functions of:
mixed precision unit configured to decompose (claim 24) … decompose (claim 24) … decompose (claim 28) …
multiply-accumulation unit configured to perform (claim 24) … generate (claim 27) …
sparsity unit configured to skip (claim 24) …
compression unit configured to compress (claim 25) … compress (claim 26) … compress (claim 29) …
addition unit configured to compute (claim 30) …
The specification does not demonstrate that applicant has made an invention that achieves the claimed function because the invention is not described with sufficient detail such that one of ordinary skill in the art can reasonably conclude that the inventor had possession of the claimed invention.
Claims 25-30 are rejected under 35 U.S.C. 112(a) due to their dependence, either directly or indirectly, on claims 24-30.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim(s) 24-43 is/are rejected under 35 U.S.C. 101, because the claim(s) is/are directed to an abstract idea, and because the claim elements, whether considered individually or in combination, do not amount to significantly more than the abstract idea, see Alice Corporation Pty. Ltd. V. CLS Bank International et al., 573 US 208 (2014).
Regarding claim 24, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 24 is directed to a(n) apparatus, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) apparatus for deep learning.
The limitation of decompose an element in an input feature map of a deep learning operation into two input elements, the element in the input feature map having a first precision, the two input elements having a second precision that is lower than the first precision, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of decompose a weight of the deep learning operation into two weight elements, the weight associated with the element in the input feature map, the weight having the first precision, the two weight elements having the second precision, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of perform a computation using the two input elements and the two weight elements, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses calculate a value.
The limitation of skip one or more computations of one or more zero-valued elements in the input feature map, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. If a claim limitation, under its broadest reasonable interpretation, covers performance of mathematical concepts, then it falls within the "Mathematical Concepts" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites additional element(s) – apparatus, mixed precision unit, multiply-accumulation unit, sparsity unit. The additional element(s) is/are recited at a high-level of generality (i.e., as generic computer components performing generic computer functions of executing instructions on the computers) such that it amounts to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b)).
The claim recites additional element(s) – deep learning. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
apparatus, mixed precision unit, multiply-accumulation unit, sparsity unit amount(s) to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b))
deep learning amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 25, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 25 is directed to a(n) apparatus, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) apparatus for deep learning.
The limitation of compress a plurality of elements in the input feature map, the plurality of elements comprising the element, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites additional element(s) – compression unit. The additional element(s) is/are recited at a high-level of generality (i.e., as generic computer components performing generic computer functions of executing instructions on the computers) such that it amounts to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
compression unit amount(s) to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 26, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 26 is directed to a(n) apparatus, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) apparatus for deep learning.
The limitation of compress a plurality of weights of the deep learning operation, the plurality of weights comprising the weight, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated
into a practical application. The claim does not recite any additional elements which integrate the
abstract idea into a practical application and, therefore, does not impose any meaningful limits on
practicing the abstract idea. Therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to
significantly more than the judicial exception. As discussed above with respect to the integration of the
abstract idea into a practical application, the claim does not recite any additional elements which
provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 27, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 27 is directed to a(n) apparatus, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) apparatus for deep learning.
The limitation of generate an output activation of the deep learning operation from the computation, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses calculating a value.
If a claim limitation, under its broadest reasonable interpretation, covers performance of mathematical concepts, then it falls within the "Mathematical Concepts" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated
into a practical application. The claim does not recite any additional elements which integrate the
abstract idea into a practical application and, therefore, does not impose any meaningful limits on
practicing the abstract idea. Therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to
significantly more than the judicial exception. As discussed above with respect to the integration of the
abstract idea into a practical application, the claim does not recite any additional elements which
provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 28, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 28 is directed to a(n) apparatus, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) apparatus for deep learning.
The limitation of decompose the output activation into two output elements having the second precision, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated
into a practical application. The claim does not recite any additional elements which integrate the
abstract idea into a practical application and, therefore, does not impose any meaningful limits on
practicing the abstract idea. Therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to
significantly more than the judicial exception. As discussed above with respect to the integration of the
abstract idea into a practical application, the claim does not recite any additional elements which
provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 29, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 29 is directed to a(n) apparatus, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) apparatus for deep learning.
The limitation of compress a plurality of output activations of the deep learning operation, the plurality of output operations comprising the output activation, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites additional element(s) – compression unit. The additional element(s) is/are recited at a high-level of generality (i.e., as generic computer components performing generic computer functions of executing instructions on the computers) such that it amounts to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
compression unit amount(s) to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 30, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 30 is directed to a(n) apparatus, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) apparatus for deep learning.
The limitation of compute the element in the input feature map based on an activation function, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses calculate a value.
If a claim limitation, under its broadest reasonable interpretation, covers performance of mathematical concepts, then it falls within the "Mathematical Concepts" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites additional element(s) – additional unit. The additional element(s) is/are recited at a high-level of generality (i.e., as generic computer components performing generic computer functions of executing instructions on the computers) such that it amounts to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
additional unit amount(s) to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 31, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 31 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method for deep learning.
The limitation of decomposing an element in an input feature map of a deep learning operation into two input elements, the element in the input feature map having a first precision, the two input elements having a second precision that is lower than the first precision, the input feature map further comprising one or more zero-valued elements, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of decomposing a weight of the deep learning operation into two weight elements, the weight associated with the element in the input feature map, the weight having the first precision, the two weight elements having the second precision, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of performing a computation using the two input elements and the two weight elements, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses calculating a value.
The limitation of skipping one or more computations of the one or more zero-valued elements, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. If a claim limitation, under its broadest reasonable interpretation, covers performance of mathematical concepts, then it falls within the "Mathematical Concepts" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites additional element(s) – deep learning. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
deep learning amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 32, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 32 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method for deep learning.
The limitation of compressing the input feature map, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites additional element(s) – memory. The additional element(s) is/are recited at a high-level of generality (i.e., as generic computer components performing generic computer functions of executing instructions on the computers) such that it amounts to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b)).
The claim recites storing a compressed version of the input feature map in a memory, which is simply storing data recited at a high level of generality. This is nothing more than insignificant extra-solution activity (MPEP 2106.05(g)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
memory amount(s) to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b))
storing data amount(s) to no more than insignificant extra-solution activity (MPEP 2106.05(g)), wherein the insignificant extra-solution activity is the well-understood routine and conventional activit(y/ies) of storing and retrieving information in memory (MPEP 2016.05(d))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 33, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 33 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method for deep learning.
The limitation of compressing a plurality of weights of the deep learning operation, the plurality of weights comprising the weight, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated
into a practical application. The claim does not recite any additional elements which integrate the
abstract idea into a practical application and, therefore, does not impose any meaningful limits on
practicing the abstract idea. Therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to
significantly more than the judicial exception. As discussed above with respect to the integration of the
abstract idea into a practical application, the claim does not recite any additional elements which
provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 34, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 34 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method for deep learning.
The limitation of generating an output activation of the deep learning operation from the computation, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses calculating a value.
If a claim limitation, under its broadest reasonable interpretation, covers performance of mathematical concepts, then it falls within the "Mathematical Concepts" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated
into a practical application. The claim does not recite any additional elements which integrate the
abstract idea into a practical application and, therefore, does not impose any meaningful limits on
practicing the abstract idea. Therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to
significantly more than the judicial exception. As discussed above with respect to the integration of the
abstract idea into a practical application, the claim does not recite any additional elements which
provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 35, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 35 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method for deep learning.
The limitation of decomposing the output activation into two output elements having the second precision, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated
into a practical application. The claim does not recite any additional elements which integrate the
abstract idea into a practical application and, therefore, does not impose any meaningful limits on
practicing the abstract idea. Therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to
significantly more than the judicial exception. As discussed above with respect to the integration of the
abstract idea into a practical application, the claim does not recite any additional elements which
provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 36, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 36 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method for deep learning.
The limitation of compressing a plurality of output activations of the deep learning operation, the plurality of output operations comprising the output activation, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated
into a practical application. The claim does not recite any additional elements which integrate the
abstract idea into a practical application and, therefore, does not impose any meaningful limits on
practicing the abstract idea. Therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to
significantly more than the judicial exception. As discussed above with respect to the integration of the
abstract idea into a practical application, the claim does not recite any additional elements which
provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 37, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 37 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method for deep learning.
The limitation of computing the element in the input feature map based on an activation function, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses calculating a value.
If a claim limitation, under its broadest reasonable interpretation, covers performance of mathematical concepts, then it falls within the "Mathematical Concepts" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated
into a practical application. The claim does not recite any additional elements which integrate the
abstract idea into a practical application and, therefore, does not impose any meaningful limits on
practicing the abstract idea. Therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to
significantly more than the judicial exception. As discussed above with respect to the integration of the
abstract idea into a practical application, the claim does not recite any additional elements which
provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 38, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 38 is directed to computer-readable media, which is directed to an article of manufacture, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) one or more non-transitory computer-readable media.
The limitation of decomposing an element in an input feature map of a deep learning operation into two input elements, the element in the input feature map having a first precision, the two input elements having a second precision that is lower than the first precision, the input feature map further comprising one or more zero-valued elements, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of decomposing a weight of the deep learning operation into two weight elements, the weight associated with the element in the input feature map, the weight having the first precision, the two weight elements having the second precision, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of performing a computation using the two input elements and the two weight elements, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses calculating a value.
The limitation of skipping one or more computations of the one or more zero-valued elements, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. If a claim limitation, under its broadest reasonable interpretation, covers performance of mathematical concepts, then it falls within the "Mathematical Concepts" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites additional element(s) – one or more ... computer-readable media, instructions. The additional element(s) is/are recited at a high-level of generality (i.e., as generic computer components performing generic computer functions of executing instructions on the computers) such that it amounts to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b)).
The claim recites additional element(s) – deep learning. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
one or more ... computer-readable media, instructions amount(s) to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b))
deep learning amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 39, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 39 is directed to computer-readable media, which is directed to an article of manufacture, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) one or more non-transitory computer-readable media.
The limitation of compressing the input feature map, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites additional element(s) – memory. The additional element(s) is/are recited at a high-level of generality (i.e., as generic computer components performing generic computer functions of executing instructions on the computers) such that it amounts to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b)).
The claim recites storing a compressed version of the input feature map in a memory, which is simply storing data recited at a high level of generality. This is nothing more than insignificant extra-solution activity (MPEP 2106.05(g)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
memory amount(s) to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b))
storing data amount(s) to no more than insignificant extra-solution activity (MPEP 2106.05(g)), wherein the insignificant extra-solution activity is the well-understood routine and conventional activit(y/ies) of storing and retrieving information in memory (MPEP 2016.05(d))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 40, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 40 is directed to computer-readable media, which is directed to an article of manufacture, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) one or more non-transitory computer-readable media.
The limitation of compressing a plurality of weights of the deep learning operation, the plurality of weights comprising the weight, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated
into a practical application. The claim does not recite any additional elements which integrate the
abstract idea into a practical application and, therefore, does not impose any meaningful limits on
practicing the abstract idea. Therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to
significantly more than the judicial exception. As discussed above with respect to the integration of the
abstract idea into a practical application, the claim does not recite any additional elements which
provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 41, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 41 is directed to computer-readable media, which is directed to an article of manufacture, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) one or more non-transitory computer-readable media.
The limitation of generating an output activation of the deep learning operation from the computation, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses calculating a value.
If a claim limitation, under its broadest reasonable interpretation, covers performance of mathematical concepts, then it falls within the "Mathematical Concepts" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated
into a practical application. The claim does not recite any additional elements which integrate the
abstract idea into a practical application and, therefore, does not impose any meaningful limits on
practicing the abstract idea. Therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to
significantly more than the judicial exception. As discussed above with respect to the integration of the
abstract idea into a practical application, the claim does not recite any additional elements which
provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 42, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 42 is directed to computer-readable media, which is directed to an article of manufacture, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) one or more non-transitory computer-readable media.
The limitation of decomposing the output activation into two output elements having the second precision, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated
into a practical application. The claim does not recite any additional elements which integrate the
abstract idea into a practical application and, therefore, does not impose any meaningful limits on
practicing the abstract idea. Therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to
significantly more than the judicial exception. As discussed above with respect to the integration of the
abstract idea into a practical application, the claim does not recite any additional elements which
provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 43, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 43 is directed to computer-readable media, which is directed to an article of manufacture, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) one or more non-transitory computer-readable media.
The limitation of computing the element in the input feature map based on an activation function, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses calculating a value.
If a claim limitation, under its broadest reasonable interpretation, covers performance of mathematical concepts, then it falls within the "Mathematical Concepts" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated
into a practical application. The claim does not recite any additional elements which integrate the
abstract idea into a practical application and, therefore, does not impose any meaningful limits on
practicing the abstract idea. Therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to
significantly more than the judicial exception. As discussed above with respect to the integration of the
abstract idea into a practical application, the claim does not recite any additional elements which
provide an inventive concept, and, therefore, the claim is not patent eligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 24, 27-28, 31, 34-35, 38, 41-42 is/are rejected under 35 U.S.C. 103 as being unpatentable over Peluso et al. (Energy-Accuracy Scalable Deep Convolutional Neural Networks: A Pareto Analysis, hereinafter referred to as “Peluso”).
Regarding claim 24 (New), Peluso teaches an apparatus for deep learning (Peluso, section 3.2 – teaches hardware implementation; Peluso, section 3.3 – teaches an apparatus including hardware and software implementing the DL strategy; see also Peluso, Fig. 2), comprising:
a mixed precision unit configured (Peluso, section 3.2 – teaches hardware implementation; see also Peluso, Fig. 2) to:
decompose an element in an input feature map of a deep learning operation into two input elements, the element in the input feature map having a first precision, the two input elements having a second precision that is lower than the first precision (Peluso, section 3.1 – teaches decomposing an input element if an input feature map of a convolutional neural network into two halfwords of most significant bits and least significant bits), and
decompose a weight of the deep learning operation into two weight elements, the weight associated with the element in the input feature map, the weight having the first precision, the two weight elements having the second precision (Peluso, section 3.1 – teaches decomposing a weight element of a kernel a convolutional neural network into two halfwords of most significant bits and least significant bits);
a multiply-accumulation unit (Peluso, section 3.2 – teaches hardware implementation; see also Peluso, Fig. 2) configured to perform a computation using the two input elements and the two weight elements (Peluso, section 3.1 – teaches performing computations using the halfwords for the input and weight; see also Peluso, Fig. 1); and
a sparsity unit (Peluso, section 3.2 – teaches hardware implementation; see also Peluso, Fig. 2) configured to skip one or more computations of one or more zero-valued elements in the input feature map by the multiply-accumulation unit (Peluso, section 3.2 – teaches a zero-skipping strategy the skips computation if one of the operands is zero).
Regarding claim 27 (New), Peluso teaches all of the limitations of the apparatus of claim 24 as noted above. Peluso further teaches wherein multiply-accumulation unit is further configured to:
generate an output activation of the deep learning operation from the computation (Peluso, sections 3.1-3.2 – teaches the MAC outputting the dot product of the input and weight elements; see also Peluso, Algorithm 1).
Regarding claim 28 (New), Peluso teaches all of the limitations of the apparatus of claim 27 as noted above. Peluso further teaches wherein the mixed precision unit is further configured to:
decompose the output activation into two output elements having the second precision (Peluso, section 3.1 – teaches decomposing an input element if an input feature map of a convolutional neural network into two halfwords of most significant bits and least significant bits; Peluso, section 4.2 – teaches decomposing inputs for each layer [decomposing inputs for a subsequent layer would require decomposing the output of the preceding layer]).
Regarding claim 31 (New), Peluso teaches a method for deep learning, comprising:
decomposing an element in an input feature map of a deep learning operation into two input elements, the element in the input feature map having a first precision, the two input elements having a second precision that is lower than the first precision (Peluso, section 3.1 – teaches decomposing an input element if an input feature map of a convolutional neural network into two halfwords of most significant bits and least significant bits), the input feature map further comprising one or more zero-valued elements (Peluso, section 3.2 – teaches a zero-skipping strategy the skips computation if one of the operands is zero); and
decomposing a weight of the deep learning operation into two weight elements, the weight associated with the element in the input feature map, the weight having the first precision, the two weight elements having the second precision (Peluso, section 3.1 – teaches decomposing a weight element of a kernel a convolutional neural network into two halfwords of most significant bits and least significant bits);
performing a computation using the two input elements and the two weight elements (Peluso, section 3.1 – teaches performing computations using the halfwords for the input and weight; see also Peluso, Fig. 1); and
skipping one or more computations of the one or more zero-valued elements (Peluso, section 3.2 – teaches a zero-skipping strategy the skips computation if one of the operands is zero).
Regarding claim 34, the rejection of claim 31 is incorporated herein. Further, the limitations in this claim are taught by Peluso for the reasons set forth in the rejection of claim 27.
Regarding claim 35, the rejection of claim 34 is incorporated herein. Further, the limitations in this claim are taught by Peluso for the reasons set forth in the rejection of claim 28.
Regarding claim 38, it is the computer-readable media embodiment of claim 31 with similar limitations to claim 31 and is rejected using the same reasoning found in claim 31. Peluso further teaches one or more non-transitory computer-readable media storing instructions executable to perform operations (Peluso, section 3.2 – teaches hardware implementation; Peluso, section 3.3 – teaches an apparatus including hardware and software implementing the DL strategy; see also Peluso, Fig. 2) …
Regarding claim 41, the rejection of claim 38 is incorporated herein. Further, the limitations in this claim are taught by Peluso for the reasons set forth in the rejection of claim 34.
Regarding claim 42, the rejection of claim 41 is incorporated herein. Further, the limitations in this claim are taught by Peluso for the reasons set forth in the rejection of claim 35.
Claim(s) 25-26, 29-30, 32-33, 36-37, 39-40, 43 is/are rejected under 35 U.S.C. 103 as being unpatentable over Peluso in view of Kim et al. (Deep Convolutional Neural Network Accelerator Featuring Conditional Computing and Low External Memory Access, hereinafter referred to as "Kim").
Regarding claim 25 (New), Peluso teaches all of the limitations of the apparatus of claim 24 as noted above. However, Peluso does not explicitly teach a compression unit configured to compress a plurality of elements in the input feature map, the plurality of elements comprising the element.
Kim teaches a compression unit configured to compress a plurality of elements in the input feature map, the plurality of elements comprising the element (Kim, section III – teaches loading non-zero input element values into the processing element [Not loading zero value inputs is a compression of the input]).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to modify Peluso with the teachings of Kim in order to reduce redundant operations and increase sparsity in the field of accelerating neural network computations (Kim, Abstract – “This paper presents an ASIC accelerator for deep convolutional neural networks (DCNNs) featuring a novel conditional computing scheme that synergistically combines precision-cascading with zero-skipping. To reduce many redundant convolution operations that are followed by max-pooling operations, we propose precision-cascading, where the input features are divided into a number of low-precision groups and approximate convolutions with only the most significant bits (MSBs) are performed first. Based on this approximate computation, the full-precision convolution is performed only on the maximum pooling output that is found... Precision-cascading provides the added benefit of increased sparsity per low-precision group, which we exploit with zero-skipping to eliminate clock cycles as well as external memory access that involve zero inputs...”).
Regarding claim 26 (New), Peluso in view of Kim teaches all of the limitations of the apparatus of claim 25 as noted above.
Kim further teaches wherein the compression unit configured to compress a plurality of weights of the deep learning operation, the plurality of weights comprising the weight (Kim, section III – teaches skipping kernel features of [weights] of zero-value inputs [Not loading weights for zero value inputs is a compression of the weight]).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to combine the teachings of Peluso and Kim in order to compress weights to reduce redundant operations and increase sparsity (Kim, Abstract).
Regarding claim 29 (New), Peluso teaches all of the limitations of the apparatus of claim 27 as noted above. However, Peluso does not explicitly teach a compression unit configured to compress a plurality of output activations of the deep learning operation, the plurality of output operations comprising the output activation.
Kim teaches a compression unit configured to compress a plurality of output activations of the deep learning operation, the plurality of output operations comprising the output activation (Kim, section II.B – teaches ReLU activation functions which compress negative output activations to zero; see also Kim, section III).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to modify Peluso with the teachings of Kim in order to reduce redundant operations and increase sparsity in the field of accelerating neural network computations (Kim, Abstract – “This paper presents an ASIC accelerator for deep convolutional neural networks (DCNNs) featuring a novel conditional computing scheme that synergistically combines precision-cascading with zero-skipping. To reduce many redundant convolution operations that are followed by max-pooling operations, we propose precision-cascading, where the input features are divided into a number of low-precision groups and approximate convolutions with only the most significant bits (MSBs) are performed first. Based on this approximate computation, the full-precision convolution is performed only on the maximum pooling output that is found... Precision-cascading provides the added benefit of increased sparsity per low-precision group, which we exploit with zero-skipping to eliminate clock cycles as well as external memory access that involve zero inputs...”).
Regarding claim 30 (New), Peluso teaches all of the limitations of the apparatus of claim 24 as noted above. However, Peluso does not explicitly teach an additional unit configured to compute the element in the input feature map based on an activation function.
Kim teaches an additional unit configured to compute the element in the input feature map based on an activation function (Kim, section II.B – teaches ReLU activation functions to generate input feature map elements for the next layer; see also Kim, section III).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to modify Peluso with the teachings of Kim in order to reduce redundant operations and increase sparsity in the field of accelerating neural network computations (Kim, Abstract – “This paper presents an ASIC accelerator for deep convolutional neural networks (DCNNs) featuring a novel conditional computing scheme that synergistically combines precision-cascading with zero-skipping. To reduce many redundant convolution operations that are followed by max-pooling operations, we propose precision-cascading, where the input features are divided into a number of low-precision groups and approximate convolutions with only the most significant bits (MSBs) are performed first. Based on this approximate computation, the full-precision convolution is performed only on the maximum pooling output that is found... Precision-cascading provides the added benefit of increased sparsity per low-precision group, which we exploit with zero-skipping to eliminate clock cycles as well as external memory access that involve zero inputs...”).
Regarding claim 32 (New), Peluso teaches all of the limitations of the method of claim 31 as noted above. However, Peluso does not explicitly teach compressing the input feature map; and storing a compressed version of the input feature map in a memory.
Kim teaches
compressing the input feature map (Kim, section III – teaches loading non-zero input element values into the processing element [Not loading zero value inputs is a compression of the input]); and
storing a compressed version of the input feature map in a memory (Kim, section III teaches loading non-zero input values [compressed input] into the processing element array).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to modify Peluso with the teachings of Kim in order to reduce redundant operations and increase sparsity in the field of accelerating neural network computations (Kim, Abstract – “This paper presents an ASIC accelerator for deep convolutional neural networks (DCNNs) featuring a novel conditional computing scheme that synergistically combines precision-cascading with zero-skipping. To reduce many redundant convolution operations that are followed by max-pooling operations, we propose precision-cascading, where the input features are divided into a number of low-precision groups and approximate convolutions with only the most significant bits (MSBs) are performed first. Based on this approximate computation, the full-precision convolution is performed only on the maximum pooling output that is found... Precision-cascading provides the added benefit of increased sparsity per low-precision group, which we exploit with zero-skipping to eliminate clock cycles as well as external memory access that involve zero inputs...”).
Regarding claim 33, the rejection of claim 31 is incorporated herein. Further, the limitations in this claim are taught by Peluso in view of Kim for the reasons set forth in the rejection of claim 26.
Regarding claim 36, the rejection of claim 34 is incorporated herein. Further, the limitations in this claim are taught by Peluso in view of Kim for the reasons set forth in the rejection of claim 29.
Regarding claim 37, the rejection of claim 31 is incorporated herein. Further, the limitations in this claim are taught by Peluso in view of Kim for the reasons set forth in the rejection of claim 30.
Regarding claim 39, the rejection of claim 38 is incorporated herein. Further, the limitations in this claim are taught by Peluso in view of Kim for the reasons set forth in the rejection of claim 32.
Regarding claim 40, the rejection of claim 38 is incorporated herein. Further, the limitations in this claim are taught by Peluso in view of Kim for the reasons set forth in the rejection of claim 33.
Regarding claim 43, the rejection of claim 38 is incorporated herein. Further, the limitations in this claim are taught by Peluso in view of Kim for the reasons set forth in the rejection of claim 37.
Conclusion
Any inquiry concerning this communication or earlier communication from the examiner should be directed to MARSHALL WERNER whose telephone number is (469) 295-9143. The examiner can normally be reached on Monday – Thursday 7:30 AM – 4:30 PM ET.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kamran Afshar, can be reached at (571) 272-7796. The fax number for the organization where this application or proceeding is assigned is (571) 273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MARSHALL L WERNER/ Primary Examiner, Art Unit 2125