DETAILED ACTION
This action is in response to the Applicant Response filed 18 February 2026 for application 18/044,661 filed 09 March 2023.
Claim(s) 1, 3-11, 13-16 is/are currently amended.
Claim(s) 2, 12 is/are cancelled.
Claim(s) 1, 3-11, 13-16 is/are pending.
Claim(s) 1, 3-11, 13-16 is/are rejected.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement filed 09 March 2023 fails to comply with 37 CFR 1.98(a)(2), which requires a legible copy of each cited foreign patent document; each non-patent literature publication or that portion which caused it to be listed; and all other information or that portion which caused it to be listed. It has been placed in the application file, but the information referred to therein has not been considered.
Response to Arguments
Applicant's arguments regarding the objections to the specification have been fully considered and, in light of the amendments to the specification, are persuasive.
Applicant's arguments regarding the 35 U.S.C. 112(b) rejection(s) of claim(s) 14 have been fully considered and, in light of the amendments to the claims, are persuasive. The 35 U.S.C. 112(b) rejection(s) of claim(s) 14 has/have been withdrawn.
Applicant’s arguments regarding the 35 U.S.C. 101 rejection of claims 1, 3-11, 13-16 have been fully considered but are not persuasive.
Applicant first argues that the claims recite features specific to quantization of a neural network and therefore do not recite an abstract idea. Examiner respectfully disagrees. While applicant asserts that quantization is not an abstract idea, applicant provides no evidence to support this assertion. As noted below, quantization is an abstract idea.
Applicant next argues that quantization reduces computational and/or processing costs while maintaining accuracy. As noted in the MPEP, the judicial exception alone cannot provide the improvement. MPEP 2106.05(a). The MPEP further states that an improvement in the abstract idea itself is not an improvement in technology. MPEP 2106.05(a). As noted below, step steps of filter selection and/or elimination, as well as quantization, are abstract ideas, and, therefore, cannot provide the improvement. Moreover, the MPEP state that claiming improved speed or efficiency inherent with applying the abstract idea on a computer does not integrate a judicial exception into a practical application or provide an inventive concept. Therefore, improved computational/processing costs do not create an improvement.
Lastly, applicant makes similar arguments with respect to step 2B, and Examiner respectfully disagrees for the reasons noted above.
Therefore the 35 U.S.C. 101 rejection of claims 1, 3-11, 13-16 is maintained.
Applicant’s arguments regarding the e5 U.S.C. 103 rejections of claims 1, 3-11, 13-16 have been fully considered but are not persuasive.
Applicant argues that the cited references do not teach the recited limitations. Specifically, applicant argues that the cited references do not teach selecting filters in the BN layer, but instead teach selecting filters in the depth-wise convolutional layer. Examiner respectfully disagrees. Mao states that the filters are sorted by variance after normalization. This means that the filters are selected in the batch normalization layer based on variance. Therefore, the cited references do, in fact, teach selecting filters in the BN layer.
Therefore, claims 1, 3-11, 13-16 stand rejected under 35 U.S.C. 103.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim(s) 1, 3-11, 13-16 is/are rejected under 35 U.S.C. 101, because the claim(s) is/are directed to an abstract idea, and because the claim elements, whether considered individually or in combination, do not amount to significantly more than the abstract idea, see Alice Corporation Pty. Ltd. V. CLS Bank International et al., 573 US 208 (2014).
Regarding claim 1, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 1 is directed to a(n) apparatus, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) information processing apparatus.
The limitation of select a filter in a neural network, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of determine that the selected filter has a variance smaller than a threshold, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of eliminate the selected filter in the Batch normalization layer based on the determination that the selected filter has the variance smaller than the threshold, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of quantize the neural network based on the elimination of the selected filter, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites additional element(s) – information processing apparatus, central processing unit. The additional element(s) is/are recited at a high-level of generality (i.e., as generic computer components performing generic computer functions of executing instructions on the computers) such that it amounts to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b)).
The claim recites additional element(s) – neural network, Depthwise convolution layer, Pointwise convolution layer, Batch normalization layer. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
The claim recites wherein the neural network includes a Depthwise convolution layer, a Pointwise convolution layer, and a Batch normalization layer, and the selection of the filter is in the Batch normalization layer subsequent to the Depthwise convolution layer which is simply additional information regarding the model, and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
information processing apparatus, central processing unit amount(s) to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b))
neural network, Depthwise convolution layer, Pointwise convolution layer, Batch normalization layer amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
additional information regarding the model do(es) not apply the exception in a meaningful way (MPEP 2106.05(e))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 3, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 3 is directed to a(n) apparatus, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) information processing apparatus.
The limitation of replace an output value of the Batch normalization layer with an approximate expression in the elimination of the selected filter, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated
into a practical application. The claim does not recite any additional elements which integrate the
abstract idea into a practical application and, therefore, does not impose any meaningful limits on
practicing the abstract idea. Therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to
significantly more than the judicial exception. As discussed above with respect to the integration of the
abstract idea into a practical application, the claim does not recite any additional elements which
provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 4, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 4 is directed to a(n) apparatus, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) information processing apparatus.
The limitation of adjust a parameter in a convolution layer of the neural network, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites additional element(s) – convolution layer. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
The claim recites in the neural network, the convolution layer is subsequent to the Batch normalization layer which is simply additional information regarding the model, and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
convolution layer amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
additional information regarding the model do(es) not apply the exception in a meaningful way (MPEP 2106.05(e))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 5, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 5 is directed to a(n) apparatus, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) information processing apparatus.
The limitation of adjust a bias parameter of the convolution layer subsequent to the Batch normalization layer, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated
into a practical application. The claim does not recite any additional elements which integrate the
abstract idea into a practical application and, therefore, does not impose any meaningful limits on
practicing the abstract idea. Therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to
significantly more than the judicial exception. As discussed above with respect to the integration of the
abstract idea into a practical application, the claim does not recite any additional elements which
provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 6, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 6 is directed to a(n) apparatus, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) information processing apparatus.
The limitation of incorporate a parameter of the Batch normalization layer into a convolution layer of the neural network, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites additional element(s) – convolution layer. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
convolution layer amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 7, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 7 is directed to a(n) apparatus, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) information processing apparatus. The Step 2A Prong One Analysis for claim 6 is applicable here since claim 7 carries out the apparatus of claim 6 but for the recitation of additional element(s) of wherein the convolution layer corresponds to the Depthwise convolution layer that precedes the Batch normalization layer.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the model and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the model do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 8, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 8 is directed to a(n) apparatus, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) information processing apparatus. The Step 2A Prong One Analysis for claim 1 is applicable here since claim 8 carries out the apparatus of claim 1 but for the recitation of additional element(s) of wherein the threshold corresponds to a variable in a denominator of an expression, the expression represents a filter process of the Batch normalization layer, and the variable avoids division by zero in the filter process of the Batch normalization layer.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the model and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the model do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 9, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 9 is directed to a(n) method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method.
The limitation of selecting a filter in a neural network, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of determining that the selected filter has a variance smaller than a threshold, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of eliminating the selected filter in the Batch normalization layer based on the determination that the selected filter has the variance smaller than the threshold, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of quantizing the neural network based on the elimination of the selected filter, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites additional element(s) – computer device. The additional element(s) is/are recited at a high-level of generality (i.e., as generic computer components performing generic computer functions of executing instructions on the computers) such that it amounts to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b)).
The claim recites additional element(s) – neural network, Depthwise convolution layer, Pointwise convolution layer, Batch normalization layer. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
The claim recites wherein the neural network includes a Depthwise convolutional layer, a Pointwise convolutional layer, and a Batch normalization layer, and the selection of the filter is in the Batch normalization layer subsequent to the Depthwise convolution layer which is simply additional information regarding the model, and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
computer device amount(s) to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b))
neural network, Depthwise convolution layer, Pointwise convolution layer, Batch normalization layer amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
additional information regarding the model do(es) not apply the exception in a meaningful way (MPEP 2106.05(e))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 10, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 10 is directed to a(n) apparatus, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) information processing apparatus.
The limitation of select a first filter in a neural network, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of determine that the selected first filter has a variance smaller than a threshold, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of eliminate the selected first filter in the Batch normalization layer based on the determination that the selected first filter has the variance smaller than the threshold, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of quantize the neural network based on the elimination of the selected first filter, wherein the quantization of the neural network includes: quantization of activation data associated with the neural network, and quantization of weight data associated with the neural network subsequent to the quantization of the activation data, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites additional element(s) – information processing apparatus, central processing unit. The additional element(s) is/are recited at a high-level of generality (i.e., as generic computer components performing generic computer functions of executing instructions on the computers) such that it amounts to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b)).
The claim recites additional element(s) – neural network, Depthwise convolution layer, Pointwise convolution layer, Batch normalization layer. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
The claim recites wherein the neural network includes a Depthwise convolution layer, a Pointwise convolution layer, and a Batch normalization layer, and the selection of the first filter is in the Batch normalization layer subsequent to the Depthwise convolution layer which is simply additional information regarding the model, and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
information processing apparatus, central processing unit amount(s) to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b))
neural network, Depthwise convolution layer, Pointwise convolution layer, Batch normalization layer amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
additional information regarding the model do(es) not apply the exception in a meaningful way (MPEP 2106.05(e))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 11, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 11 is directed to a(n) apparatus, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) information processing apparatus. The Step 2A Prong One Analysis for claim 10 is applicable here since claim 11 carries out the apparatus of claim 10 but for the recitation of additional element(s) of execute a first relearning process after the quantization of the activation data; and execute a second relearning process after the quantization of the weight data.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites execute a first relearning process after the quantization of the activation data; and execute a second relearning process after the quantization of the weight data which is simply generic training to perform the abstract idea of model generation and amounts to mere instructions to apply the exception (MPEP 2106.05(f)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
generic training to perform the abstract idea amount(s) to no more than mere instructions to apply the exception (MPEP 2106.05(f))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 13, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 13 is directed to a(n) apparatus, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) information processing apparatus. The Step 2A Prong One Analysis for claim 10 is applicable here since claim 13 carries out the apparatus of claim 10 but for the recitation of additional element(s) of wherein each of the quantization of the activation data and the quantization of the weight data is after the elimination of the selected first filter.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the quantization and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the quantization do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 14, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 14 is directed to a(n) apparatus, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) information processing apparatus.
The limitation of eliminate a second filter after the quantization of the activation data, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites the elimination of the selected first filter is before the quantization of the activation data which is simply additional information regarding the model, and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
additional information regarding the model do(es) not apply the exception in a meaningful way (MPEP 2106.05(e))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 15, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 15 is directed to a(n) apparatus, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) information processing apparatus. The Step 2A Prong One Analysis for claim 13 is applicable here since claim 15 carries out the apparatus of claim 13 but for the recitation of additional element(s) of wherein the selected first filter is eliminated only once prior to each of the quantization of the activation data and the quantization of the weight data.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the filter and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the filter do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 16, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 16 is directed to a(n) method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method.
The limitation of selecting a filter in a neural network, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of determining that the selected filter has a variance smaller than a threshold, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of eliminating the selected filter in the Batch normalization layer based on the determination that the selected filter has the variance smaller than the threshold, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of quantizing activation data associated with the neural network based on the elimination of the selected filter, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of quantizing weight data associated with the neural network subsequent to the quantization of the activation data, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites additional element(s) – computer device. The additional element(s) is/are recited at a high-level of generality (i.e., as generic computer components performing generic computer functions of executing instructions on the computers) such that it amounts to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b)).
The claim recites additional element(s) – neural network, Depthwise convolution layer, Pointwise convolution layer, Batch normalization layer. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
The claim recites wherein the neural network includes a Depthwise convolution layer, a Pointwise convolution layer, and a Batch normalization layer, and the selection of the filter is in the Batch normalization layer subsequent to the Depthwise convolution layer which is simply additional information regarding the model, and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
computer device amount(s) to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b))
neural network, Depthwise convolution layer, Pointwise convolution layer, Batch normalization layer amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
additional information regarding the model do(es) not apply the exception in a meaningful way (MPEP 2106.05(e))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 3-4, 6-7, 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gadosey et al. (On Pruned, Quantized and Compact CNN Architectures for Vision Applications, hereinafter referred to as “Gadosey”) in view of Mao et al. (Efficient Convolution Neural Networks for Object Tracking Using Separable Convolution and Filter Pruning, hereinafter referred to as “Mao”).
Regarding claim 1 (Currently Amended), Gadosey teaches an information processing apparatus, comprising:
a central processing unit (CPU) (Gadosey, section 4 – teaches running python packages on GPUs and CPUs) configured to:
select a filter in a neural network (Gadosey, section 3.1.1 – teaches pruning filters [requiring selection]), wherein
the neural network includes a Depthwise convolution layer, a Pointwise convolution layer, and a Batch normalization layer (Gadosey, section 2.3 – teaches MobileNet architecture based on depthwise separable convolutions comprising a depthwise convolutional layer followed by a pointwise convolutional layer with a batch normalization layer and a ReLU activation between each convolutional layer), and
quantize the neural network based on the elimination of the selected filter (Gadosey, section 3.1.2 – teaches quantizing the neural network after pruning filters; see also Gadosey, Algorithm 1).
While Gadosey teaches pruning, quantization and depthwise separable convolutions, Gadosey does not explicitly teach a filter elimination processing section that eliminates a filter whose variance is smaller than a threshold in a neural network having a Depthwise convolution layer, a Pointwise convolution layer, and a Batch normalization layer.
Mao teaches
select a filter in a neural network (Mao, section III.C. – teaches removing filters [requiring selection]), wherein
the neural network includes a Depthwise convolution layer, a Pointwise convolution layer, and a Batch normalization layer (Mao, section. III.B. – teaches a separable convolutional structure with depthwise convolutional layers, pointwise convolutional layers and batch normalization layers), and
the selection of the filter is in the Batch normalization layer subsequent to the Depthwise convolution layer (Mao, section III.C. – teaches pruning the filter in depthwise convolutional layer based on the variance of the normalized filter weights [filter set in BN layer]);
determine that the selected filter has a variance smaller than a threshold (Mao, section III.C. – teaches removing filters based on a small [threshold] variance);
eliminate the selected filter in the Batch normalization layer based on the determination that the selected filter has the variance smaller than the threshold (Mao, section III.C. – teaches pruning the filter in depthwise convolutional layer based on a small variance of the normalized filter weights [filter set in BN layer]).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to modify Gadosey with the teachings of Mao in order to improve efficiency and reduce calculations in the field of model compression (Mao, Abstract – “Object tracking based on deep learning is a hot topic in computer vision with many applications. Due to high computation and memory costs, it is difficult to deploy convolutional neural networks (CNNs) for object tracking on embedded systems with limited hardware resources. This paper uses the Siamese network to construct the backbone of our tracker. The convolution layers used to extract features often have the highest costs, so more improvements should be focused on them to make the tracking more efficient. In this paper, the standard convolution is optimized by the separable convolution, which mainly includes a depthwise convolution and a pointwise convolution. To further reduce the calculation, filters in the depthwise convolution layer are pruned with filters variance. As there are different weight distributions in convolution layers, the filter pruning is guided by a hyper-parameter designed. With the improvements, the number of parameters is decreased to 13% of the original network and the computation is reduced to 23%. On the NVIDIA Jetson TX2, the tracking speed increased to 3.65 times on the CPU and 2.08 times on the GPU, without significant degradation of tracking performance in VOT benchmark.”).
Regarding claim 3 (Currently Amended), Gadosey in view of Mao teaches all of the limitations of the apparatus of claim 1 as noted below. Mao further teaches wherein the CPU is further configured to replace an output value of the Batch normalization layer with an approximate expression in the elimination of the selected filter (Mao, section III.C. – teaches reducing the number of channels in the subsequent pointwise layer due to the filter pruning [This modifies the BN output to an approximation]).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to combine the teachings of Gadosey and Mao in order to eliminate filters to improve efficiency and reduce calculations (Mao, Abstract).
Regarding claim 4 (Currently Amended), Gadosey in view of Mao teaches all of the limitations of the apparatus of claim 1 as noted below. Mao further teaches wherein
the CPU is further configured to adjust a parameter in a convolution layer of the neural network (Mao, section III.C. – teaches reducing the number of channels in the subsequent pointwise layer due to the filter pruning), and
in the neural network, the convolution layer is subsequent to the Batch normalization layer (Mao, section III.C. – teaches reducing the number of channels in the subsequent pointwise layer due to the filter pruning).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to combine the teachings of Gadosey and Mao in order to eliminate filters to improve efficiency and reduce calculations (Mao, Abstract).
Regarding claim 6 (Currently Amended), Gadosey in view of Mao teaches all of the limitations of the apparatus of claim 1 as noted below. Mao further teaches wherein the CPU is further configured to incorporate a parameter of the Batch normalization layer into a convolution layer of the neural network (Mao, section III.C. - teaches determining the variance and a pruning ratio after normalization to determine which filters to pruning and pruning those filters, based on the ratio, from the depthwise layer).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to combine the teachings of Gadosey and Mao in order to eliminate filters to improve efficiency and reduce calculations (Mao, Abstract).
Regarding claim 7 (Currently Amended), Gadosey in view of Mao teaches all of the limitations of the apparatus of claim 6 as noted below. Mao further teaches wherein the convolution layer corresponds to the Depthwise convolution layer that precedes the Batch normalization layer (Mao, section III.C. - teaches determining the variance and a pruning ratio after normalization to determine which filters to pruning and pruning those filters, based on the ratio, from the depthwise layer).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to combine the teachings of Gadosey and Mao in order to eliminate filters to improve efficiency and reduce calculations (Mao, Abstract).
Regarding claim 9, it is the method embodiment of claim 1 with similar limitations to claim 1 and is rejected using the same reasoning found in claim 1.
Claim(s) 5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gadosey in view of Mao and further in view of Luo et al. (ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression, hereinafter referred to as “Luo”).
Regarding claim 5 (Currently Amended), Gadosey in view of Mao teaches all of the limitations of the apparatus of claim 4 as noted below. However, Gadosey in view of Mao does not explicitly teach wherein the CPU is further configured to adjust a bias parameter of the convolution layer subsequent to the Batch normalization layer.
Luo teaches wherein the CPU is further configured to adjust a bias parameter of the convolution layer subsequent to the Batch normalization layer (Luo, section 3.2.1 – teaches adjusting a bias parameter for the subsequent layer after the filter pruning).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to modify Gadosey in view of Mao with the teachings of Luo in order to provide an effective strategy to compress and accelerate off the shelf models in the field of model compression (Luo, Abstract – “We propose an efficient and unified framework, namely ThiNet, to simultaneously accelerate and compress CNN models in both training and inference stages. We focus on the filter level pruning, i.e., the whole filter would be discarded if it is less important. Our method does not change the original network structure, thus it can be perfectly supported by any off-the-shelf deep learning libraries. We formally establish filter pruning as an optimization problem, and reveal that we need to prune filters based on statistics information computed from its next layer, not the current layer, which differentiates ThiNet from existing methods. Experimental results demonstrate the effectiveness of this strategy, which has advanced the state-of-the-art. We also show the performance of ThiNet on ILSVRC-12 benchmark. ThiNet achieves 3.31x FLOPs reduction and 16.63x compression on VGG-16, with only 0.52% top-5 accuracy drop. Similar experiments with ResNet-50 reveal that even for a compact network, ThiNet can also reduce more than half of the parameters and FLOPs, at the cost of roughly 1% top-5 accuracy drop. Moreover, the original VGG-16 model can be further pruned into a very small model with only 5.05MB model size, preserving AlexNet level accuracy but showing much stronger generalization ability.”).
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gadosey in view of Mao and further in view of Sheng et al. (A Quantization-Friendly Separable Convolution for MobileNets, hereinafter referred to as “Sheng”).
Regarding claim 8 (Currently Amended), Gadosey in view of Mao teaches all of the limitations of the apparatus of claim 1 as noted below. However, Gadosey in view of Mao does not explicitly teach wherein the threshold corresponds to a variable in a denominator of an expression, the expression represents a filter process of the Batch normalization layer, and the variable avoids division by zero in the filter process of the Batch normalization layer.
Sheng teaches wherein
the threshold corresponds to a variable in a denominator of an expression (Sheng, section 2.3.1 - teaches properly handling normalization values of zero variance channels [or filters] to better preserve quantization bits [Because every variance below a threshold is made zero, it would be obvious to make the BN denominator constant equal to the threshold to preserve quantization]),
the expression represents a filter process of the Batch normalization layer (Sheng, section 2.3.1 - teaches properly handling normalization values of zero variance channels [or filters] to better preserve quantization bits [Because every variance below a threshold is made zero, it would be obvious to make the BN denominator constant equal to the threshold to preserve quantization]), and
the variable avoids division by zero in the filter process of the Batch normalization layer (Sheng, section 2.3.1 - teaches properly handling normalization values of zero variance channels [or filters] to better preserve quantization bits [Because every variance below a threshold is made zero, it would be obvious to make the BN denominator constant equal to the threshold to preserve quantization]).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to modify Gadosey in view of Mao with the teachings of Sheng in order to provide a quantization-friendly separable convolution architecture and improve accuracy in the field of model compression (Sheng, Abstract – “As deep learning (DL) is being rapidly pushed to edge computing, researchers invented various ways to make inference computation more efficient on mobile/IoT devices, such as network pruning, parameter compression, and etc. Quantization, as one of the key approaches, can effectively offload GPU, and make it possible to deploy DL on fixed-point pipeline. Unfortunately, not all existing networks design are friendly to quantization. For example, the popular lightweight MobileNetV1 ..., while it successfully reduces parameter size and computation latency with separable convolution, our experiment shows its quantized models have large accuracy gap against its float point models. To resolve this, we analyzed the root cause of quantization loss and proposed a quantization-friendly separable convolution architecture. By evaluating the image classification task on ImageNet2012 dataset, our modified MobileNetV1 model can archive 8-bit inference top-1 accuracy in 68.03%, almost closed the gap to the float pipeline.”).
Claim(s) 10-11, 13, 15-16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gadosey et al. (On Pruned, Quantized and Compact CNN Architectures for Vision Applications, hereinafter referred to as “Gadosey”) in view of Mao et al. (Efficient Convolution Neural Networks for Object Tracking Using Separable Convolution and Filter Pruning, hereinafter referred to as “Mao”) and further in view of Yao et al. (US 2022/0129759 A1 – Universal Loss-Error-Aware Quantization for Deep Neural Networks with Flexible Ultra-Low-Bit Weights and Activations, hereinafter referred to as “Yao”).
Regarding claim 10 (Currently Amended), Gadosey teaches an information processing apparatus, comprising:
a central processing unit (CPU) (Gadosey, section 4 – teaches running python packages on GPUs and CPUs) configured to:
select a first filter in a neural network (Gadosey, section 3.1.1 – teaches pruning filters [requiring selection]), wherein
the neural network includes a Depthwise convolution layer, a Pointwise convolution layer, and a Batch normalization layer (Gadosey, section 2.3 – teaches MobileNet architecture based on depthwise separable convolutions comprising a depthwise convolutional layer followed by a pointwise convolutional layer with a batch normalization layer and a ReLU activation between each convolutional layer), and
quantize the neural network based on the elimination of the selected first filter (Gadosey, section 3.1.2 – teaches quantizing the neural network after pruning filters; see also Gadosey, Algorithm 1), wherein the quantization of the neural network includes:
quantization of activation data associated with the neural network (Gadosey, section 2.2 – teaches quantizing weights and activations using different bit-widths), and
quantization of weight data associated with the neural network (Gadosey, section 2.2 – teaches quantizing weights and activations using different bit-widths) ...
While Gadosey teaches pruning, quantization and depthwise separable convolutions, Gadosey does not explicitly teach a filter elimination processing section that eliminates a filter whose variance is smaller than a threshold in a neural network having a Depthwise convolution layer, a Pointwise convolution layer, and a Batch normalization layer. Further, while Gadosey teaches quantizing weights and activations, Gadosey does not explicitly teach quantization of weight data associated with the neural network subsequent to the quantization of the activation data.
Mao teaches
select a first filter in a neural network (Mao, section III.C. – teaches removing filters [requiring selection]), wherein
the neural network includes a Depthwise convolution layer, a Pointwise convolution layer, and a Batch normalization layer (Mao, section. III.B. – teaches a separable convolutional structure with depthwise convolutional layers, pointwise convolutional layers and batch normalization layers), and
the selection of the first filter is in the Batch normalization layer subsequent to the Depthwise convolution layer (Mao, section III.C. – teaches pruning the filter in depthwise convolutional layer based on the variance of the normalized filter weights [filter set in BN layer]);
determine that the selected first filter has a variance smaller than a threshold (Mao, section III.C. – teaches removing filters based on a small [threshold] variance);
eliminate the selected first filter in the Batch normalization layer based on the determination that the selected first filter has the variance smaller than the threshold (Mao, section III.C. – teaches pruning the filter in depthwise convolutional layer based on a small variance of the normalized filter weights [filter set in BN layer]).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to modify Gadosey with the teachings of Mao in order to improve efficiency and reduce calculations in the field of model compression (Mao, Abstract – “Object tracking based on deep learning is a hot topic in computer vision with many applications. Due to high computation and memory costs, it is difficult to deploy convolutional neural networks (CNNs) for object tracking on embedded systems with limited hardware resources. This paper uses the Siamese network to construct the backbone of our tracker. The convolution layers used to extract features often have the highest costs, so more improvements should be focused on them to make the tracking more efficient. In this paper, the standard convolution is optimized by the separable convolution, which mainly includes a depthwise convolution and a pointwise convolution. To further reduce the calculation, filters in the depthwise convolution layer are pruned with filters variance. As there are different weight distributions in convolution layers, the filter pruning is guided by a hyper-parameter designed. With the improvements, the number of parameters is decreased to 13% of the original network and the computation is reduced to 23%. On the NVIDIA Jetson TX2, the tracking speed increased to 3.65 times on the CPU and 2.08 times on the GPU, without significant degradation of tracking performance in VOT benchmark.”).
While Gadosey in view of Mao teaches quantizing weights and activations, Gadosey in view of Mao does not explicitly teach quantization of weight data associated with the neural network subsequent to the quantization of the activation data.
Yao teaches wherein the quantization of the neural network includes:
quantization of activation data associated with the neural network (Yao, [0279]-[0280] - teaches a quantization of activations followed by a quantization of weights; see also Yao, Figs. 19-20; Yao, [0252]-[0256]), and
quantization of weight data associated with the neural network subsequent to the quantization of the activation data (Yao, [0279]-[0280] - teaches a quantization of activations followed by a quantization of weights; see also Yao, Figs. 19-20; Yao, [0252]-[0256]).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to modify Gadosey in view of Mao with the teachings of Yao in order to improve accuracy training efficiency and ease of implementation in the field of neural network quantization (Yao, [0243] – “In embodiments of this invention, based on our previous ELQ design, the present design further presents Universal Loss-error-aware Quantization (ULQ), a leading (accuracy, training efficiency and easy implementation) full ultra-low-bit DNN quantization which differs with existing solutions both in the optimization formulation and quantization strategy.”).
Regarding claim 11 (Currently Amended), Gadosey in view of Mao and further in view of Yao teaches all of the limitations of the apparatus of claim 10 as noted above. Yao further teaches wherein the CPU is further configured to:
execute a first relearning process after the quantization of the activation data (Yao, [0279]-[0280] - teaches a quantization of activations followed by an optimization based on a loss function which is then followed by a quantization and updating based on a loss function of the weights; see also Yao, Fig. 20); and
execute a second relearning process after the quantization of the weight data (Yao, [0279]-[0280] - teaches a quantization of activations followed by an optimization based on a loss function which is then followed by a quantization and updating based on a loss function of the weights; see also Yao, Fig. 20).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to combine the teachings of Gadosey, Mao and Yao in order to retrain after quantization to improve accuracy training efficiency and ease of implementation (Yao, [0243]).
Regarding claim 13 (Currently Amended), Gadosey in view of Mao and further in view of Yao teaches all of the limitations of the apparatus of claim 10 as noted above. Gadosey further teaches wherein each of the quantization of the activation data and the quantization of the weight data is after the elimination of the selected first filter (Gadosey, section 3.1.2 – teaches quantization after successful pruning; see also Gadosey, Algorithm 1).
It would have been obvious to one of ordinary skill in the art before the filing data of the claimed invention to combine the teachings of Gadosey, Mao and Yao for the same reasons as disclosed in claim 10 above.
Regarding claim 15 (Currently Amended), Gadosey in view of Mao and further in view of Yao teaches all of the limitations of the apparatus of claim 13 as noted above. Gadosey further teaches wherein the selected first filter is eliminated only once prior to each of the quantization of the activation data and the quantization of the weight data (Gadosey, section 3.1.2 – teaches quantization after successful pruning; see also Gadosey, Algorithm 1).
It would have been obvious to one of ordinary skill in the art before the filing data of the claimed invention to combine the teachings of Gadosey, Mao and Yao for the same reasons as disclosed in claim 13 above.
Regarding claim 16 (Currently Amended), Gadosey teaches a method, comprising:
in a computer device (Gadosey, section 4 – teaches running python packages on GPUs and CPUs):
selecting a filter in a neural network (Gadosey, section 3.1.1 – teaches pruning filters [requiring selection]), wherein
the neural network includes a Depthwise convolution layer, a Pointwise convolution layer, and a Batch normalization layer (Gadosey, section 2.3 – teaches MobileNet architecture based on depthwise separable convolutions comprising a depthwise convolutional layer followed by a pointwise convolutional layer with a batch normalization layer and a ReLU activation between each convolutional layer), and
quantizing activation data associated with the neural network based on the elimination of the selected filter (Gadosey, section 2.2 – teaches quantizing weights and activations using different bit-widths; Gadosey, section 3.1.2 – teaches quantizing the neural network after pruning filters; see also Gadosey, Algorithm 1); and
quantizing weight data associated with the neural network (Gadosey, section 2.2 – teaches quantizing weights and activations using different bit-widths; Gadosey, section 3.1.2 – teaches quantizing the neural network after pruning filters; see also Gadosey, Algorithm 1) ...
While Gadosey teaches pruning, quantization and depthwise separable convolutions, Gadosey does not explicitly teach a filter elimination processing section that eliminates a filter whose variance is smaller than a threshold in a neural network having a Depthwise convolution layer, a Pointwise convolution layer, and a Batch normalization layer. Further, while Gadosey teaches quantizing weights and activations, Gadosey does not explicitly teach quantizing weight data associated with the neural network subsequent to the quantization of the activation data.
Mao teaches
selecting a filter in a neural network (Mao, section III.C. – teaches removing filters [requiring selection]), wherein
the neural network includes a Depthwise convolution layer, a Pointwise convolution layer, and a Batch normalization layer (Mao, section. III.B. – teaches a separable convolutional structure with depthwise convolutional layers, pointwise convolutional layers and batch normalization layers), and
the selection of the filter is in the Batch normalization layer subsequent to the Depthwise convolution layer (Mao, section III.C. – teaches pruning the filter in depthwise convolutional layer based on the variance of the normalized filter weights [filter set in BN layer]);
determining that the selected filter has a variance smaller than a threshold (Mao, section III.C. – teaches removing filters based on a small [threshold] variance);
eliminating the selected filter in the Batch normalization layer based on the determination that the selected filter has the variance smaller than the threshold (Mao, section III.C. – teaches pruning the filter in depthwise convolutional layer based on a small variance of the normalized filter weights [filter set in BN layer]).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to modify Gadosey with the teachings of Mao in order to improve efficiency and reduce calculations in the field of model compression (Mao, Abstract – “Object tracking based on deep learning is a hot topic in computer vision with many applications. Due to high computation and memory costs, it is difficult to deploy convolutional neural networks (CNNs) for object tracking on embedded systems with limited hardware resources. This paper uses the Siamese network to construct the backbone of our tracker. The convolution layers used to extract features often have the highest costs, so more improvements should be focused on them to make the tracking more efficient. In this paper, the standard convolution is optimized by the separable convolution, which mainly includes a depthwise convolution and a pointwise convolution. To further reduce the calculation, filters in the depthwise convolution layer are pruned with filters variance. As there are different weight distributions in convolution layers, the filter pruning is guided by a hyper-parameter designed. With the improvements, the number of parameters is decreased to 13% of the original network and the computation is reduced to 23%. On the NVIDIA Jetson TX2, the tracking speed increased to 3.65 times on the CPU and 2.08 times on the GPU, without significant degradation of tracking performance in VOT benchmark.”).
While Gadosey in view of Mao teaches quantizing weights and activations, Gadosey in view of Mao does not explicitly teach quantizing weight data associated with the neural network subsequent to the quantization of the activation data.
Yao teaches quantizing weight data associated with the neural network subsequent to the quantization of the activation data (Yao, [0279]-[0280] - teaches a quantization of activations followed by a quantization of weights; see also Yao, Figs. 19-20; Yao, [0252]-[0256]).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to modify Gadosey in view of Mao with the teachings of Yao in order to improve accuracy training efficiency and ease of implementation in the field of neural network quantization (Yao, [0243] – “In embodiments of this invention, based on our previous ELQ design, the present design further presents Universal Loss-error-aware Quantization (ULQ), a leading (accuracy, training efficiency and easy implementation) full ultra-low-bit DNN quantization which differs with existing solutions both in the optimization formulation and quantization strategy.”).
Claim(s) 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gadosey in view of Mao, further in view of Yao and further in view of Luo et al. (ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression, hereinafter referred to as “Luo”).
Regarding claim 14 (Currently Amended), Gadosey in view of Mao and further in view of Yao teaches all of the limitations of the apparatus of claim 13 as noted above. However, Gadosey in view of Mao and further in view of Yao does not explicitly teach wherein the elimination of the selected first filter is before the quantization of the activation data, and the CPU is further configured to eliminate a second filter after the quantization of the activation data.
Luo teaches wherein
the elimination of the selected first filter is before the quantization of the activation data (Luo, sections 1, 2 - teaches further compressing the pruning with quantization; Luo, section 3 - teaches a layer by layer pruning process; [combining quantization with pruning means pruning a first layer, quantizing the first layer, pruning a second layer]), and
the CPU is further configured to eliminate a second filter after the quantization of the activation data (Luo, sections 1, 2 - teaches further compressing the pruning with quantization; Luo, section 3 - teaches a layer by layer pruning process; [combining quantization with pruning means pruning a first layer, quantizing the first layer, pruning a second layer]).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to modify Gadosey in view of Mao and further in view of Yao with the teachings of Luo in order to provide an effective strategy to compress and accelerate off the shelf models in the field of model compression (Luo, Abstract – “We propose an efficient and unified framework, namely ThiNet, to simultaneously accelerate and compress CNN models in both training and inference stages. We focus on the filter level pruning, i.e., the whole filter would be discarded if it is less important. Our method does not change the original network structure, thus it can be perfectly supported by any off-the-shelf deep learning libraries. We formally establish filter pruning as an optimization problem, and reveal that we need to prune filters based on statistics information computed from its next layer, not the current layer, which differentiates ThiNet from existing methods. Experimental results demonstrate the effectiveness of this strategy, which has advanced the state-of-the-art. We also show the performance of ThiNet on ILSVRC-12 benchmark. ThiNet achieves 3.31x FLOPs reduction and 16.63x compression on VGG-16, with only 0.52% top-5 accuracy drop. Similar experiments with ResNet-50 reveal that even for a compact network, ThiNet can also reduce more than half of the parameters and FLOPs, at the cost of roughly 1% top-5 accuracy drop. Moreover, the original VGG-16 model can be further pruned into a very small model with only 5.05MB model size, preserving AlexNet level accuracy but showing much stronger generalization ability.”).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communication from the examiner should be directed to MARSHALL WERNER whose telephone number is (469) 295-9143. The examiner can normally be reached on Monday – Thursday 7:30 AM – 4:30 PM ET.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kamran Afshar, can be reached at (571) 272-7796. The fax number for the organization where this application or proceeding is assigned is (571) 273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MARSHALL L WERNER/ Primary Examiner, Art Unit 2125
Prosecution Insights