DETAILED ACTION
This action is responsive to the amendment filed on 09/14/2026. Claims 1-2, 4, 6-10, 12-14, and 16-19 are pending in the case. Claims 1-2, 4, 6-9, 13, and 18-19 are currently amended. Claims 1, 9, 13, 18, and 19 are independent claims.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 09/14/2026 has been entered.
Priority
Acknowledgment is made of applicant's claim for domestic priority based on PCT application No. PCT/JP2021/024836 filed on 06/30/2021, which claims priority to a foreign application filed in Japan on 06/30/2020. Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 06/11/2026 is being considered by the examiner.
The information disclosure statement (IDS) submitted on 07/13/2026 is being considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-2, 4, 6-10, 12-14, and 16-19 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claim 1:
Step 1 Statutory Category: Claim 1 is directed to a device, which falls under one of the four statutory categories.
Step 2A Prong 1 Judicial exception: Claim 1 recites, in part, “… generate the neural network execution model based on hardware information regarding hardware in which the neural network execution model is operating and network information regarding the neural network”. This limitation, under the broadest reasonable interpretation, and in light of paragraph 0080 of applicant’s specification, covers the recitation of mathematical concepts, see MPEP §2106.04(a)(2)(I). Further, the claim recites: “… generate learned parameters of the generated neural network execution model”. This limitation, under the claim interpretation under 35 U.S.C. 112(f) and applicant’s specification paragraph 0181, covers the recitation of mathematical concepts, see MPEP §2106.04(a)(2)(I). Further, the claim recites: “partition convolution operations implemented in the neural network execution model so that the first memory stores partitioned input data and the second memory stores partitioned output data”. This limitation, under the broadest reasonable interpretation, and according to applicant’s specification paragraph 0082, covers the recitation of mathematical concepts, see MPEP §2106.04(a)(2)(I).
Step 2A Prong 2 Integration into a practical application: This judicial exception is not integrated into a practical application. In particular the claim recites: “a neural network generating device”, “a processor”, “the processor is configured to”, and “the neural network execution model includes a first memory for storing input data, a convolution operation circuit for performing a convolution operation on the input data stored in the first memory, a second memory for storing convolution operation output data of the convolution operation circuit, and a quantization operation circuit for performing a quantization operation on the convolution operation output data stored in the second memory”. These limitations are additional elements that amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §2106.05(f). Further, the claim recites: “a neural network execution model for operating a neural network” and “the neural network execution model being capable of being embedded in an embedded device”. These limitations are additional elements that generally link the use of the judicial exception to a particular technological environment or field of use, see MPEP §2106.05(h).
Step 2B Significantly more: The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements: “a neural network generating device”, “a processor”, “the processor is configured to”, and “the neural network execution model includes a first memory for storing input data, a convolution operation circuit for performing a convolution operation on the input data stored in the first memory, a second memory for storing convolution operation output data of the convolution operation circuit, and a quantization operation circuit for performing a quantization operation on the convolution operation output data stored in the second memory” amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. Elements that merely amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process cannot provide an inventive concept. Further, the additional elements: “a neural network execution model for operating a neural network” and “the neural network execution model being capable of being embedded in an embedded device” generally link the use of the judicial exception to a particular technological environment or field of use. Elements that merely generally link the use of the judicial exception to a particular technological environment or field of use cannot provide an inventive concept. The claim is not patent eligible.
Regarding claim 2, the rejection of claim 1 is incorporated, and further the claim recites: “…generate a neural network hardware model based on the hardware information and the neural network execution model”. This limitation, under the broadest reasonable interpretation, recites mathematical concepts in addition to those identified in the rejection of the parent claim, thus the claim recites a judicial exception.
Further, the claim recites: “the processor is configured to”. This limitation is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §2106.05(f). Elements that merely amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process cannot provide an inventive concept. The claim is not patent eligible.
Regarding claim 4, the rejection of claim 1 is incorporated, and further, the claim recites: “…perform associated operations implemented when generating the learned parameters with higher precision than operations implemented by the neural network execution model”. This limitation is a continuation of the “… generates learned parameters of the generated neural network execution model” limitation identified as an abstract idea in the rejection of the parent claim, thus the claim recites a judicial exception.
Further, the claim recites: “the processor is configured to”. This limitation is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §2106.05(f). Elements that merely amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process cannot provide an inventive concept. The claim is not patent eligible.
Regarding claim 6, the rejection of claim 5 is incorporated, and further, the claim recites: “…perform convolution operations implemented when generating the learned parameter with higher precision than operations implemented by the convolution operation circuit”. This limitation is a continuation of the “… generates learned parameters of the generated neural network execution model” limitation identified as an abstract idea in the rejection of the parent claim, thus the claim recites a judicial exception.
Further, the claim recites: “the processor is configured to”. This limitation is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §2106.05(f). Elements that merely amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process cannot provide an inventive concept. The claim is not patent eligible.
Regarding claim 7, the rejection of claim 5 is incorporated, and further, the claim recites: “…learn quantization parameters that the quantization operation circuit uses for the quantization operations”. This limitation, according to applicant’s specification paragraph 0181, recites mathematical concepts in addition to those identified in the rejection of the parent claim, thus the claim recites a judicial exception.
Further, the claim recites: “the processor is configured to”. This limitation is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §2106.05(f). Elements that merely amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process cannot provide an inventive concept. The claim is not patent eligible.
Regarding claim 8, the rejection of claim 7 is incorporated, and further, the claim recites: “…learn a scaling factor for quantization operation output data quantized by the quantization parameters when learning the quantization parameters”. This limitation, under the broadest reasonable interpretation, covers the recitation of mathematical concepts in addition to those identified in the rejection of the parent claim, thus the claim recites a judicial exception.
Further, the claim recites: “the processor is configured to”. This limitation is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §2106.05(f). Elements that merely amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process cannot provide an inventive concept. The claim is not patent eligible.
Regarding claim 9:
Step 1 Statutory Category: Claim 9 is directed to a method, which falls under one of the four statutory categories.
Step 2A Prong 1 Judicial exception: Claim 9 recites, in part, “generating the neural network execution model … based on the hardware information and the network information”. This limitation, under the broadest reasonable interpretation, and in light of paragraph 0080 of applicant’s specification, covers the recitation of mathematical concepts, see MPEP §2106.04(a)(2)(I). Further, the claim recites: “learning learned parameters of the generated neural network execution model”. This limitation, under the claim interpretation under 35 U.S.C. 112(f) and applicant’s specification paragraph 0181, covers the recitation of mathematical concepts, see MPEP §2106.04(a)(2)(I). Further, the claim recites: “generating the neural network execution model comprises partitioning convolution operations implemented in the neural network execution model so that the first memory stores partitioned input data and the second memory stores partitioned output data”. This limitation, under the broadest reasonable interpretation, and according to applicant’s specification paragraph 0082, covers the recitation of mathematical concepts, see MPEP §2106.04(a)(2)(I).
Step 2A Prong 2 Integration into a practical application: This judicial exception is not integrated into a practical application. In particular the claim recites: “acquiring hardware information regarding hardware in which the neural network execution model is operating” and “setting network information regarding the neural network”. These limitations, according to applicant’s specification paragraphs 0060 and 0063 respectively, are additional elements that amount to mere data gathering. It is necessary to acquire the data in order to use the recited judicial exception. Therefore, these limitations are insignificant extra-solution activity to the judicial exception, see MPEP §2106.05(g). Further, the claim recites: “a neural network execution model for operating a neural network” and “which is capable of being embedded in an embedded device”. These limitations are additional elements that generally link the use of the judicial exception to a particular technological environment or field of use. See MPEP §2106.05(h). Further, the claim recites: “the neural network execution model includes a first memory for storing input data, a convolution operation circuit for performing a convolution operation on the input data stored in the first memory, a second memory for storing convolution operation output data of the convolution operation circuit, and a quantization operation circuit for performing a quantization operation on the convolution operation output data stored in the second memory”. This limitation is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §2106.05(f).
Step 2B Significantly more: The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements: “acquiring hardware information regarding hardware in which the neural network execution model is operating” and “setting network information regarding the neural network” amount to mere data gathering and are thus insignificant extra-solution activity to the judicial exception. Further, these elements are directed to receiving or transmitting data over a network which courts have recognized as well-understood, routine, and conventional when they are claimed in a generic manner, see MPEP §2106.05(d)(II). Further, the additional elements: “a neural network execution model for operating a neural network” and “which is capable of being embedded in an embedded device” generally link the use of the judicial exception to a particular technological environment or field of use. Elements that merely generally link the use of the judicial exception to a particular technological environment or field of use cannot provide an inventive concept. Further, the additional element “the neural network execution model includes a first memory for storing input data, a convolution operation circuit for performing a convolution operation on the input data stored in the first memory, a second memory for storing convolution operation output data of the convolution operation circuit, and a quantization operation circuit for performing a quantization operation on the convolution operation output data stored in the second memory” amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. Elements that merely amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process cannot provide an inventive concept. The claim is not patent eligible.
Regarding claim 10, the rejection of claim 9 is incorporated, and further, claim 10 is substantially similar to claim 2 respectively, and is rejected in the same manner and reasoning applying.
Regarding claim 12, the rejection of claim 9 is incorporated, and further, claim 12 is substantially similar to claim 4 respectively, and is rejected in the same manner and reasoning applying.
Regarding claim 13:
Step 1 Statutory Category: Claim 13 is directed to a machine, which falls under one of the four statutory categories.
Step 2A Prong 1 Judicial exception: Claim 13 recites, in part, “… generate the neural network execution model … based on the hardware information and the network information”. This limitation, under the broadest reasonable interpretation, and in light of paragraph 0080 of applicant’s specification, covers the recitation of mathematical concepts, see MPEP §2106.04(a)(2)(I). Further, the claim recites: “… learn learned parameters of the generated neural network execution model”. This limitation, under the claim interpretation under 35 U.S.C. 112(f) and applicant’s specification paragraph 0181, covers the recitation of mathematical concepts, see MPEP §2106.04(a)(2)(I). Further, the claim recites: “partitioning convolution operations implemented in the neural network execution model so that the first memory stores partitioned input data and the second memory stores partitioned output data”. This limitation, under the broadest reasonable interpretation, and according to applicant’s specification paragraph 0082, covers the recitation of mathematical concepts, see MPEP §2106.04(a)(2)(I).
Step 2A Prong 2 Integration into a practical application: This judicial exception is not integrated into a practical application. In particular the claim recites: “a non-transitory computer-readable recording medium”, “a computer”, “making the computer”, and “the neural network execution model includes a first memory for storing input data, a convolution operation circuit for performing a convolution operation on the input data stored in the first memory, a second memory for storing convolution operation output data of the convolution operation circuit, and a quantization operation circuit for performing a quantization operation on the convolution operation output data stored in the second memory”. These limitations are additional elements that amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §2106.05(f). Further, the claim recites: “… acquire hardware information regarding hardware in which the neural network execution model is operating” and “… set network information regarding the neural network”. These limitations, according to applicant’s specification paragraphs 0060 and 0063 respectively, are additional elements that amount to mere data gathering. It is necessary to acquire the data in order to use the recited judicial exception. Therefore, these limitations are insignificant extra-solution activity to the judicial exception, see MPEP §2106.05(g). Further, the claim recites: “a neural network execution model for operating a neural network” and “which is capable of being embedded in an embedded device”. These limitations are additional elements that generally link the use of the judicial exception to a particular technological environment or field of use. See MPEP §2106.05(h).
Step 2B Significantly more: The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements: “a non-transitory computer-readable recording medium”, “a computer”, “making the computer”, and “the neural network execution model includes a first memory for storing input data, a convolution operation circuit for performing a convolution operation on the input data stored in the first memory, a second memory for storing convolution operation output data of the convolution operation circuit, and a quantization operation circuit for performing a quantization operation on the convolution operation output data stored in the second memory” amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. Elements that merely amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process cannot provide an inventive concept. Further, the additional elements: “… acquire hardware information regarding hardware in which the neural network execution model is operating” and “… set network information regarding the neural network” amount to mere data gathering and are thus insignificant extra-solution activity to the judicial exception. Further, these elements are directed to receiving or transmitting data over a network which courts have recognized as well-understood, routine, and conventional when they are claimed in a generic manner, see MPEP §2106.05(d)(II). Further, the additional elements: “a neural network execution model for operating a neural network” and “which is capable of being embedded in an embedded device” generally link the use of the judicial exception to a particular technological environment or field of use. Elements that merely generally link the use of the judicial exception to a particular technological environment or field of use cannot provide an inventive concept. The claim is not patent eligible.
Regarding claim 14, the rejection of claim 13 is incorporated, and further, claim 14 is substantially similar to claim 2 and claim 10 respectively, and is rejected in the same manner and reasoning applying.
Regarding claim 16, the rejection of claim 13 is incorporated, and further, claim 16 is substantially similar to claim 4 and claim 12 respectively, and is rejected in the same manner and reasoning applying.
Regarding claim 17, the rejection of claim 1 is incorporated, and further, the claim recites: “wherein: the first memory and the second memory are a first memory area and a second memory area in a same memory”. This limitation is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §2106.05(f). Elements that merely amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process cannot provide an inventive concept. The claim is not patent eligible.
Regarding claim 18:
Step 1 Statutory Category: Claim 18 is directed to a device, which falls under one of the four statutory categories.
Step 2A Prong 1 Judicial exception: Claim 18 recites, in part, “…generate the neural network execution model based on hardware information regarding hardware in which the neural network execution model is operating and network information regarding the neural network”. This limitation, under the broadest reasonable interpretation, and in light of paragraph 0080 of applicant’s specification, covers the recitation of mathematical concepts, see MPEP §2106.04(a)(2)(I). Further, the claim recites: “…generate learned parameters of the generated neural network execution model”. This limitation, under the claim interpretation under 35 U.S.C. 112(f) and applicant’s specification paragraph 0181, covers the recitation of mathematical concepts, see MPEP §2106.04(a)(2)(I). Further, the claim recites: “… perform convolution operations implemented when generating the learned parameter through backpropagation with higher precision than operations implemented by the convolution operation circuit; and … perform the operations performed when calculating output data through forward propagation based on the convolution operation circuit”. This limitation, under the claim interpretation under 35 U.S.C. 112(f) and applicant’s specification paragraph 0181, covers the recitation of mathematical concepts, see MPEP §2106.04(a)(2)(I).
Step 2A Prong 2 Integration into a practical application: This judicial exception is not integrated into a practical application. In particular the claim recites: “a neural network generating device”, “a processor”, “the processor is configured to”, and “the neural network execution model comprises a convolution operation circuit that implements convolution operations and a quantization operation circuit that implements quantization operations”. These limitations are additional elements that amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §2106.05(f). Further, the claim recites: “a neural network execution model for operating a neural network” and “the neural network execution model being capable of being embedded in an embedded device”. These limitations are additional elements that generally link the use of the judicial exception to a particular technological environment or field of use. See MPEP §2106.05(h).
Step 2B Significantly more: The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements: “a neural network generating device”, “a processor”, “the processor is configured to”, and “the neural network execution model comprises a convolution operation circuit that implements convolution operations and a quantization operation circuit that implements quantization operations” amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. Elements that merely amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process cannot provide an inventive concept. Further, the additional elements: “a neural network execution model for operating a neural network” and “the neural network execution model being capable of being embedded in an embedded device” generally link the use of the judicial exception to a particular technological environment or field of use. Elements that merely generally link the use of the judicial exception to a particular technological environment or field of use cannot provide an inventive concept. The claim is not patent eligible.
Regarding claim 19:
Step 1 Statutory Category: Claim 19 is directed to a device, which falls under one of the four statutory categories.
Step 2A Prong 1 Judicial exception: Claim 19 recites, in part, “…generate the neural network execution model based on hardware information regarding hardware in which the neural network execution model is operating and network information regarding the neural network”. This limitation, under the broadest reasonable interpretation, and in light of paragraph 0080 of applicant’s specification, covers the recitation of mathematical concepts, see MPEP §2106.04(a)(2)(I). Further, the claim recites: “…generate learned parameters of the generated neural network execution model”. This limitation, under the claim interpretation under 35 U.S.C. 112(f) and applicant’s specification paragraph 0181, covers the recitation of mathematical concepts, see MPEP §2106.04(a)(2)(I). Further, the claim recites: “learn quantization parameters that the quantization operation circuit uses for the quantization operations; … learn a scaling factor for quantization operation output data quantized by the quantization parameters when learning the quantization parameters”. This limitation, under the claim interpretation under 35 U.S.C. 112(f) and applicant’s specification paragraph 0181, covers the recitation of mathematical concepts, see MPEP §2106.04(a)(2)(I). Further, the claim recites: “performs the quantization operation using the quantization parameters incorporating the scaling factor”. This limitation, under the broadest reasonable interpretation, and in light of paragraph 0080 of applicant’s specification, covers the recitation of mathematical concepts, see MPEP §2106.04(a)(2)(I).
Step 2A Prong 2 Integration into a practical application: This judicial exception is not integrated into a practical application. In particular the claim recites: “a neural network generating device”, “a processor”, “the processor is configured to”, and “the neural network execution model comprises a convolution operation circuit that implements convolution operations and a quantization operation circuit that implements quantization operations”, and “the quantization operation circuit…”. These limitations are additional elements that amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §2106.05(f). Further, the claim recites: “a neural network execution model for operating a neural network” and “the neural network execution model being capable of being embedded in an embedded device”. These limitations are additional elements that generally link the use of the judicial exception to a particular technological environment or field of use. See MPEP §2106.05(h).
Step 2B Significantly more: The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements: “a neural network generating device”, “a processor”, “the processor is configured to”, and “the neural network execution model comprises a convolution operation circuit that implements convolution operations and a quantization operation circuit that implements quantization operations”, and “the quantization operation circuit…” amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. Elements that merely amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process cannot provide an inventive concept. Further, the additional elements: “a neural network execution model for operating a neural network” and “the neural network execution model being capable of being embedded in an embedded device” generally link the use of the judicial exception to a particular technological environment or field of use. Elements that merely generally link the use of the judicial exception to a particular technological environment or field of use cannot provide an inventive concept. The claim is not patent eligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 10, 13-14, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Konishi et al., U.S. Patent Application Publication No. 20210056420, hereinafter referred to as “Konishi” in view of Shibata et al., U.S. Patent Application Publication No. 20210110236, hereinafter referred to as “Shibata”.
Regarding claim 1, Konishi teaches A neural network generating device that generates a neural network execution model for operating a neural network, the neural network generating device comprising a processor (Konishi, Paragraph 0005, Lines 1-4, “the present disclosure provides a neural network construction device that contributes to an improvement in the efficiency of obtaining an optimal neural network by narrowing candidate neural networks”; Konishi, Paragraph 0190, “Some or all of the structural elements included in each device in the above embodiments may be one system LSI (Large Scale Integration: large scale integrated circuit). The system LSI is a super multifunctional LSI manufactured by integrating a plurality of components onto a single chip. Specifically, the system LSI is a computer system configured of a microprocessor, read-only memory (ROM), random-access memory (RAM), and so on. A computer program is stored in the RAM. The system LSI achieves its function as a result of the microprocessor operating according to the computer program”), wherein:
…
the neural network execution model being capable of being embedded in an embedded device (Konishi, Paragraph 0050, Lines 1-5, “In the above neural network construction device, the scale of the generated model of the neural network fits within the range where the model can be mounted on the embedded device according to a predetermined hardware specification”);
the processor is configured to generate the neural network execution model based on hardware information regarding hardware in which the neural network execution model is operating and network information regarding the neural network (Konishi, Paragraph 0049, Lines 1-3, “the first condition may include a resource condition related to a computational resource of an embedded device”; Konishi, Paragraph 0053, Lines 1-4, “ the first condition may include information of at least one of a size of input data input to the neural network or a size of output data output from the neural network”; Konishi, Paragraph 0053, Lines 10-17, “More specifically, the size of the input data may be dimensionality of the input data, the size of the output data may be dimensionality of the output data, and the one or more candidate hyperparameters may include both a total number of layers in the neural network and a total number of nodes in the neural network. Furthermore, the first condition may further include information indicating that the neural network is a convolutional neural network”; The “resource condition” is considered to be “hardware information” and the “embedded device” is considered to be the “hardware in which the neural network execution model is operating”. Further, the “size of the input data”, “size of output data” and the indication of the type of the neural network are considered to be the “network information”; Konishi, Paragraph 0045, Lines 1-10, “A neural network construction device according to the above technique includes: an obtainer which obtains a first condition and a second condition, the first condition being used to determine a candidate hyperparameter that is a candidate of a hyperparameter of a neural network to be constructed … a generator which generates the model of the neural network using the candidate hyperparameter”; The “model of the neural network” is considered to be the “neural network execution model”, and because the generator uses the “candidate hyperparameter” which is determined using the “first condition” which includes both the hardware and network information, thus the neural network execution model is generated “based on” the hardware and network information);
the processor is configured to generate learned parameters of the generated neural network execution model (Konishi, Paragraph 0083, Lines 1-2, “Learning unit 19 performs, using the learning data, learning of the model generated by generator 13”; Konishi, Paragraph 0057, Lines 1-2, “As a result of parameters such as the weight being determined by such learning”); and
Konishi does not explicitly teach the neural network execution model includes a first memory for storing input data, a convolution operation circuit for performing a convolution operation on the input data stored in the first memory, a second memory for storing convolution operation output data of the convolution operation circuit, and a quantization operation circuit for performing a quantization operation on the convolution operation output data stored in the second memory, nor the processor is configured to partition convolution operations implemented in the neural network execution model so that the first memory stores partitioned input data and the second memory stores partitioned output data.
Shibata teaches the neural network execution model includes a first memory for storing input data (Shibata, Paragraph 0090, Lines 7-10, “The primary memory includes the primary weight storage part 211, the primary input data memory part 112, and the primary operation result memory part 213”; The “primary input data memory part” is considered to be the “first memory”), a convolution operation circuit for performing a convolution operation on the input data stored in the first memory (Shibata, Paragraph 0092, “The convolutional operation part 43 executes the convolutional operation using the data stored in the primary input data storage part 212 and the weights stored in the primary weight storage part 211”), a second memory for storing convolution operation output data of the convolution operation circuit (Shibata, Paragraph 0091, Lines 4-7, “the primary operation result memory part 213 stores the result of the operation by the multiplication/addition part 204 (the result of a single convolution operation)”; The “primary operation result memory part” is considered to be the “second memory”), and a quantization operation circuit for performing a quantization operation on the convolution operation output data stored in the second memory (Shibata, Paragraph 0048, Lines 4-6, “The quantization part 101 quantizes a result of a convolutional operation in a convolutional neural network using input data and weights”; see also Shibata, Figure 1); and
the processor is configured to partition convolution operations implemented in the neural network execution model so that the first memory stores partitioned input data and the second memory stores partitioned output data (Shibata, Paragraph 0125, Lines 1-4, “the input data conversion part 402 extends one channel included in the input data of the first layer to a number of subchannels equal to the number of bits of the element that forms the input data of the first layer”; Shibata, Paragraph 0133, Lines 1-4, “The input data conversion part 402 supplies the subchannels generated as described above to the convolutional operation part 43 as logarithmic quantized input data (activation)”; Shibata, Paragraph 0134, “Since each element of the subchannel is an exponential power or truly zero, the multiplication required for the convolutional operation in the convolutional operation part 43 (or more precisely, the multiplication/addition part 204) can be realized by a shift operation. Also, in the convolutional operation, the result of the multiplication is equal to the result of the convolutional operation using the input data before the conversion and the result of the convolutional operation using the input data after the conversion, because the results of the multiplications are added-up after the multiplication of the corresponding elements of channels and weights”; Shibata, Paragraph 0099, “the convolutional operation control part 201 specifies the range of input data to perform the convolutional operation between the above weight group and the input data, and instructs the input data input/output part 203 to read the above input data (step S02)”; Shibata, Paragraph 0091, Lines 4-7, “the primary operation result memory part 213 stores the result of the operation by the multiplication/addition part 204 (the result of a single convolution operation)”; see also Shibata, Figure 8; Performing the convolution operations using “shift operation[s]” and then adding the results, is considered to be partitioned convolution operations, the convolutional operation control part specifies a “range” or partition of input data to perform a convolution on, and then the result is stored in the second memory; according to Figure 8, this is does iteratively, thus “partitioned output data” is stored in the second memory).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to have modified the neural network generating method of Konishi to include partitioning convolution operations and including two memories for use with the convolution circuit and quantization circuit as taught by Shibata. The motivation for doing so would have been that performing the multiplication process in a single shift operation reduces the number of multiplications required for convolution and allows for a reduced hardware scale, and using multiple memories reduces the memory load of each respectively (Shibata, Paragraph 0022, “an input data to be multiplied in a convolutional operation is rounded to a power of two representation, and the multiplication process of the rounded input data and its weights is performed in a single shift operation. As a result, the number of multiplications required for convolutional operation can be reduced and the hardware scale for performing convolutional operation can be reduced”; Shibata, Paragraph 0049, Lines 1-3, “The above inferential device 100 is configured so that the multiplication process in the convolutional operation can be realized by the shift operation”).
Regarding claim 2, the rejection of claim 1 is incorporated, and further, the proposed combination teaches the processor (Konishi, Paragraph 0190, “Some or all of the structural elements included in each device in the above embodiments may be one system LSI (Large Scale Integration: large scale integrated circuit). The system LSI is a super multifunctional LSI manufactured by integrating a plurality of components onto a single chip. Specifically, the system LSI is a computer system configured of a microprocessor, read-only memory (ROM), random-access memory (RAM), and so on. A computer program is stored in the RAM. The system LSI achieves its function as a result of the microprocessor operating according to the computer program”) is configured to generate a neural network hardware model based on the hardware information and the neural network execution model (Konishi, Paragraph 0056, Lines 22-31, “the neural network construction device may further include: a learning unit; and an outputter. The obtainer may further obtain learning data on the neural network, the determination unit may output data indicating a model generated by the generator and determined as meeting the second condition, the learning unit may perform, using the learning data, learning of the model indicated in the data output by the determination unit, and the outputter may output at least a part of the model that has already been learned”; Konishi, Paragraph 0068, “obtain an optimal neural network by selecting the optimal neural network from among candidates narrowed by excluding neural networks that cannot meet the condition”; Konishi, Paragraph 0062, “the outputter may output the model in a format of a source code in a language dependent on an arithmetic processing device. Furthermore, for example, the outputter may output the model in a format of a hardware description language”; Outputting the model in a “format of a source code in a language dependent on an arithmetic processing device” or “in a format of a hardware description language” is considered to be generating “based on the hardware information”; since the “optimal neural network” is selected for outputting, the generating is performed “based on the neural network execution model”).
Regarding claim 9, Konishi teaches A neural network generating method for generating a neural network execution model for operating a neural network (Konishi, Paragraph 0005, Lines 1-4, “the present disclosure provides a neural network construction device that contributes to an improvement in the efficiency of obtaining an optimal neural network by narrowing candidate neural networks”), the neural network generating method comprising:
acquiring hardware information regarding hardware in which the neural network execution model is operating; setting network information regarding the neural network (Konishi, Paragraph 0045, Lines 1-6, “A neural network construction device according to the above technique includes: an obtainer which obtains a first condition and a second condition, the first condition being used to determine a candidate hyperparameter that is a candidate of a hyperparameter of a neural network to be constructed”; Konishi, Paragraph 0049, Lines 1-3, “the first condition may include a resource condition related to a computational resource of an embedded device”; Konishi, Paragraph 0053, Lines 1-4, “ the first condition may include information of at least one of a size of input data input to the neural network or a size of output data output from the neural network”; Konishi, Paragraph 0053, Lines 10-17, “More specifically, the size of the input data may be dimensionality of the input data, the size of the output data may be dimensionality of the output data, and the one or more candidate hyperparameters may include both a total number of layers in the neural network and a total number of nodes in the neural network. Furthermore, the first condition may further include information indicating that the neural network is a convolutional neural network”; The “resource condition” is considered to be “hardware information” and the “embedded device” is considered to be the “hardware in which the neural network execution model is operating”. Further, the “size of the input data”, “size of output data” and the indication of the type of the neural network are considered to be the “network information”);
generating the neural network execution model which is capable of being embedded in an embedded device (Konishi, Paragraph 0050, Lines 1-5, “In the above neural network construction device, the scale of the generated model of the neural network fits within the range where the model can be mounted on the embedded device according to a predetermined hardware specification”) based on the hardware information and the network information (Konishi, Paragraph 0049, Lines 1-3, “the first condition may include a resource condition related to a computational resource of an embedded device”; Konishi, Paragraph 0053, Lines 1-4, “ the first condition may include information of at least one of a size of input data input to the neural network or a size of output data output from the neural network”; Konishi, Paragraph 0053, Lines 10-17, “More specifically, the size of the input data may be dimensionality of the input data, the size of the output data may be dimensionality of the output data, and the one or more candidate hyperparameters may include both a total number of layers in the neural network and a total number of nodes in the neural network. Furthermore, the first condition may further include information indicating that the neural network is a convolutional neural network”; The “resource condition” is considered to be “hardware information”; Further, the “size of the input data”, “size of output data” and the indication of the type of the neural network are considered to be the “network information”; Konishi, Paragraph 0045, Lines 1-10, “A neural network construction device according to the above technique includes: an obtainer which obtains a first condition and a second condition, the first condition being used to determine a candidate hyperparameter that is a candidate of a hyperparameter of a neural network to be constructed … a generator which generates the model of the neural network using the candidate hyperparameter”; The “generator” is considered to be the “execution model generation step”, the “model of the neural network” is considered to be the “neural network execution model”, and because the generator uses the “candidate hyperparameter” which is determined using the “first condition” which includes both the hardware and network information, thus the neural network execution model is generated “based on” the hardware and network information); and
learning learned parameters of the generated neural network execution model (Konishi, Paragraph 0083, Lines 1-2, “Learning unit 19 performs, using the learning data, learning of the model generated by generator 13”; Konishi, Paragraph 0057, Lines 1-2, “As a result of parameters such as the weight being determined by such learning”).
Konishi does not explicitly teach wherein: the neural network execution model includes a first memory for storing input data, a convolution operation circuit for performing a convolution operation on the input data stored in the first memory, a second memory for storing convolution operation output data of the convolution operation circuit, and a quantization operation circuit for performing a quantization operation on the convolution operation output data stored in the second memory; and generating the neural network execution model comprises partitioning convolution operations implemented in the neural network execution model so that the first memory stores partitioned input data and the second memory stores partitioned output data.
Shibata teaches wherein:
the neural network execution model includes a first memory for storing input data (Shibata, Paragraph 0090, Lines 7-10, “The primary memory includes the primary weight storage part 211, the primary input data memory part 112, and the primary operation result memory part 213”; The “primary input data memory part” is considered to be the “first memory”), a convolution operation circuit for performing a convolution operation on the input data stored in the first memory (Shibata, Paragraph 0092, “The convolutional operation part 43 executes the convolutional operation using the data stored in the primary input data storage part 212 and the weights stored in the primary weight storage part 211”), a second memory for storing convolution operation output data of the convolution operation circuit (Shibata, Paragraph 0091, Lines 4-7, “the primary operation result memory part 213 stores the result of the operation by the multiplication/addition part 204 (the result of a single convolution operation)”; The “primary operation result memory part” is considered to be the “second memory”), and a quantization operation circuit for performing a quantization operation on the convolution operation output data stored in the second memory (Shibata, Paragraph 0048, Lines 4-6, “The quantization part 101 quantizes a result of a convolutional operation in a convolutional neural network using input data and weights”; see also Shibata, Figure 1); and
generating the neural network execution model comprises partitioning convolution operations implemented in the neural network execution model so that the first memory stores partitioned input data and the second memory stores partitioned output data (Shibata, Paragraph 0125, Lines 1-4, “the input data conversion part 402 extends one channel included in the input data of the first layer to a number of subchannels equal to the number of bits of the element that forms the input data of the first layer”; Shibata, Paragraph 0133, Lines 1-4, “The input data conversion part 402 supplies the subchannels generated as described above to the convolutional operation part 43 as logarithmic quantized input data (activation)”; Shibata, Paragraph 0134, “Since each element of the subchannel is an exponential power or truly zero, the multiplication required for the convolutional operation in the convolutional operation part 43 (or more precisely, the multiplication/addition part 204) can be realized by a shift operation. Also, in the convolutional operation, the result of the multiplication is equal to the result of the convolutional operation using the input data before the conversion and the result of the convolutional operation using the input data after the conversion, because the results of the multiplications are added-up after the multiplication of the corresponding elements of channels and weights”; Shibata, Paragraph 0099, “the convolutional operation control part 201 specifies the range of input data to perform the convolutional operation between the above weight group and the input data, and instructs the input data input/output part 203 to read the above input data (step S02)”; Shibata, Paragraph 0091, Lines 4-7, “the primary operation result memory part 213 stores the result of the operation by the multiplication/addition part 204 (the result of a single convolution operation)”; see also Shibata, Figure 8; Performing the convolution operations using “shift operation[s]” and then adding the results, is considered to be partitioned convolution operations, the convolutional operation control part specifies a “range” or partition of input data to perform a convolution on, and then the result is stored in the second memory; according to Figure 8, this is does iteratively, thus “partitioned output data” is stored in the second memory).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to have modified the neural network generating device of Konishi to include partitioning convolution operations and including two memories for use with the convolution circuit and quantization circuit as taught by Shibata. The motivation for doing so would have been that performing the multiplication process in a single shift operation reduces the number of multiplications required for convolution and allows for a reduced hardware scale, and using multiple memories reduces the memory load of each respectively (Shibata, Paragraph 0022, “an input data to be multiplied in a convolutional operation is rounded to a power of two representation, and the multiplication process of the rounded input data and its weights is performed in a single shift operation. As a result, the number of multiplications required for convolutional operation can be reduced and the hardware scale for performing convolutional operation can be reduced”; Shibata, Paragraph 0049, Lines 1-3, “The above inferential device 100 is configured so that the multiplication process in the convolutional operation can be realized by the shift operation”).
Regarding claim 10, the rejection of claim 9 is incorporated, and further, the proposed combination teaches generating a neural network hardware model based on the hardware information and the neural network execution model (Konishi, Paragraph 0056, Lines 22-31, “the neural network construction device may further include: a learning unit; and an outputter. The obtainer may further obtain learning data on the neural network, the determination unit may output data indicating a model generated by the generator and determined as meeting the second condition, the learning unit may perform, using the learning data, learning of the model indicated in the data output by the determination unit, and the outputter may output at least a part of the model that has already been learned”; Konishi, Paragraph 0068, “obtain an optimal neural network by selecting the optimal neural network from among candidates narrowed by excluding neural networks that cannot meet the condition”; Konishi, Paragraph 0062, “ the outputter may output the model in a format of a source code in a language dependent on an arithmetic processing device. Furthermore, for example, the outputter may output the model in a format of a hardware description language”; Outputting the model in a “format of a source code in a language dependent on an arithmetic processing device” or “in a format of a hardware description language” is considered to be generating “based on the hardware information”; since the “optimal neural network” is selected for outputting, the generating is performed “based on the neural network execution model”).
Regarding claim 13, Konishi teaches A non-transitory computer-readable recording medium storing a neural network generating program for making a computer generate a neural network execution model for operating a neural network (Konishi, Paragraph 0008, Lines 1-8, “a recording medium according to one aspect of the present disclosure is a non-transitory computer-readable recording medium having recorded thereon a program to be executed by an arithmetic processing device included in a neural network construction device including the arithmetic processing device and a storage device. The program is executed by the arithmetic processing device to cause the neural network construction device to execute”; Konishi, Paragraph 0005, Lines 1-4, “the present disclosure provides a neural network construction device that contributes to an improvement in the efficiency of obtaining an optimal neural network by narrowing candidate neural networks”), the neural network generating program comprising:
making the computer acquire hardware information regarding hardware in which the neural network execution model is operating; making the computer set network information regarding the neural network (Konishi, Paragraph 0045, Lines 1-6, “A neural network construction device according to the above technique includes: an obtainer which obtains a first condition and a second condition, the first condition being used to determine a candidate hyperparameter that is a candidate of a hyperparameter of a neural network to be constructed”; Konishi, Paragraph 0049, Lines 1-3, “the first condition may include a resource condition related to a computational resource of an embedded device”; Konishi, Paragraph 0053, Lines 1-4, “ the first condition may include information of at least one of a size of input data input to the neural network or a size of output data output from the neural network”; Konishi, Paragraph 0053, Lines 10-17, “More specifically, the size of the input data may be dimensionality of the input data, the size of the output data may be dimensionality of the output data, and the one or more candidate hyperparameters may include both a total number of layers in the neural network and a total number of nodes in the neural network. Furthermore, the first condition may further include information indicating that the neural network is a convolutional neural network”; The “resource condition” is considered to be “hardware information” and the “embedded device” is considered to be the “hardware in which the neural network execution model is operating”. Further, the “size of the input data”, “size of output data” and the indication of the type of the neural network are considered to be the “network information”);
making the computer generate the neural network execution model which is capable of being embedded in an embedded device (Konishi, Paragraph 0050, Lines 1-5, “In the above neural network construction device, the scale of the generated model of the neural network fits within the range where the model can be mounted on the embedded device according to a predetermined hardware specification”), based on the hardware information and the network information (Konishi, Paragraph 0049, Lines 1-3, “the first condition may include a resource condition related to a computational resource of an embedded device”; Konishi, Paragraph 0053, Lines 1-4, “ the first condition may include information of at least one of a size of input data input to the neural network or a size of output data output from the neural network”; Konishi, Paragraph 0053, Lines 10-17, “More specifically, the size of the input data may be dimensionality of the input data, the size of the output data may be dimensionality of the output data, and the one or more candidate hyperparameters may include both a total number of layers in the neural network and a total number of nodes in the neural network. Furthermore, the first condition may further include information indicating that the neural network is a convolutional neural network”; The “resource condition” is considered to be “hardware information”; Further, the “size of the input data”, “size of output data” and the indication of the type of the neural network are considered to be the “network information”; Konishi, Paragraph 0045, Lines 1-10, “A neural network construction device according to the above technique includes: an obtainer which obtains a first condition and a second condition, the first condition being used to determine a candidate hyperparameter that is a candidate of a hyperparameter of a neural network to be constructed … a generator which generates the model of the neural network using the candidate hyperparameter”; The “generator” is considered to be the “execution model generation step”, the “model of the neural network” is considered to be the “neural network execution model”, and because the generator uses the “candidate hyperparameter” which is determined using the “first condition” which includes both the hardware and network information, thus the neural network execution model is generated “based on” the hardware and network information); and
making the computer learn learned parameters of the generated neural network execution model (Konishi, Paragraph 0083, Lines 1-2, “Learning unit 19 performs, using the learning data, learning of the model generated by generator 13”; Konishi, Paragraph 0057, Lines 1-2, “As a result of parameters such as the weight being determined by such learning”).
Konishi does not explicitly teach wherein: the neural network execution model includes a first memory for storing input data, a convolution operation circuit for performing a convolution operation on the input data stored in the first memory, a second memory for storing convolution operation output data of the convolution operation circuit, and a quantization operation circuit for performing a quantization operation on the convolution operation output data stored in the second memory; and making the computer generate the neural network execution model comprises partitioning convolution operations implemented in the neural network execution model so that the first memory stores partitioned input data and the second memory stores partitioned output data.
Shibata teaches wherein:
the neural network execution model includes a first memory for storing input data (Shibata, Paragraph 0090, Lines 7-10, “The primary memory includes the primary weight storage part 211, the primary input data memory part 112, and the primary operation result memory part 213”; The “primary input data memory part” is considered to be the “first memory”), a convolution operation circuit for performing a convolution operation on the input data stored in the first memory (Shibata, Paragraph 0092, “The convolutional operation part 43 executes the convolutional operation using the data stored in the primary input data storage part 212 and the weights stored in the primary weight storage part 211”), a second memory for storing convolution operation output data of the convolution operation circuit (Shibata, Paragraph 0091, Lines 4-7, “the primary operation result memory part 213 stores the result of the operation by the multiplication/addition part 204 (the result of a single convolution operation)”; The “primary operation result memory part” is considered to be the “second memory”), and a quantization operation circuit for performing a quantization operation on the convolution operation output data stored in the second memory (Shibata, Paragraph 0048, Lines 4-6, “The quantization part 101 quantizes a result of a convolutional operation in a convolutional neural network using input data and weights”; see also Shibata, Figure 1); and
making the computer generate the neural network execution model comprises partitioning convolution operations implemented in the neural network execution model so that the first memory stores partitioned input data and the second memory stores partitioned output data (Shibata, Paragraph 0125, Lines 1-4, “the input data conversion part 402 extends one channel included in the input data of the first layer to a number of subchannels equal to the number of bits of the element that forms the input data of the first layer”; Shibata, Paragraph 0133, Lines 1-4, “The input data conversion part 402 supplies the subchannels generated as described above to the convolutional operation part 43 as logarithmic quantized input data (activation)”; Shibata, Paragraph 0134, “Since each element of the subchannel is an exponential power or truly zero, the multiplication required for the convolutional operation in the convolutional operation part 43 (or more precisely, the multiplication/addition part 204) can be realized by a shift operation. Also, in the convolutional operation, the result of the multiplication is equal to the result of the convolutional operation using the input data before the conversion and the result of the convolutional operation using the input data after the conversion, because the results of the multiplications are added-up after the multiplication of the corresponding elements of channels and weights”; Shibata, Paragraph 0099, “the convolutional operation control part 201 specifies the range of input data to perform the convolutional operation between the above weight group and the input data, and instructs the input data input/output part 203 to read the above input data (step S02)”; Shibata, Paragraph 0091, Lines 4-7, “the primary operation result memory part 213 stores the result of the operation by the multiplication/addition part 204 (the result of a single convolution operation)”; see also Shibata, Figure 8; Performing the convolution operations using “shift operation[s]” and then adding the results, is considered to be partitioned convolution operations, the convolutional operation control part specifies a “range” or partition of input data to perform a convolution on, and then the result is stored in the second memory; according to Figure 8, this is does iteratively, thus “partitioned output data” is stored in the second memory).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to have modified the execution model generating device of Konishi to include partitioning convolution operations and including two memories for use with the convolution circuit and quantization circuit as taught by Shibata. The motivation for doing so would have been that performing the multiplication process in a single shift operation reduces the number of multiplications required for convolution and allows for a reduced hardware scale, and using multiple memories reduces the memory load of each respectively (Shibata, Paragraph 0022, “an input data to be multiplied in a convolutional operation is rounded to a power of two representation, and the multiplication process of the rounded input data and its weights is performed in a single shift operation. As a result, the number of multiplications required for convolutional operation can be reduced and the hardware scale for performing convolutional operation can be reduced”; Shibata, Paragraph 0049, Lines 1-3, “The above inferential device 100 is configured so that the multiplication process in the convolutional operation can be realized by the shift operation”).
Regarding claim 14, the rejection of claim 13 is incorporated, and further, the proposed combination teaches making the computer generate a neural network hardware model based on the hardware information and the neural network execution model (Konishi, Paragraph 0056, Lines 22-31, “the neural network construction device may further include: a learning unit; and an outputter. The obtainer may further obtain learning data on the neural network, the determination unit may output data indicating a model generated by the generator and determined as meeting the second condition, the learning unit may perform, using the learning data, learning of the model indicated in the data output by the determination unit, and the outputter may output at least a part of the model that has already been learned”; Konishi, Paragraph 0068, “obtain an optimal neural network by selecting the optimal neural network from among candidates narrowed by excluding neural networks that cannot meet the condition”; Konishi, Paragraph 0062, “ the outputter may output the model in a format of a source code in a language dependent on an arithmetic processing device. Furthermore, for example, the outputter may output the model in a format of a hardware description language”; Outputting the model in a “format of a source code in a language dependent on an arithmetic processing device” or “in a format of a hardware description language” is considered to be generating “based on the hardware information”; since the “optimal neural network” is selected for outputting, the generating is performed “based on the neural network execution model”).
Regarding claim 17, the rejection of claim 1 is incorporated, and further, the proposed combination teaches wherein: the first memory and the second memory are a first memory area and a second memory area m a same memory (Shibata, Paragraph 0090, Lines 7-10, “The primary memory includes the primary weight storage part 211, the primary input data memory part 112, and the primary operation result memory part 213”).
Claims 4, 6, 12, 16, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Konishi in view of Shibata in further view of Nagel et al., Up or Down? Adaptive Rounding for Post-Training Quantization, 04/22/2020, https://arxiv.org/pdf/2004.10568v1, hereinafter referred to as Nagel.
Regarding claim 4, the rejection of claim 1 is incorporated.
The proposed combination does not explicitly teach the processor is configured to perform associated operations implemented when generating the learned parameters with higher precision than operations implemented by the neural network execution model.
Nagel teaches the processor is configured to perform associated operations implemented when generating the learned parameters with higher precision than operations implemented by the neural network execution model (Nagel, Page 1, Section 1, Paragraph 4, “post-training quantization methods (Nagel et al., 2019; Cai et al., 2020; Choukroun et al., 2019; Banner et al., 2019), which can be more easily applied in practice. These types of methods allow for network quantization to happen on-the-fly when deploying models, without the user of the model spending time and energy on quantization” Because the quantization is not performed until after training, the operations performed by the learning unit would be at full precision, and afterward the quantization would occur, resulting in the operations performed by the neural network execution model having a lower precision).
It would have been obvious, to a person of ordinary skill in the art, before the effective filing date of the invention, to have modified the neural network generating device taught by the proposed combination to include post-training quantization as taught by Nagel. The motivation for doing so would have been that post-training quantization does not require re-training models and is more easily applied in practice (Nagel, Page 1, Section 1, Paragraph 3, Lines 3-7, “Although many methods exist that do quantization-aware training (Jacob et al., 2018; Louizos et al., 2019) and get excellent results, these methods require a user to spend significant time on re-training models and hyperparameter tuning”; Nagel et al., 2019; Cai et al., 2020; Choukroun et al., 2019; Banner et al., 2019), which can be more easily applied in practice. These types of methods allow for network quantization to happen on-the-fly when deploying models, without the user of the model spending time and energy on quantization”).
Regarding claim 6, the rejection of claim 5 is incorporated.
The proposed combination thus far does not explicitly teach the processor is configured to perform convolution operations implemented when generating the learned parameter with higher precision than operations implemented by the convolution operation circuit.
Nagel teaches the processor is configured to perform convolution operations implemented when generating the learned parameter with higher precision than operations implemented by the convolution operation circuit (Nagel, Page 1, Section 1, Paragraph 4, “post-training quantization methods (Nagel et al., 2019; Cai et al., 2020; Choukroun et al., 2019; Banner et al., 2019), which can be more easily applied in practice. These types of methods allow for network quantization to happen on-the-fly when deploying models, without the user of the model spending time and energy on quantization” Because the quantization is not performed until after training, the operations performed by the learning unit would be at full precision, and because the learning unit performs backpropagation of errors, convolutional operations are performed during training, and afterward the quantization would occur, resulting in the operations performed by the neural network execution model, which includes the convolution operation circuit, having a lower precision).
It would have been obvious, to a person of ordinary skill in the art, before the effective filing date of the invention, to have modified the neural network generating device taught by the proposed combination to include post-training quantization as taught by Nagel. The motivation for doing so would have been that post-training quantization does not require re-training models and is more easily applied in practice (Nagel, Page 1, Section 1, Paragraph 3, Lines 3-7, “Although many methods exist that do quantization-aware training (Jacob et al., 2018; Louizos et al., 2019) and get excellent results, these methods require a user to spend significant time on re-training models and hyperparameter tuning”; Nagel, Page 1, Section 1, Paragraph 4, “post-training quantization methods Nagel et al., 2019; Cai et al., 2020; Choukroun et al., 2019; Banner et al., 2019), which can be more easily applied in practice. These types of methods allow for network quantization to happen on-the-fly when deploying models, without the user of the model spending time and energy on quantization”).
Regarding claim 12, the rejection of claim 9 is incorporated.
The proposed combination does not explicitly teach learning learned parameters of the generated neural network execution model comprises performing associated operations implemented when generating the learned parameters with higher precision than operations implemented by the neural network execution model.
Nagel teaches learning learned parameters of the generated neural network execution model comprises performing associated operations implemented when generating the learned parameters with higher precision than operations implemented by the neural network execution model (Nagel, Page 1, Section 1, Paragraph 4, “post-training quantization methods (Nagel et al., 2019; Cai et al., 2020; Choukroun et al., 2019; Banner et al., 2019), which can be more easily applied in practice. These types of methods allow for network quantization to happen on-the-fly when deploying models, without the user of the model spending time and energy on quantization” Because the quantization is not performed until after training, the operations performed by the learning step would be at full precision, and afterward the quantization would occur, resulting in the operations performed by the neural network execution model having a lower precision).
It would have been obvious, to a person of ordinary skill in the art, before the effective filing date of the invention, to have modified the neural network generating method taught by the proposed combination to include post-training quantization as taught by Nagel. The motivation for doing so would have been that post-training quantization does not require re-training models and is more easily applied in practice (Nagel, Page 1, Section 1, Paragraph 3, Lines 3-7, “Although many methods exist that do quantization-aware training (Jacob et al., 2018; Louizos et al., 2019) and get excellent results, these methods require a user to spend significant time on re-training models and hyperparameter tuning”; Nagel et al., 2019; Cai et al., 2020; Choukroun et al., 2019; Banner et al., 2019), which can be more easily applied in practice. These types of methods allow for network quantization to happen on-the-fly when deploying models, without the user of the model spending time and energy on quantization”).
Regarding claim 16, the rejection of claim 13 is incorporated.
The proposed combination does not explicitly teach making the computer learn learned parameters of the generated neural network execution model comprises performing associated operations implemented when generating the learned parameters with higher precision than operations implemented by the neural network execution model.
Nagel teaches making the computer learn learned parameters of the generated neural network execution model comprises performing associated operations implemented when generating the learned parameters with higher precision than operations implemented by the neural network execution model (Nagel, Page 1, Section 1, Paragraph 4, “post-training quantization methods (Nagel et al., 2019; Cai et al., 2020; Choukroun et al., 2019; Banner et al., 2019), which can be more easily applied in practice. These types of methods allow for network quantization to happen on-the-fly when deploying models, without the user of the model spending time and energy on quantization” Because the quantization is not performed until after training, the operations performed by the learning step would be at full precision, and afterward the quantization would occur, resulting in the operations performed by the neural network execution model having a lower precision).
It would have been obvious, to a person of ordinary skill in the art, before the effective filing date of the invention, to have modified the neural network generating method taught by the proposed combination to include post-training quantization as taught by Nagel. The motivation for doing so would have been that post-training quantization does not require re-training models and is more easily applied in practice (Nagel, Page 1, Section 1, Paragraph 3, Lines 3-7, “Although many methods exist that do quantization-aware training (Jacob et al., 2018; Louizos et al., 2019) and get excellent results, these methods require a user to spend significant time on re-training models and hyperparameter tuning”; Nagel et al., 2019; Cai et al., 2020; Choukroun et al., 2019; Banner et al., 2019), which can be more easily applied in practice. These types of methods allow for network quantization to happen on-the-fly when deploying models, without the user of the model spending time and energy on quantization”).
Regarding claim 18, Konishi teaches A neural network generating device that generates a neural network execution model for operating a neural network, the neural network generating device comprising a processor, wherein:
…
the neural network execution model being capable of being embedded in an embedded device (Konishi, Paragraph 0050, Lines 1-5, “In the above neural network construction device, the scale of the generated model of the neural network fits within the range where the model can be mounted on the embedded device according to a predetermined hardware specification”);
the processor is configured to generate the neural network execution model based on hardware information regarding hardware in which the neural network execution model is operating and network information regarding the neural network (Konishi, Paragraph 0049, Lines 1-3, “the first condition may include a resource condition related to a computational resource of an embedded device”; Konishi, Paragraph 0053, Lines 1-4, “ the first condition may include information of at least one of a size of input data input to the neural network or a size of output data output from the neural network”; Konishi, Paragraph 0053, Lines 10-17, “More specifically, the size of the input data may be dimensionality of the input data, the size of the output data may be dimensionality of the output data, and the one or more candidate hyperparameters may include both a total number of layers in the neural network and a total number of nodes in the neural network. Furthermore, the first condition may further include information indicating that the neural network is a convolutional neural network”; The “resource condition” is considered to be “hardware information” and the “embedded device” is considered to be the “hardware in which the neural network execution model is operating”. Further, the “size of the input data”, “size of output data” and the indication of the type of the neural network are considered to be the “network information”; Konishi, Paragraph 0045, Lines 1-10, “A neural network construction device according to the above technique includes: an obtainer which obtains a first condition and a second condition, the first condition being used to determine a candidate hyperparameter that is a candidate of a hyperparameter of a neural network to be constructed … a generator which generates the model of the neural network using the candidate hyperparameter”; The “model of the neural network” is considered to be the “neural network execution model”, and because the generator uses the “candidate hyperparameter” which is determined using the “first condition” which includes both the hardware and network information, thus the neural network execution model is generated “based on” the hardware and network information);
the processor is configured to generate learned parameters of the generated neural network execution model (Konishi, Paragraph 0083, Lines 1-2, “Learning unit 19 performs, using the learning data, learning of the model generated by generator 13”; Konishi, Paragraph 0057, Lines 1-2, “As a result of parameters such as the weight being determined by such learning”).
Konishi does not explicitly teach the neural network execution model comprises a convolution operation circuit that implements convolution operations and a quantization operation circuit that implements quantization operations; the processor is configured to perform convolution operations implemented when generating the learned parameter through backpropagation with higher precision than operations implemented by the convolution operation circuit; and the processor is configured to perform the operations performed when calculating output data through forward propagation based on the convolution operation circuit.
Shibata teaches the neural network execution model comprises a convolution operation circuit that implements convolution operations and a quantization operation circuit that implements quantization operations (Shibata, Paragraph 0048, Lines 1-8, “An inferential device according to an exemplary embodiment includes a quantization part 101, a convolutional operation part 102, and an input data conversion part 103 (see FIG. 1). The quantization part 101 quantizes a result of a convolutional operation in a convolutional neural network using input data and weights. The convolutional operation part 102 performs convolutional operation using the quantized operation result as input data”; The “quantization part” is considered to be the “quantization operation circuit” and the “convolutional operation part” is considered to be the “convolution operation circuit”);
the processor is configured to perform convolution operations implemented when generating the learned parameter through backpropagation … and the processor is configured to perform the operations performed when calculating output data through forward propagation based on the convolution operation circuit (Shibata, Paragraph 0189, Lines 11-17, “the learning device can be configured by adding an error back propagation part that performs error back propagation, a weight update part that performs weight update, and the like to the inferential device 10 described above. The error back propagation part and the weight update part can be realized by a known algorithm, etc., and the explanation is omitted”; A person of ordinary skill in the art would recognize that in order to use backpropagation to generate learned parameters, forward propagation is required to perform calculating output).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to have modified the neural network execution model of Konishi to include a convolution operation circuit and a quantization operation circuit and using backpropagation for learning as taught by Shibata. The motivation for doing so would have been that performing quantization allows for convolution operations to be performed using a single shift operation, which reduces the number of multiplications required for convolution operations and reduces the hardware scale required, and backpropagation is easy to implement as it is a well-known and well tested method (Shibata, Paragraph 0022, "an input data to be multiplied in a convolutional operation is rounded to a power of two representation, and the multiplication process of the rounded input data and its weights is performed in a single shift operation. As a result, the number of multiplications required for convolutional operation can be reduced and the hardware scale for performing convolutional operation can be reduced"; Shibata, Paragraph 0049, Lines 1-3, "The above inferential device 100 is configured so that the multiplication process in the convolutional operation can be realized by the shift operation"; Shibata, Paragraph 0189).
The proposed combination does not explicitly teach the processor is configured to perform convolution operations implemented when generating the learned parameter … with higher precision than operations implemented by the convolution operation circuit.
Nagel teaches the processor is configured to perform convolution operations implemented when generating the learned parameter … with higher precision than operations implemented by the convolution operation circuit (Nagel, Page 1, Section 1, Paragraph 4, “post-training quantization methods (Nagel et al., 2019; Cai et al., 2020; Choukroun et al., 2019; Banner et al., 2019), which can be more easily applied in practice. These types of methods allow for network quantization to happen on-the-fly when deploying models, without the user of the model spending time and energy on quantization” Because the quantization is not performed until after training, the operations performed by the learning unit would be at full precision, and afterward the quantization would occur, resulting in the operations performed by the neural network execution model having a lower precision).
It would have been obvious, to a person of ordinary skill in the art, before the effective filing date of the invention, to have modified the neural network generating device taught by the proposed combination to include post-training quantization as taught by Nagel. The motivation for doing so would have been that post-training quantization does not require re-training models and is more easily applied in practice (Nagel, Page 1, Section 1, Paragraph 3, Lines 3-7, “Although many methods exist that do quantization-aware training (Jacob et al., 2018; Louizos et al., 2019) and get excellent results, these methods require a user to spend significant time on re-training models and hyperparameter tuning”; Nagel et al., 2019; Cai et al., 2020; Choukroun et al., 2019; Banner et al., 2019), which can be more easily applied in practice. These types of methods allow for network quantization to happen on-the-fly when deploying models, without the user of the model spending time and energy on quantization”).
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Konishi in view of Shibata, in further view of Yoshiyama et al., U.S. Patent Application Publication No. 20210110260, hereinafter referred to as “Yoshiyama”.
Regarding claim 7, the rejection of claim 1 is incorporated.
The proposed combination thus far does not explicitly teach the processor is configured to learn quantization parameters that the quantization operation circuit uses for the quantization operations.
Yoshiyama teaches the processor is configured to learn quantization parameters that the quantization operation circuit uses for the quantization operations (Yoshiyama, Paragraph 55, Lines 7-12, “a learning unit 110 that optimizes parameters that determine a dynamic range by an error back propagation and a stochastic gradient descent in a quantization function of a neural network in which the parameters that determine the dynamic range are arguments”).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to have modified the neural network generating device of the proposed combination to include learning quantization parameters as taught by Yoshiyama. The motivation to do so would have been that learning the quantization parameters using error backpropagation using the learning unit reduces processing load and realizes learning with higher accuracy (Yoshiyama, Paragraph 0008, “As described above, according to the present disclosure, it is possible to reduce processing load of operation and to realize learning with higher accuracy”).
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Konishi in view of Shibata, in further view of Yoshiyama in further view of Choi et al., PACT: PARAMETERIZED CLIPPING ACTIVATION FOR QUANTIZED NEURAL NETWORKS, 07/17/2018, https://arxiv.org/pdf/1805.06085, hereinafter referred to as “Choi”.
Regarding claim 8, the rejection of claim 7 is incorporated.
The proposed combination does not explicitly teach the processor is configured to learn a scaling factor for quantization operation output data quantized by the quantization parameters when learning the quantization parameters.
Choi teaches the processor is configured to learn a scaling factor for quantization operation output data quantized by the quantization parameters when learning the quantization parameters (Choi, Page 1, Abstract, Lines 7-9, “PArameterized Clipping acTivation (PACT), uses an activation clipping parameter α that is optimized during training to find the right quantization scale” The “clipping parameter α” is considered to be the “scaling factor; Choi, Page 5, Section 4.2, Paragraph 2, “the best scope for α was to share α per layer. This choice also reduces hardware complexity because α needs to be multiplied only once after all multiply-accumulate (MAC) operations in reduced-precision in a layer are completed”).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to have modified the neural network generating device as taught by the proposed combination to include learning a scaling factor as taught by Choi. The motivation for doing so would have been that the scaling factor enables neural networks to maintain accuracy while working with ultra-low precision weights and activations (Choi, Abstract, Lines 4-7, “This paper proposes a novel quantization scheme for activations during training - that enables neural networks to work well with ultra low precision weights and activations without any significant accuracy degradation”).
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Konishi in view of Shibata, in further view of Esser et al., LEARNED STEP SIZE QUANTIZATION, 05/07/2020, https://arxiv.org/pdf/1902.08153, hereinafter referred to as “Esser”.
Regarding claim 19, Konishi teaches A neural network generating device that generates a neural network execution model for operating a neural network, the neural network generating device comprising a process (Konishi, Paragraph 0005, Lines 1-4, “the present disclosure provides a neural network construction device that contributes to an improvement in the efficiency of obtaining an optimal neural network by narrowing candidate neural networks”), wherein:
…
the neural network execution model being capable of being embedded in an embedded device (Konishi, Paragraph 0050, Lines 1-5, “In the above neural network construction device, the scale of the generated model of the neural network fits within the range where the model can be mounted on the embedded device according to a predetermined hardware specification”);
the processor is configured to generate the neural network execution model based on hardware information regarding hardware in which the neural network execution model is operating and network information regarding the neural network (Konishi, Paragraph 0049, Lines 1-3, “the first condition may include a resource condition related to a computational resource of an embedded device”; Konishi, Paragraph 0053, Lines 1-4, “ the first condition may include information of at least one of a size of input data input to the neural network or a size of output data output from the neural network”; Konishi, Paragraph 0053, Lines 10-17, “More specifically, the size of the input data may be dimensionality of the input data, the size of the output data may be dimensionality of the output data, and the one or more candidate hyperparameters may include both a total number of layers in the neural network and a total number of nodes in the neural network. Furthermore, the first condition may further include information indicating that the neural network is a convolutional neural network”; The “resource condition” is considered to be “hardware information” and the “embedded device” is considered to be the “hardware in which the neural network execution model is operating”. Further, the “size of the input data”, “size of output data” and the indication of the type of the neural network are considered to be the “network information”; Konishi, Paragraph 0045, Lines 1-10, “A neural network construction device according to the above technique includes: an obtainer which obtains a first condition and a second condition, the first condition being used to determine a candidate hyperparameter that is a candidate of a hyperparameter of a neural network to be constructed … a generator which generates the model of the neural network using the candidate hyperparameter”; The “model of the neural network” is considered to be the “neural network execution model”, and because the generator uses the “candidate hyperparameter” which is determined using the “first condition” which includes both the hardware and network information, thus the neural network execution model is generated “based on” the hardware and network information);
the processor is configured to generate learned parameters of the generated neural network execution model (Konishi, Paragraph 0083, Lines 1-2, “Learning unit 19 performs, using the learning data, learning of the model generated by generator 13”; Konishi, Paragraph 0057, Lines 1-2, “As a result of parameters such as the weight being determined by such learning”).
Konishi does not explicitly teach the neural network execution model comprises a convolution operation circuit that implements convolution operations and a quantization operation circuit that implements quantization operations; the processor is configured to learn quantization parameters that the quantization operation circuit uses for the quantization operations; the processor is configured to learn a scaling factor for quantization operation output data quantized by the quantization parameters when learning the quantization parameters; and the quantization operation circuit performs the quantization operation using the quantization parameters incorporating the scaling factor.
Shibata teaches the neural network execution model comprises a convolution operation circuit that implements convolution operations and a quantization operation circuit that implements quantization operations (Shibata, Paragraph 0048, Lines 1-8, “An inferential device according to an exemplary embodiment includes a quantization part 101, a convolutional operation part 102, and an input data conversion part 103 (see FIG. 1). The quantization part 101 quantizes a result of a convolutional operation in a convolutional neural network using input data and weights. The convolutional operation part 102 performs convolutional operation using the quantized operation result as input data”; The “quantization part” is considered to be the “quantization operation circuit” and the “convolutional operation part” is considered to be the “convolution operation circuit”);
the quantization operation circuit performs the quantization operation … (Shibata, Paragraph 0048, Lines 4-6, “The quantization part 101 quantizes a result of a convolutional operation in a convolutional neural network using input data and weights”).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to have modified the neural network execution model of Konishi to include a convolution operation circuit and a quantization operation circuit as taught by Shibata. The motivation for doing so would have been that performing quantization allows for convolution operations to be performed using a single shift operation, which reduces the number of multiplications required for convolution operations and reduces the hardware scale required (Shibata, Paragraph 0022, "an input data to be multiplied in a convolutional operation is rounded to a power of two representation, and the multiplication process of the rounded input data and its weights is performed in a single shift operation. As a result, the number of multiplications required for convolutional operation can be reduced and the hardware scale for performing convolutional operation can be reduced"; Shibata, Paragraph 0049, Lines 1-3, "The above inferential device 100 is configured so that the multiplication process in the convolutional operation can be realized by the shift operation").
The proposed combination does not explicitly teach ; the processor is configured to learn quantization parameters that the quantization operation circuit uses for the quantization operations; the processor is configured to learn a scaling factor for quantization operation output data quantized by the quantization parameters when learning the quantization parameters; nor quantization being performed using the quantization parameters incorporating the scaling factor.
Esser teaches learn quantization parameters that the quantization operation circuit uses for the quantization operations; the processor is configured to learn a scaling factor for quantization operation output data quantized by the quantization parameters when learning the quantization parameters (Esser, Page 3, Section 2, Lines 3-6, “Given data to quantize v, quantizer step size s, the number of positive and negative quantization levels
Q
P
and
Q
N
, respectively, we define a quantizer that computes
v
-
, a quantized and integer scaled representation of the data, and
v
^
, a quantized representation of the data at the same scale as v”; Esser, Page 4, Section 2.3, Lines 1-3, “Model quantizers are trained with LSQ by making their step sizes learnable parameters with loss gradient computed using the quantizer gradient described above, while other model parameters can be trained using existing techniques”; Esser, Page 3, Section 2.1, Lines 1-2 and Equation 3, “LSQ provides a means to learn s based on the training loss by introducing the following gradient through the quantizer to the step size parameter:
∂
v
^
∂
s
=
-
v
s
+
[
v
s
]
i
f
-
Q
N
<
v
s
<
Q
P
-
Q
N
i
f
v
s
≤
-
Q
N
Q
P
i
f
v
s
≥
Q
P
(3)”; “s” is considered to be the “scaling factor”) and
quantization being performed using the quantization parameters incorporating the scaling factor (Esser, Page 5, Section 3.2, Lines 1-2, “We trained several networks using LSQ and compare accuracy with other quantized networks and full precision baselines (Table 1)”; see also Esser, Table 1; In order to determine the collection of results in Table 1, quantization must have been performed according to “LSQ” meaning “using the quantization parameters incorporating the scaling factor”).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to have modified the neural network generating device of the proposed combination to include learning quantization parameters including a scaling parameter as taught by Esser and to modify the quantization circuit of the proposed combination to include performing quantization using the parameters and scaling factor as taught by Esser. The motivation to do so would have been that the method of Esser achieves the highest accuracy to date with a variety of architectures and requires only a simple modification of existing training code (Esser, Page 1, Abstract, Lines 4-7, “Learned Step Size Quantization, that achieves the highest accuracy to date on the ImageNet dataset when using models, from a variety of architectures, with weights and activations quantized to 2-, 3- or 4-bits of precision, and that can train 3-bit models that reach full precision baseline accuracy; Esser, Page 1, Abstract, Lines 12-14, “This approach works using different levels of precision as needed for a given system and requires only a simple modification of existing training code”).
Response to Arguments
Applicant’s arguments regarding the 35 U.S.C. 101 rejections of the claims have been fully considered but are unpersuasive.
Argument 1:
Applicant first argues, in the final paragraph of page 12 of the response, that the limitations of “generate the neural network execution model based on hardware information regarding hardware in which the neural network execution model is operating and network information regarding the neural network” and “partition convolution operations implemented in the neural network execution model so that the first memory stores partitioned input data and the second memory stores partitioned output data” do not claim mathematical calculations but rather merely involves mathematical concepts.
Examiner’s Response:
Examiner respectfully disagrees. The limitations indicated by the applicant, when read in light of the applicant’s specification recite mathematical calculations. Applicant’s specification paragraph 0067 discloses that the neural network execution model generation step includes layer mapping, convolution operation circuit generation, quantization operation circuit generation, and a DMAC generation; applicant’s specification paragraphs 0082-0088, 0090-0100, and additional paragraphs detailing steps S13-1 – S13-4, clearly disclose that mathematical calculations are required to generate the neural network generation model. Thus, the neural network execution model generation limitation recites mathematical limitations, and thus is considered an abstract idea. Further, applicant’s specification paragraphs 0082-0084 clearly disclose that mathematical calculations are required for partition the convolutional operations. Thus, the partitioning convolution operations limitation recites mathematical limitations, and thus is considered an abstract idea.
With regard to applicant’s argument in paragraph 1 of page 13 of the response, upon consideration of applicant’s amendments to the claims, the limitations indicated have been identified as mathematical concepts only.
Argument 2:
Applicant next argues, in paragraph 4 of page 15 of the response, that applicant’s claim 1 integrates the judicial exception into a practical application or amounts to significantly more than the judicial exception, in particular, applicant argues the practical application is “generating a neural network execution model that is embeddable in a particular device and can generate a neural network adapted to hardware configurations of the particular device”.
Examiner’s Response:
Examiner respectfully disagrees. Applicant’s alleged improvement is a mathematical concept as identified in the updated 35 U.S.C. 101 rejection above, and it is important to note that an inventive concept cannot be furnished by the unpatentable abstract idea itself, see MPEP 2106.05(I). Further, an important consideration in determining whether a claim improves technology is the extent to which the claim covers a particular solution to a problem or a particular way to achieve a desired outcome, as opposed to merely claiming the idea of a solution of outcome. Thus, simply claiming the idea of “generat[ing] the neural network execution model based on hardware information regarding hardware in which the neural network execution model is operating and network information regarding the neural network” does not provide a particular solution to a problem or a particular way to achieve a desired outcome, see MPEP 2106.05(a).
Argument 3:
Applicant next argues, on page 15, final paragraph of the response, that claim 1 integrates the exception into a practical application because the generated neural network execution model is tailored to run efficiently on an embedded device with limited resources.
Examiner’s Response:
Examiner respectfully disagrees. The added limitation of “the neural network execution model being capable of being embedded in an embedded device” merely amounts to an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. Elements that merely amount to generally linking the use of the judicial exception to a particular technological environment or field of use cannot provide an inventive concept. See MPEP §2106.05(h).
Argument 4:
Applicant next argues, on page 15, final paragraph – page 16, paragraph 1 of the response, that the convolutional and quantization circuits provides a specific solution that improves the efficiency of physical devices beyond the capabilities of generic computers, reducing the number of memory accesses and hardware size and power consumption.
Examiner’s Response:
Examiner respectfully disagrees. It is important to note that claiming the improved speed or efficiency inherent with applying the abstract idea on a computer does not integrate a judicial exception into a practical application or provide an inventive concept, see MPEP 2106.05(f).
Applicant's arguments regarding the remainder of the claims rely upon the arguments asserted with respect to the independent claims, and are thus unpersuasive.
Applicant’s arguments regarding the 35 U.S.C. 103 rejections of the claims have been fully considered but are unpersuasive.
Applicant first argues, in paragraph 2 of page 17 of the response, that “the partitioning in Shibata is not performed in response to memory capacity constraints typical of embedded devices with limited memory” and that partitioning the convolution operation “based on the first memory and the second memory is not obvious, particularly when operating on embedded devices”; and thus, Shibata fails to teach the partitioning limitation of claim 1. Examiner respectfully disagrees, in response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies (i.e., partitioning being performed in response to memory capacity constraints of embedded devices and partitioning based on two memories on embedded devices) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). While the claim does recite that the neural network execution model is capable of being embedded in an embedded device, the “memory capacity” is not positively recited in the claim and the hardware information and network information are not connected to the embedded device.
Applicant next argues, in the final paragraph of page 18 of the response, that the prior art of records does not teach varying computational precision between the forward propagation and backpropagation because it discloses post-training quantization methods rather than high-precision calculations during training. Examiner respectfully disagrees. In response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies (i.e., high-precision calculations during ) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). The claim only requires the convolutions performed through backpropagation to be higher precision than operations of the convolution operation circuit, which is accomplished by quantization after training as taught by Nagel.
Applicant next argues, in paragraph 2 of page 19 of the response that the prior art of record does not teach claim 19 as amended. Examiner respectfully disagrees. Applicant has provided no specific limitations or rationale for why the claim is believed to be distinguished from the prior art. Claim 19 is taught by the prior art for at least the reasons given with regard to the other independent claims as well as the rationale provided in the updated 35 U.S.C. 103 rejection seen above.
Applicant's arguments regarding the remainder of the claims rely upon the arguments asserted with respect to the independent claims, and are thus unpersuasive.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MOLLY CLARKE SIPPEL whose telephone number is (571)272-3270. The examiner can normally be reached Monday - Friday, 7:30 a.m. - 4:30 p.m. ET..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kakali Chaki can be reached at (571)272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/M.C.S./ Examiner, Art Unit 2122
/MICHAEL H HOANG/ PRIMARY EXAMINER, Art Unit 2122
Prosecution Insights