DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 05/03/2023 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea and does not integrate the judicial exception into a practical application or amount to significantly more than the judicial exception.
Regarding claim 1
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites multiple mental processes, as explained below. The claim recites, inter alia:
“…the method comprising: obtaining: …and a quantization operator that is based on a power function having a power exponent; and quantizing the neural network based on the quantization operator,
“the quantization including searching for an optimal value of the power exponent based on a quantization error associated with the quantization operator.”
This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed by the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) III. C.).
Step 2A Prong 2: This judicial exception is not integrated into a practical. In particular, the claim only recites additional elements that are mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). The additional element of “A computer-implemented method for neural network quantization… a neural network”, as drafted, is reciting generic computer components. The generic computer components in these steps are recited at a high-level of generality (i.e., as a generic computer component performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into practical application, the additional element of using generic computer components to perform the abstract idea amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Thus, the claim is not patent eligible.
Regarding claim 2
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1:
“wherein searching for an optimal value of the power exponent includes: minimizing the quantization error; or determining a power exponent value compliant with the following criteria: minimization of the quantization error; and use a default power exponent, and/or minimization of a distance between the obtained neural network prior to quantization and the quantized neural network.”
This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed by the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) III. C.).
Step 2A Prong 2: This judicial exception is not integrated into a practical. In particular, the claim only recites additional elements that are mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). The additional element of “neural network”, as drafted, is reciting generic computer components. The generic computer components in these steps are recited at a high-level of generality (i.e., as a generic computer component performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of using generic computer components to perform the abstract idea amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Thus, the claim is not patent eligible.
Regarding claim 3
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1:
PNG
media_image1.png
198
611
media_image1.png
Greyscale
This limitation is directed to the abstract idea of a mathematical concept which can be performed by the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) III. C.).
Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B.
Regarding claim 4
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
PNG
media_image2.png
241
681
media_image2.png
Greyscale
Step 2A Prong 1: The claim recites multiple mental processes, as explained below. The claim recites, inter alia:
This limitation is directed to the abstract idea of a mathematical concept which can be performed by the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) III. C.).
This limitation is directed to the abstract idea of a mathematical concept which can be performed by the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) III. C.).
Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B.
Regarding claim 5
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites multiple mental processes, as explained below. The claim recites, inter alia:
“wherein, for at least one layer with a signed activation function Act with input x and weights
PNG
media_image3.png
18
53
media_image3.png
Greyscale
(Qa(x + CAct)Qa(W)) is approximated by xW + CAct W, CAct W being a bias term.”
This limitation is directed to the abstract idea of a mathematical concept which can be performed by the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) III. C.).
Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B.
Regarding claim 6
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1:
Step 2A Prong 2: This judicial exception is not integrated into a practical. In particular, the claim only recites additional elements that are mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). The additional element of “wherein the method comprises performing the quantization during a post-training calibration of the neural network.”, as drafted, is reciting generic computer components. The generic computer components in these steps are recited at a high-level of generality (i.e., as a generic computer component performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The additional element of using generic computer components to perform the abstract idea amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Regarding claim 7
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1:
Step 2A Prong 2: This judicial exception is not integrated into a practical. In particular, the claim only recites additional elements that are mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). The additional element of “wherein performing the quantization during a post- training calibration of the neural network uses a gradient descent.”, as drafted, is reciting generic computer components. The generic computer components in these steps are recited at a high-level of generality (i.e., as a generic computer component performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer component.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into practical application, the additional element of using generic computer components to perform the abstract idea amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Thus, the claim is not patent eligible.
Regarding claim 8
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1:
Step 2A Prong 2: This judicial exception is not integrated into a practical. In particular, the claim only recites additional elements that are mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). The additional element of “wherein the method comprises performing the quantization by training the neural network, or by finetuning the neural network, the training comprising using power functions for quantization simulation”, as drafted, is reciting generic computer components. The generic computer components in these steps are recited at a high-level of generality (i.e., as a generic computer component performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer component.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into practical application, the additional element of using generic computer components to perform the abstract idea amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Thus, the claim is not patent eligible.
Regarding claim 9
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites multiple mental processes, as explained below. The claim recites, inter alia:
Step 2A Prong 2: This judicial exception is not integrated into a practical. In particular, the claim only recites additional elements that are mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). The additional element of “wherein the quantization using the quantization operator concerns at least a part of the layers of the neural network.”, as drafted, is reciting generic computer components. The generic computer components in these steps are recited at a high-level of generality (i.e., as a generic computer component performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The additional element of using generic computer components to perform the abstract idea amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Regarding claim 10
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites multiple mental processes, as explained below. The claim recites, inter alia:
“wherein, during inference using the quantized…, power of products are accumulated instead of accumulating the quantized products.”
This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed by the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) III. C.).
Step 2A Prong 2: This judicial exception is not integrated into a practical. In particular, the claim only recites additional elements that are mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). The additional element of “neural networks”, as drafted, is reciting generic computer components. The generic computer components in these steps are recited at a high-level of generality (i.e., as a generic computer component performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into practical application, the additional element of using generic computer components to perform the abstract idea amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Thus, the claim is not patent eligible.
Regarding claim 11
Step 1: The claim recites a method; therefore, it falls into the statutory category of processes.
Step 2A Prong 1: The claim recites multiple mental processes, as explained below. The claim recites, inter alia:
“A …method for performing multiplication of quantized matrices with a quantization operator that is based on a power function having a power exponent, the method comprising accumulating power of products.”
This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed by the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) III. C.).
Step 2A Prong 2: This judicial exception is not integrated into a practical. In particular, the claim only recites additional elements that are mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). The additional element of “computer-implemented”, as drafted, is reciting generic computer components. The generic computer components in these steps are recited at a high-level of generality (i.e., as a generic computer component performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into practical application, the additional element of using generic computer components to perform the abstract idea amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Thus, the claim is not patent eligible.
Regarding claims 12-20
Claim 12-20 recite analogous limitations to claims 1-11 and therefore is rejected on the same ground as claims 1-11.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-2, 8-13 and 17-18 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Yvinec et al. (“POWERQUANT: AUTOMORPHISM SEARCH FOR NONUNIFORM QUANTIZATION”).
Regarding claim 1
Yvinec teaches a computer-implemented method for neural network quantization, (section 3 “Let F be a trained feed forward neural network with L layers, each comprising a weight tensor Wl. Let Q be a quantization operator such that the quantized weights Q(Wl) are represented on b bits. The most popular such operator is the uniform one.”)
the method comprising: obtaining: a neural network, and a quantization operator that is based on a power function having a power exponent; (abstract “To find this parameter, we propose to optimize the reconstruction error of each layer: in particular, we show that this procedure is locally convex and admits a unique solution. At inference time, we show that our approach, dubbed PowerQuant, only require simple modifications in the quantized DNN activation functions.”)
and quantizing the neural network based on the quantization operator, (abstract “In this paper, we identity the uniformity of the quantization operator as a limitation of existing approaches, and propose a data-free non-uniform method. More specifically, we argue that to be readily usable without dedicated hardware and implementation, non-uniform quantization shall not change the nature of the mathematical operations performed by the DNN”)
the quantization including searching for an optimal value of the power exponent based on a quantization error associated with the quantization operator. (Section 3 “We argue that, despite its simplicity, the choice of such a uniform operator is responsible for a significant part of the quantization error. In fact, the weights Wl most often follow a bell-shaped distribution for which uniform quantization is ill-suited: intuitively, in such a case, we would want to quantize more precisely the small weights on the peak of the distribution. For this reason, the most popular non-uniform quantization scheme is logarithmic quantization, outputting superior performance.” Also see the optimal value section 4.2 “Empirically, optimal values a ∗ for the exponent parameter are centered on 0.55, which approximately corresponds to the first distribution in Fig 2. Still, as shown on Table 1 we observe some variations on the best value for a which motivates the optimization of a for each network and bitwidth. Furthermore, our results provide a novel insight on the difference between pruning and quantization.”)
Regarding claim 8
Yvinec teaches the method of claim 1.
Yvinec further teaches wherein the method comprises performing the quantization by training the neural network, or by finetuning the neural network, (section 3 “Let F be a trained feed forward neural network with L layers, each comprising a weight tensor Wl . Let Q be a quantization operator such that the quantized weights Q(Wl) are represented on b bits. The most popular such operator is the uniform one. We argue that, despite its simplicity, the choice of such a uniform operator is responsible for a significant part of the quantization error. In fact, the weights Wl most often follow a bell-shaped distribution for which uniform quantization is ill-suited: intuitively, in such a case, we would want to quantize more precisely the small weights on the peak of the distribution”)
the training comprising using power functions for quantization simulation. (Abstract “This leads to search among the continuous automorphisms of (R ∗ +, ×), which boils down to the power functions defined by their exponent. To find this parameter, we propose to optimize the reconstruction error of each layer: in particular, we show that this procedure is locally convex and admits a unique solution.”)
Regarding claim 9
Yvinec teaches the method of claim 1.
Yvinec further teaches wherein the quantization using the quantization operator concerns at least a part of the layers of the neural network. (Abstract “In this paper, we identity the uniformity of the quantization operator as a limitation of existing approaches, and propose a data-free non-uniform method. More specifically, we argue that to be readily usable without dedicated hardware and implementation, non-uniform quantization shall not change the nature of the mathematical operations performed by the DNN. This leads to search among the continuous automorphisms of (R ∗ +, ×), which boils down to the power functions defined by their exponent. To find this parameter, we propose to optimize the reconstruction error of each layer: in particular, we show that this procedure is locally convex and admits a unique solution.”)
Regarding claim 10
Yvinec teaches the method of claim 1.
Yvinec further teaches wherein, during inference using the quantized neural networks, power of products are accumulated instead of accumulating the quantized products. (Section 3 “Let F be a trained feed forward neural network with L layers, each comprising a weight tensor Wl . Let Q be a quantization operator such that the quantized weights Q(Wl) are represented on b bits. The most popular such operator is the uniform one. We argue that, despite its simplicity, the choice of such a uniform operator is responsible for a significant part of the quantization error. In fact, the weights Wl most often follow a bell-shaped distribution for which uniform quantization is ill-suited: intuitively, in such a case, we would want to quantize more precisely the small weights on the peak of the distribution.”)
Regarding claim 11
Yvinec teaches a computer-implemented method for performing multiplication of quantized matrices with a quantization operator that is based on a power function having a power exponent, the method comprising accumulating power of products. (Abstract “In this paper, we identity the uniformity of the quantization operator as a limitation of existing approaches, and propose a data-free non-uniform method. More specifically, we argue that to be readily usable without dedicated hardware and implementation, non-uniform quantization shall not change the nature of the mathematical operations performed by the DNN. This leads to search among the continuous automorphisms of (R ∗ +, ×), which boils down to the power functions defined by their exponent. To find this parameter, we propose to optimize the reconstruction error of each layer: in particular, we show that this procedure is locally convex and admits a unique solution. At inference time, we show that our approach, dubbed PowerQuant, only require simple modifications in the quantized DNN activation functions”)
Regarding claim 12
Yvinec teaches a device comprising a non-transitory computer-readable data storage medium having recorded thereon: (Pg. 21 “Overall, the proposed approach can be easily implemented and induces negligible overhead in inference on GPU. To furthermore justify the practicality of the proposed quantization process, we recall that the only practicality concern that may arise is on the activation function as the other operations are strictly identical to standard uniform quantization.”)
a computer program comprising instructions for performing: a method for neural network quantization, (abstract “In this paper, we identity the uniformity of the quantization operator as a limitation of existing approaches, and propose a data-free non-uniform method. More specifically, we argue that to be readily usable without dedicated hardware and implementation, non-uniform quantization shall not change the nature of the mathematical operations performed by the DNN. This leads to search among the continuous automorphisms of (R ∗ +, ×), which boils down to the power functions defined by their exponent. To find this parameter, we propose to optimize the reconstruction error of each layer: in particular, we show that this procedure is locally convex and admits a unique solution.”)
the method comprising: obtaining: a neural network, and a quantization operator that is based on a power function having a power exponent; (abstract “To find this parameter, we propose to optimize the reconstruction error of each layer: in particular, we show that this procedure is locally convex and admits a unique solution. At inference time, we show that our approach, dubbed PowerQuant, only require simple modifications in the quantized DNN activation functions.”)
quantizing the neural network based on the quantization operator, the quantization including searching for an optimal value of the power exponent based on a quantization error associated with the quantization operator,
and/or a method for performing multiplication of quantized matrices with a quantization operator that is based on a power function having a power exponent, (Section 3 “We argue that, despite its simplicity, the choice of such a uniform operator is responsible for a significant part of the quantization error. In fact, the weights Wl most often follow a bell-shaped distribution for which uniform quantization is ill-suited: intuitively, in such a case, we would want to quantize more precisely the small weights on the peak of the distribution. For this reason, the most popular non-uniform quantization scheme is logarithmic quantization, outputting superior performance.” Also see the optimal value section 4.2 “Empirically, optimal values a ∗ for the exponent parameter are centered on 0.55, which approximately corresponds to the first distribution in Fig 2. Still, as shown on Table 1 we observe some variations on the best value for a which motivates the optimization of a for each network and bitwidth. Furthermore, our results provide a novel insight on the difference between pruning and quantization.”)
the method comprising accumulating power of products, and/or a quantized neural network obtainable according to the method for neural network quantization. (Section F “The proposed PowerQuant method preserves the multiplication operations, i.e. a multiplication in the floating point space remains a multiplication in the quantized space (integers). This allows one to leverage current implementations of uniform quantization available on most hardware Gholami et al. (2021); Zhou et al. (2016). However, while PowerQuant preserves multiplications it doesn’t preserve additions which are significantly less costly than multiplications. Consequently, in order to infer under the PowerQuant transformation, instead of accumulating the quantized products, as done in standard quantization Jacob et al. (2018), one need to accumulate the powers of said products. Formally, let’s consider two quantized weights w1, w2 and their respective quantized inputs x1, x2. The standard accumulation would be performed as follows w1x1+w2x2. In the case of PowerQuant, this would be done as (w1x1) 1 a +(w2x2) 1.”)
Regarding claim 17
Yvinec teaches the method of claim 12.
Yvinec further teaches the method further comprising a processor coupled to the data storage medium. (Pg. 21 “Overall, the proposed approach can be easily implemented and induces negligible overhead in inference on GPU. To furthermore justify the practicality of the proposed quantization process, we recall that the only practicality concern that may arise is on the activation function as the other operations are strictly identical to standard uniform quantization.”)
Regarding claim 18
Yvinec teaches the method of claim 13.
Yvinec further teaches the method further comprising a processor coupled to the data storage medium. (Pg. 21 “Overall, the proposed approach can be easily implemented and induces negligible overhead in inference on GPU. To furthermore justify the practicality of the proposed quantization process, we recall that the only practicality concern that may arise is on the activation function as the other operations are strictly identical to standard uniform quantization.”)
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 2 and 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yvinec et al. (“POWERQUANT: AUTOMORPHISM SEARCH FOR NONUNIFORM QUANTIZATION”) in view of Weng et al. (“Neural Network Quantization for Efficient Inference: A Survey”).
Regarding claim 2
Yvinec teaches the method of claim 1.
Yvinec further teaches wherein searching for an optimal value of the power exponent includes: minimizing the quantization error; or determining a power exponent value compliant with the following criteria: minimization of the quantization error; and use a default power exponent, and/or minimization of a distance between the obtained neural network prior to quantization and the quantized neural network.
Weng teaches wherein searching for an optimal value of the power exponent includes: minimizing the quantization error; (pg. 6 “In effect, they are introducing the noise produced by the chosen quantization method randomly and incrementally during training. This way the network is allowed to adjust to the quantization noise, minimizing the quantization error associated with the use of STE during QAT, which becomes especially apparent when quantizing to fewer than 8 bits.”)
or determining a power exponent value compliant with the following criteria: minimization of the quantization error;
and use a default power exponent, and/or minimization of a distance between the obtained neural network prior to quantization and the quantized neural network. (Pg. 2 left col “We want to quantize the weights in a way that maintains the network’s overall classification accuracy. In fact, trying to minimize the distance between the floating point and quantized weight representations does not directly translate to maintaining classification accuracy because of the over-parameterization of neural networks [18, 30]. Therefore, the over-parameterization wrinkle presents opportunities to reduce precision in clever ways without any cost to accuracy—at times, quantizing a NN even improves its accuracy”)
Yvinec and Weng are analogous art because they are both directed to Machine learning.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined method and system for optimizing each reconstruction error of each layer disclosed by Yvinec to include neural network quantization for efficient inferences of Weng in order to provide method and system for “reducing the size and complexity of neural networks by reducing the precision of a network” as disclosed by Weng (abstract “Neural network quantization has recently arisen to meet this demand of reducing the size and complexity of neural networks by reducing the precision of a network. With smaller and simpler networks, it becomes possible to run neural networks within the constraints of their target hardware. This paper surveys the many neural network quantization techniques that have been developed in the last decade”).
Regarding claim 13
Claim 13 recites analogous limitations to claim 2 and therefore is rejected on the same ground as claim 2.
Claim(s) 6-7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yvinec et al. (“POWERQUANT: AUTOMORPHISM SEARCH FOR NONUNIFORM QUANTIZATION”) in view of Nagel et al. (“A White Paper on Neural Network Quantization”).
Regarding claim 6
Yvinec teaches the method of claim 1.
Yvinec does not teach wherein the method comprises performing the quantization during a post-training calibration of the neural network.
Nagel teaches wherein the method comprises performing the quantization during a post-training calibration of the neural network. (section 3.1 “Quantization range setting refers to the method of determining clipping thresholds of the quantization grid, qmin and qmax (see equation 7). The key trade-off in range setting is between clipping and rounding error, described in section 2.2, and their impact on the final task loss for each quantizer being configured. Each of the methods described here provides a different trade-off between the two quantities. These methods typically optimize local cost functions instead of the task loss. This is because in PTQ we aim for computationally fast methods without the need for end-to-end training. Weights can usually be quantized without any need for calibration data”)
Yvinec and Nagel are analogous art because they are both directed to Machine learning.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined method and system for optimizing each reconstruction error of each layer disclosed by Yvinec to include neural network quantization using post training calibration of Nagel in order to provide a method or system with “sufficient for achieving 8-bit quantization with close to floating-point accuracy” as disclosed by Nagel (abstract “In this white paper, we introduce state-of-the-art algorithms for mitigating the impact of quantization noise on the network’s performance while maintaining low-bit weights and activations. We start with a hardware motivated introduction to quantization and then consider two main classes of algorithms: Post-Training Quantization (PTQ) and Quantization-Aware-Training (QAT). PTQ requires no re-training or labelled data and is thus a lightweight push-button approach to quantization. In most cases, PTQ is sufficient for achieving 8-bit quantization with close to floating-point accuracy. QAT requires fine-tuning and access to labeled training data but enables lower bit quantization with competitive results.”).
Regarding claim 7
Yvinec in view of Nagel teaches the method of claim 6.
Nagel further teaches wherein performing the quantization during a post-training calibration of the neural network uses a gradient descent. (Pg. 20 “Using this gradient definition we can now back-propagate through the quantization blocks. Figure 10 shows a simple computational graph for the forward and backward pass used in quantization-aware training. The forward pass is identical to that of figure 4, but in the backward pass we effectively skip the quantizer block due to the STE assumption. In earlier QAT work the quantization range”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined method and system for optimizing each reconstruction error of each layer disclosed by Yvinec to include neural network quantization using post training calibration of Nagel in order to provide a method or system with “sufficient for achieving 8-bit quantization with close to floating-point accuracy” as disclosed by Nagel (abstract “In this white paper, we introduce state-of-the-art algorithms for mitigating the impact of quantization noise on the network’s performance while maintaining low-bit weights and activations. We start with a hardware motivated introduction to quantization and then consider two main classes of algorithms: Post-Training Quantization (PTQ) and Quantization-Aware-Training (QAT). PTQ requires no re-training or labelled data and is thus a lightweight push-button approach to quantization. In most cases, PTQ is sufficient for achieving 8-bit quantization with close to floating-point accuracy. QAT requires fine-tuning and access to labeled training data but enables lower bit quantization with competitive results.”).
Allowable Subject Matter
Claims 3-5 and 14-16 and 19-20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to VAN C MANG whose telephone number is (571)270-7598. The examiner can normally be reached Mon - Fri 8:00-5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, David Yi can be reached at 5712707519. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/VAN C MANG/Primary Examiner, Art Unit 2126