Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 3/7/2026 has been entered.
Remarks
This Office Action is responsive to Applicants' Amendment filed on March 7, 2026, in which claims 1, 4, 7, 14, and 17 are currently amended. Claims 6 and 19 are canceled. Claims 26-27 are newly added. Claims 1-5, 17-18, and 20-27 are currently pending.
Claim Objections
Claims 8 and 21 objected to because of the following informalities:
Regarding claim 8, "of the" appears in isolation, this appears to be a typo.
Regarding claims 8 and 21, "quantizing a layer of group" should read "quantizing a layer of a group".
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 8-9 and 21-22 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claims 8 and 21, "quantizing a layer of group into high precision format" is indefinite. It would be unclear to one of ordinary skill in the art how a layer could be quantized into a high precision format. This appears to be an oxymoron as one of ordinary skill in the art would expect quantization to reduce the precision (into a low precision format). There is no provided basis for relative comparison to determine what the high precision format is high relative to. In the interest of further examination "high precision format" is interpreted as simply a compound pronoun for the result format of any quantization.
Claims 9 and 22 are rejected with respect to their dependence on claims 8 and 21, respectively.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 7, 8, 10, 14, 21, and 23 are rejected under U.S.C. §103 as being unpatentable over the combination of Khan (“Learning to Quantize Deep Neural Networks: A Competitive-Collaborative Approach”, 2020), Wu (“MIXED PRECISION QUANTIZATION OF CONVNETS VIA DIFFERENTIABLE NEURAL ARCHITECTURE SEARCH”, 2018), and Barreto (“A Unifying Methodology for the Evaluation of Neural Network Models on Novelty Detection Tasks”, 2013).
Regarding claim 1, Khan teaches A method of generating a mixed precision quantization model for performing image processing, the method comprising:([p. 6 §V] "This framework runs a competition between layers to be selected and quantized to the next level, and then all layers collaborate to recover
the degradation of accuracy from quantization. Hence, it has mixed-precision levels for different layers, that is learned gradually through the competition and collaboration stages")
receiving, by a processor of a mixed precision quantization system, a validation dataset of images as input for quantization aware training of a neural network model comprising a plurality of layers in a low precision format;([p. 4 §IV] "a) Experimental settings: For our experiments, we use CIFAR10 [19] and ImageNet [20] datasets. CIFAR10 is an image classification dataset of 10 classes with 50000 training and 10000 validation images of size 32×32×3. ImageNet contains 1000 classes consisting of 1.2 million training samples and 50000 validation samples. We mostly use ResNet models as they are vastly being used by other approaches as well" [p. 6 §V] "This framework runs a competition between layers to be selected and quantized to the next level, and then all layers collaborate to recover the degradation of accuracy from quantization. Hence, it has mixed-precision levels for different layers, that is learned gradually through the competition and collaboration stages")
for each image of the validation dataset, a. providing, by the processor, the image as an input to train the neural network model;([p. 4 §IV] "a) Experimental settings: For our experiments, we use CIFAR10 [19] and ImageNet [20] datasets. CIFAR10 is an image classification dataset of 10 classes with 50000 training and 10000 validation images of size 32×32×3. ImageNet contains 1000 classes consisting of 1.2 million training samples and 50000 validation samples. We mostly use ResNet models as they are vastly being used by other approaches as well")
b. generating, by the processor, a union sensitivity list based on sensitivity values evaluated for the plurality of layers;([p. 3 Competition] "a) Competition: Based on the discussion above, the goal is to find the best quantization level for each layer in the network" [p. 3 Competition] "Formally, if at step t > 0 of the quantization, we have the quantized weight set of the previous step as Q (t−1) = n Q (t−1) 1 , . . . , Q (t−1) M o , each of which with possibly different quantization level from N (0) to N (K−1). Then, we randomly choose one layer based on the probability distribution p, say i, that has the bit precision of N (k) , 0 ≤ k < K − 1, and quantize it to the next level of bit precision" Quantized weight set interpreted as synonymous with sensitivity list corresponding to respective quantization error (sensitivity values).)
c. selecting, by the processor, [a group of] layers, of the neural network model, corresponding to a first set of sensitivity values of the union sensitivity list;([p. 2] "decide which layer to pick and quantize. After the quantization of the picked layer, we will probably have degradation in the overall accuracy, which we can recover by fine-tuning all the layers simultaneously. Then, we will continue this cycle until we reach the desired compression rate and accuracy level. This cycle suggests a two-stage framework, in which in the first stage, each layer competes with other layers to be picked and quantized" [p. 2] Quantization error driven [...] As mentioned before, the main idea behind the proposed frame work is that different parts of the model (e.g., layers) should have different levels of quantization since they are inherently different. This idea has been corroborated, even in schemes with the mere objective of quantization error")
d. generating, by the processor, a mixed precision quantization model by generating the selected [group of] layers;([p. 1] "we gradually quantize the layers, where at each quantization step, we run a competition between all layers, and based on the effect of their quantization to the next level of bit precision on the overall accuracy, one layer wins and gets quantized to the next level.")
e. computing, by the processor, accuracy of the mixed precision quantization model for comparison with a target accuracy; f. in response, by the processor, to determining that the accuracy of the mixed precision model is less than the target accuracy, perform steps c to e, by selecting a next group of layers corresponding to a next set of sensitivity values; and([p. 2 §IIIB] "the proposed algorithm iteratively alternates between competition and collaboration stages" [p. 5] "In order to mitigate this issue, we introduce an adaptive recovery scheme where we keep retraining until the network reaches an accuracy threshold" [p. 3 §IIIB(b) Collaboration] "we use the second approach, adaptive recovery, in which we set a threshold for accuracy, and continue the fine-tuning until we reach to that predefined threshold. This means that the number of epochs St will be set adaptively in training. We observed that some quantization steps need only one epoch to recover while some others need more than several epochs to fully recover the threshold. For more details on the collaboration part, the reader is referred to the next section" Khan explicitly teaches iteratively alternating between competition and collaboration until a threshold accuracy is reached based on a determined quantization policy.)
g. in response, by the processor, to determining that the accuracy of the mixed precision model is greater than or equal to the target accuracy, storing the mixed precision quantization model as a final mixed precision quantization model for image processing.([p. 3] "we use the second approach, adaptive recovery, in which we set a threshold for accuracy, and continue the fine-tuning until we reach to that predefined threshold" [p. 5] "we keep retraining until the network reaches an accuracy threshold" [p. 5] "Quantization of all the layers leads to a significant reduction in power requirement as well. We synthesized the RTL module for a MAC (Multiply and Accumulate) unit from the DesignWare Library [32] under 32nm technology node. Fig. 5 depicts the power consumption profile when we compare partially quantized networks having full precision in the first and last layers to our fully quantized ones in mixed precision" See Table II and FIG. 5 for performance of final stored mixed precision models for image processing.).
However, Khan does not explicitly teach c. selecting, by the processor, a group of layers, of the neural network model, corresponding to a first set of sensitivity values of the union sensitivity list;
d. generating, by the processor, a mixed precision quantization model by generating the selected group of layers;
wherein selecting the group of layers corresponding to the first set of sensitivity values comprises: clustering the sensitivity values based on the evaluated scores into a plurality of groups by: clustering the first set of sensitivity values associated with scores of greater than or equal to a first threshold into a first group; clustering a second set of sensitivity values associated with scores of greater than a second threshold and less than the first threshold into a second group; clustering a third set of sensitivity values associated with scores of greater than a third threshold and less than the second threshold into a third group; and clustering the remaining sensitivity values as the fourth set of sensitivity values of the union sensitivity list into a fourth group.
Wu, in the same field of endeavor, teaches c. selecting, by the processor, a group of layers, of the neural network model, corresponding to a first set of sensitivity values of the union sensitivity list;([p. 5 §5] "Each node vi in the super net corresponds to the output tensor (feature map) of layer-i. Each candidate edge ei,i+1k represents a convolution operator whose weights or activation are quantized to a lower precision" [p. 6 §6.1] "We conduct mixed precision search at the block level – all layers in one block use the same precision")
d. generating, by the processor, a mixed precision quantization model by generating the selected group of layers;([p. 6 6.1] "We conduct mixed precision search at the block level – all layers in one block use the same precision" [p. 6 §6.1] "For each block, we can choose a precision from {0, 1, 2, 3, 4, 8, 32}. If the precision is 0, we simply skip this block so the input and output are identical. If the precision is 32, we use the full-precision floating point weights. For all other precisions with k-bit, we quantize weights to k-bit fixed-point numbers. See Appendix B for more experiment details").
Khan as well as Wu are directed towards quantizing convolutional neural networks. Therefore, Khan as well as Wu are analogous art in the same field of endeavor. It would have been obvious before the effective filing date of the claimed invention to combine the teachings of Khan with the teachings of Wu by quantizing blocks/groups of layers. Wu provides as additional motivation for combination that this is a known method that would lead to obvious and expected results ([p. 7] "Similar to the CIFAR10 experiments, we conduct mixed precision search at the block level"). This motivation for combination also applies to the remaining claims which depend on this combination.
However, the combination of Khan and Wu does not explicitly teach wherein selecting the group of layers corresponding to the first set of sensitivity values comprises: clustering the sensitivity values based on the evaluated scores into a plurality of groups by: clustering the first set of sensitivity values associated with scores of greater than or equal to a first threshold into a first group; clustering a second set of sensitivity values associated with scores of greater than a second threshold and less than the first threshold into a second group; clustering a third set of sensitivity values associated with scores of greater than a third threshold and less than the second threshold into a third group; and clustering the remaining sensitivity values as the fourth set of sensitivity values of the union sensitivity list into a fourth group.
Barreto, in the same field of endeavor, teaches wherein selecting the group of layers corresponding to the first set of sensitivity values comprises: clustering the sensitivity values based on the evaluated scores into a plurality of groups by: clustering the first set of sensitivity values associated with scores of greater than or equal to a first threshold into a first group; clustering a second set of sensitivity values associated with scores of greater than a second threshold and less than the first threshold into a second group; clustering a third set of sensitivity values associated with scores of greater than a third threshold and less than the second threshold into a third group; and clustering the remaining sensitivity values as the fourth set of sensitivity values of the union sensitivity list into a fourth group.([p. 9] "In this paper we propose a double-threshold method that can be viewed as an alternative to the BOOPI approach. Instead of computing the 100α 2 th and 100(1− α 2)th percentiles for the M bootstrap samples of the quantization errors, we use the well-known statistical box-plot technique3 to determine the interval [ρ−, ρ+] based solely on the original set of quantization errors (e1,...,em). As will be shown in the simulations, the box-plot approach revealed to be one of the more robust approach to novelty detection. [...] In Box Plots ranges or distribution characteristics of values of a selected variable (or variables) are plotted separately for groups of cases defined by values of a categorical (grouping) variable. The central tendency (e.g., median or mean), and range or variation statistics (e.g., quartiles, standard errors, or standard deviations) are computed for each group of cases and the selected values are presented in the selected box plot. Outlier data points can also be plotted." Quantization errors (sensitivity values) are explicitly clustered into quartiles (four groups) with clear thresholds between groups. A boxplot is defined by dictionary.com as “a graphic representation of a distribution by a rectangle, the ends of which mark the maximum and minimum values, and in which the median and first and third quartiles are marked by lines parallel to the ends”).
PNG
media_image1.png
217
588
media_image1.png
Greyscale
Standard box plot
The combination of Khan and Wu as well as Barreto are directed towards neural network quantization. Therefore, the combination of Khan and Wu as well as Barreto are analogous art in the same field of endeavor. It would have been obvious before the effective filing date of the claimed invention to combine the teachings of the combination of Khan and Wu with the teachings of Barreto by creating a box plot of quantization error for sensitivity analysis. While box plots are fundamental statistical analysis tools that would be obvious to apply to any data series, Barreto provides as additional motivation for combination ([p. 9] “we use the well-known statistical box-plot technique3 to determine the interval [ρ−, ρ+] based solely on the original set of quantization errors (e1,...,em). As will be shown in the simulations, the box-plot approach revealed to be one of the more robust approach”). This motivation for combination also applies to the remaining claims which depend on this combination.
Regarding claim 7, the combination of Khan, Wu, and Barreto teaches The method as claimed in claim 1 further comprising:
for each sensitivity value of any of the first group, the second group, the third group,
i. determining each sensitivity value corresponds to at least one of a feature sensitivity value and a weight sensitivity value; (Khan [p. 2] "In a supervised learning task, we want to perform a prediction mapping on a training dataset T , with n training samples. This mapping is from a feature space X to a label space Y, where each sample point is denoted by (xi,yi) ∈ X × Y. Then, in this task, the goal is to learn a classifier, w, that has the minimum empirical loss on the training data for this mapping: L(w;T) = 1 n i=1 (w;xi,yi), (1) where (.;.,.) is the loss function for each sample data.")
ii. upon determining that the sensitivity value corresponds to the weight sensitivity value of a layer, convert a weight and an input of the layer to high precision format; (Khan [p. 2] "these parameters and operations for feed-forward and feed-backward parts are done in full precision of 32 bits, while the parameter vector is heavily sparse. Considering these facts, quantization of these parameters to a lower bit precision seems necessary. Hence, practitioners use a quantization mapping function Q(.; ., .) to map values from a higher bit precision to the lower one")
and iii. upon determining that the sensitivity value corresponds to the feature sensitivity value of a layer, convert parameters of layers from a previous parametric layer to a high precision parametric layer(Khan [p. 2] "these parameters and operations for feed-forward and feed-backward parts are done in full precision of 32 bits, while the parameter vector is heavily sparse. Considering these facts, quantization of these parameters to a lower bit precision seems necessary. Hence, practitioners use a quantization mapping function Q(.; ., .) to map values from a higher bit precision to the lower one").
Regarding claim 8, the combination of Khan, Wu, and Barreto teaches The method as claimed in claim 1, further comprising, for the fourth group of sensitivity values,
determining a layer corresponding to each sensitivity value; evaluating a difference value of bits for a plurality of parametric layers;
(Khan [p. 3 Competition] "a) Competition: Based on the discussion above, the goal is to find the best quantization level for each layer in the network" [p. 3 Competition] "Formally, if at step t > 0 of the quantization, we have the quantized weight set of the previous step as Q (t−1) = n Q (t−1) 1 , . . . , Q (t−1) M o , each of which with possibly different quantization level from N (0) to N (K−1). Then, we randomly choose one layer based on the probability distribution p, say i, that has the bit precision of N (k) , 0 ≤ k < K − 1, and quantize it to the next level of bit precision" Quantized weight set interpreted as synonymous with sensitivity list corresponding to respective quantization error (sensitivity values). The quantization error is obtained through a difference value (see Eqn. 3).)
sorting the layers based on the difference value of bits of each layer; cluster the layers based on the difference values into a plurality of groups;
for each group of layers, sort the layers in a descending order of the corresponding sensitivity values; (Barreto [p. 9] "In this paper we propose a double-threshold method that can be viewed as an alternative to the BOOPI approach. Instead of computing the 100α 2 th and 100(1− α 2)th percentiles for the M bootstrap samples of the quantization errors, we use the well-known statistical box-plot technique3 to determine the interval [ρ−, ρ+] based solely on the original set of quantization errors (e1,...,em). As will be shown in the simulations, the box-plot approach revealed to be one of the more robust approach to novelty detection. [...] In this paper we propose a double-threshold method that can be viewed as an alternative to the BOOPI approach. Instead of computing the 100α 2 th and 100(1− α 2)th percentiles for the M bootstrap samples of the quantization errors, we use the well-known statistical box-plot technique3 to determine the interval [ρ−, ρ+] based solely on the original set of quantization errors (e1,...,em). As will be shown in the simulations, the box-plot approach revealed to be one of the more robust approach to novelty detection." Quantization errors (sensitivity values) are explicitly clustered into quartiles (four groups) with clear thresholds between groups. While the quartiles Q1-Q4 sort the quantization errors in ascending order in Barreto, it would be trivial to simply flip the order into descending and would amount to choosing from a finite number of identified, predictable solutions (two), with a reasonable expectation of success)
and
quantizing a layer of group into high precision format and perform the steps of e-g of claim 1.(Khan [p. 2] "Consider we have a neural network model w ∈ R d , where it contains M layers (e.g., convolutional or fullyconnected) each of which has a number of parameters to be learned. Formally, we denote each layer’s parameter tensor by Wm, m ∈ [M]. For instance, if the mth layer is a convolution layer, Wm is a 4-dimensional tensor. We represent the set of all layers’ parameters as W = {W1, . . . ,WM}, where the concatenation of all these parameters in a vector format is w. In a supervised learning task, we want to perform a prediction mapping on a training dataset T , with n training samples. This mapping is from a feature space X to a label space Y, where each sample point is denoted by (xi, yi) ∈ X × Y. Then, in this task, the goal is to learn a classifier, w, that has the minimum empirical loss on the training data for this mapping" [p. 3] "we need to train KM different networks and choose the best one in terms of the overall accuracy on the validation set" See also Eqn. 1 and 3.).
Regarding claim 10, the combination of Khan, Wu, and Barreto teaches The method as claimed in claim 1, further comprising: h. selecting a group of layers corresponding to a fourth set of sensitivity values; i. generating another mixed precision quantization model by quantizing the selected group of layers into lower precision format;(Khan [p. 3 Competition] "a) Competition: Based on the discussion above, the goal is to find the best quantization level for each layer in the network" [p. 3 Competition] "Formally, if at step t > 0 of the quantization, we have the quantized weight set of the previous step as Q (t−1) = n Q (t−1) 1 , . . . , Q (t−1) M o , each of which with possibly different quantization level from N (0) to N (K−1). Then, we randomly choose one layer based on the probability distribution p, say i, that has the bit precision of N (k) , 0 ≤ k < K − 1, and quantize it to the next level of bit precision" [p. 3] "we will repeat this step for all layers in the network several times" Layer wise quantized weight set interpreted as synonymous with sensitivity list. Khan explicitly teaches selecting a sensitivity list for each respective layer to be quantized and performing the process until each layer has been quantized. Examiner notes that quantized ResNet20_CIFAR, ResNet18, and ResNet50 could also be interpreted as "another mixed precision quantization model".)
j. computing a performance value of the another mixed precision quantization model for comparison with a target performance value;(Khan [p. 2 §IIIB] "the proposed algorithm iteratively alternates between competition and collaboration stages" [p. 5] "In order to mitigate this issue, we introduce an adaptive recovery scheme where we keep retraining until the network reaches an accuracy threshold" [p. 3 §IIIB(b) Collaboration] "we use the second approach, adaptive recovery, in which we set a threshold for accuracy, and continue the fine-tuning until we reach to that predefined threshold. This means that the number of epochs St will be set adaptively in training. We observed that some quantization steps need only one epoch to recover while some others need more than several epochs to fully recover the threshold. For more details on the collaboration part, the reader is referred to the next section" Khan explicitly teaches iteratively alternating between competition and collaboration until a threshold accuracy is reached based on a determined quantization policy. Accuracy for each respective set of quantizers interpreted as performance value. Accuracy of the ResNet18 and ResNet50 models also interpreted as a performance value.)
k. in response to determining that the performance value of the another mixed precision quantization model is less than the target performance value, perform steps i to k; and(Khan [p. 3] "we use the second approach, adaptive recovery, in which we set a threshold for accuracy, and continue the fine-tuning until we reach to that predefined threshold" [p. 5] "we keep retraining until the network reaches an accuracy threshold" [p. 5] "Quantization of all the layers leads to a significant reduction in power requirement as well. We synthesized the RTL module for a MAC (Multiply and Accumulate) unit from the DesignWare Library [32] under 32nm technology node. Fig. 5 depicts the power consumption profile when we compare partially quantized networks having full precision in the first and last layers to our fully quantized ones in mixed precision" See Table II and FIG. 5 for performance of final stored mixed precision models for image processing.)
l. in response to determining that the performance value of the another mixed precision quantization model is greater than or equal to the target performance value, storing the another mixed precision quantization model as the final mixed precision quantization model for image processing.(Khan [p. 3] "we use the second approach, adaptive recovery, in which we set a threshold for accuracy, and continue the fine-tuning until we reach to that predefined threshold" [p. 5] "we keep retraining until the network reaches an accuracy threshold" [p. 5] "Quantization of all the layers leads to a significant reduction in power requirement as well. We synthesized the RTL module for a MAC (Multiply and Accumulate) unit from the DesignWare Library [32] under 32nm technology node. Fig. 5 depicts the power consumption profile when we compare partially quantized networks having full precision in the first and last layers to our fully quantized ones in mixed precision" See Table II and FIG. 5 for performance of final stored mixed precision models for image processing.).
Regarding claim 8, the combination of Khan, Wu, and Barreto teaches The method as claimed in claim 1, further comprising, for the fourth group of sensitivity values, determining a layer corresponding to each sensitivity value; evaluating a difference value of bits for a plurality of parametric layers;
(Khan [p. 3 Competition] "a) Competition: Based on the discussion above, the goal is to find the best quantization level for each layer in the network" [p. 3 Competition] "Formally, if at step t > 0 of the quantization, we have the quantized weight set of the previous step as Q (t−1) = n Q (t−1) 1 , . . . , Q (t−1) M o , each of which with possibly different quantization level from N (0) to N (K−1). Then, we randomly choose one layer based on the probability distribution p, say i, that has the bit precision of N (k) , 0 ≤ k < K − 1, and quantize it to the next level of bit precision" Quantized weight set interpreted as synonymous with sensitivity list corresponding to respective quantization error (sensitivity values). The quantization error is obtained through a difference value (see Eqn. 3).)
sorting the layers based on the difference value of bits of each layer; cluster the layers based on the difference values into a plurality of groups;
for each group of layers, sort the layers in a descending order of the corresponding sensitivity values; (Barreto [p. 9] "In this paper we propose a double-threshold method that can be viewed as an alternative to the BOOPI approach. Instead of computing the 100α 2 th and 100(1− α 2)th percentiles for the M bootstrap samples of the quantization errors, we use the well-known statistical box-plot technique3 to determine the interval [ρ−, ρ+] based solely on the original set of quantization errors (e1,...,em). As will be shown in the simulations, the box-plot approach revealed to be one of the more robust approach to novelty detection. [...] In this paper we propose a double-threshold method that can be viewed as an alternative to the BOOPI approach. Instead of computing the 100α 2 th and 100(1− α 2)th percentiles for the M bootstrap samples of the quantization errors, we use the well-known statistical box-plot technique3 to determine the interval [ρ−, ρ+] based solely on the original set of quantization errors (e1,...,em). As will be shown in the simulations, the box-plot approach revealed to be one of the more robust approach to novelty detection." Quantization errors (sensitivity values) are explicitly clustered into quartiles (four groups) with clear thresholds between groups. While the quartiles Q1-Q4 sort the quantization errors in ascending order in Barreto, it would be trivial to simply flip the order into descending and would amount to choosing from a finite number of identified, predictable solutions (two), with a reasonable expectation of success)
and quantizing a layer of group into high precision format and perform the steps of e-g of claim 1.(Khan [p. 2] "Consider we have a neural network model w ∈ R d , where it contains M layers (e.g., convolutional or fully connected) each of which has a number of parameters to be learned. Formally, we denote each layer’s parameter tensor by Wm, m ∈ [M]. For instance, if the mth layer is a convolution layer, Wm is a 4-dimensional tensor. We represent the set of all layers’ parameters as W = {W1, . . . ,WM}, where the concatenation of all these parameters in a vector format is w. In a supervised learning task, we want to perform a prediction mapping on a training dataset T , with n training samples. This mapping is from a feature space X to a label space Y, where each sample point is denoted by (xi, yi) ∈ X × Y. Then, in this task, the goal is to learn a classifier, w, that has the minimum empirical loss on the training data for this mapping" [p. 3] "we need to train KM different networks and choose the best one in terms of the overall accuracy on the validation set" See also Eqn. 1 and 3.).
Regarding claim 10, the combination of Khan, Wu, and Barreto teaches The method as claimed in claim 1, further comprising: h. selecting a group of layers corresponding to a fourth set of sensitivity values; i. generating another mixed precision quantization model by quantizing the selected group of layers into lower precision format;(Khan [p. 3 Competition] "a) Competition: Based on the discussion above, the goal is to find the best quantization level for each layer in the network" [p. 3 Competition] "Formally, if at step t > 0 of the quantization, we have the quantized weight set of the previous step as Q (t−1) = n Q (t−1) 1 , . . . , Q (t−1) M o , each of which with possibly different quantization level from N (0) to N (K−1). Then, we randomly choose one layer based on the probability distribution p, say i, that has the bit precision of N (k) , 0 ≤ k < K − 1, and quantize it to the next level of bit precision" [p. 3] "we will repeat this step for all layers in the network several times" Layer wise quantized weight set interpreted as synonymous with sensitivity list. Khan explicitly teaches selecting a sensitivity list for each respective layer to be quantized and performing the process until each layer has been quantized. Examiner notes that quantized ResNet20_CIFAR, ResNet18, and ResNet50 could also be interpreted as "another mixed precision quantization model".)
j. computing a performance value of the another mixed precision quantization model for comparison with a target performance value;(Khan [p. 2 §IIIB] "the proposed algorithm iteratively alternates between competition and collaboration stages" [p. 5] "In order to mitigate this issue, we introduce an adaptive recovery scheme where we keep retraining until the network reaches an accuracy threshold" [p. 3 §IIIB(b) Collaboration] "we use the second approach, adaptive recovery, in which we set a threshold for accuracy, and continue the fine-tuning until we reach to that predefined threshold. This means that the number of epochs St will be set adaptively in training. We observed that some quantization steps need only one epoch to recover while some others need more than several epochs to fully recover the threshold. For more details on the collaboration part, the reader is referred to the next section" Khan explicitly teaches iteratively alternating between competition and collaboration until a threshold accuracy is reached based on a determined quantization policy. Accuracy for each respective set of quantizers interpreted as performance value. Accuracy of the ResNet18 and ResNet50 models also interpreted as a performance value.)
k. in response to determining that the performance value of the another mixed precision quantization model is less than the target performance value, perform steps i to k; and(Khan [p. 3] "we use the second approach, adaptive recovery, in which we set a threshold for accuracy, and continue the fine-tuning until we reach to that predefined threshold" [p. 5] "we keep retraining until the network reaches an accuracy threshold" [p. 5] "Quantization of all the layers leads to a significant reduction in power requirement as well. We synthesized the RTL module for a MAC (Multiply and Accumulate) unit from the DesignWare Library [32] under 32nm technology node. Fig. 5 depicts the power consumption profile when we compare partially quantized networks having full precision in the first and last layers to our fully quantized ones in mixed precision" See Table II and FIG. 5 for performance of final stored mixed precision models for image processing.)
l. in response to determining that the performance value of the another mixed precision quantization model is greater than or equal to the target performance value, storing the another mixed precision quantization model as the final mixed precision quantization model for image processing.(Khan [p. 3] "we use the second approach, adaptive recovery, in which we set a threshold for accuracy, and continue the fine-tuning until we reach to that predefined threshold" [p. 5] "we keep retraining until the network reaches an accuracy threshold" [p. 5] "Quantization of all the layers leads to a significant reduction in power requirement as well. We synthesized the RTL module for a MAC (Multiply and Accumulate) unit from the DesignWare Library [32] under 32nm technology node. Fig. 5 depicts the power consumption profile when we compare partially quantized networks having full precision in the first and last layers to our fully quantized ones in mixed precision" See Table II and FIG. 5 for performance of final stored mixed precision models for image processing.).
Regarding claim 14, claim 14 is directed towards a system for performing the method of claim 1. Therefore, the rejection applied to claim 1 also applies to claim 14. Claim 14 also recites additional elements a memory; a processor coupled with memory, that is configured to perform steps of:(Wu [p. 8] "The DNAS search is very efficient, taking less than 5 hours on 8 V100 GPUs to finish the search on ResNet18").
a memory; a processor coupled with memory, that is configured to perform steps of:(Wu [p. 8] "The DNAS search is very efficient, taking less than 5 hours on 8 V100 GPUs to finish the search on ResNet18").
Similarly, regarding claims 20, 21. and 23, claims 20, 21, and 23 are directed towards a system for performing the method of claims 7, 8, and 10. Therefore, the rejections applied to claims 7, 8, and 10 also apply to claims 20, 21, and 23.
Claims 2, 3, 15, and 16 are rejected under U.S.C. §103 as being unpatentable over the combination of Khan, Wu, and Barreto and in further view of Wang (US20190050710A1).
Regarding claim 2, the combination of Khan, Wu, and Barreto teaches The method as claimed in claim 1, wherein generating the union sensitivity list based on sensitivity values evaluated for the plurality of layers comprises evaluating a weight sensitivity value for a parametric layer by: generating a base model by quantizing parameters of the plurality of layers into the high precision format; (Khan [p. 2] "these parameters and operations for feed-forward and feed-backward parts are done in full precision of 32 bits, while the parameter vector is heavily sparse. Considering these facts, quantization of these parameters to a lower bit precision seems necessary. Hence, practitioners use a quantization mapping function Q(.; ., .) to map values from a higher bit precision to the lower one")
generating a first weight evaluation model by quantizing weights of the parametric layer of the base model into low precision format; (Khan [p. 2] "these parameters and operations for feed-forward and feed-backward parts are done in full precision of 32 bits, while the parameter vector is heavily sparse. Considering these facts, quantization of these parameters to a lower bit precision seems necessary. Hence, practitioners use a quantization mapping function Q(.; ., .) to map values from a higher bit precision to the lower one")
and calculating an output of the first weight evaluation model, wherein the low precision format is an 8-bit integer representation of data;(Wu [p. 3 §3] "Quantization projects full-precision weights and activations to fixed-point numbers with lower bit-width, such as 8, 4, and 1 bit")
calculating a first weight sensitivity value based on a difference between the output of the base model and the output of the first weight evaluation model;(Khan [p. 2] "In order to prevent the quantization loss during execution, Jain et al. [25] introduce a new fixed-point number format. The key focus is on keeping the quantization error as minimum as possible […] Hence, practitioners use a quantization mapping function Q(.; ., .) to map values from a higher bit precision to the lower one:" Khan explicitly teaches that the competition stage is based on minimizing a quantization error (the difference between the base model output and output of the quantized (weight evaluation) model))
generating a second weight evaluation model by quantizing weights of all previous layers to the parametric layer, of the base model, to low precision format;(Khan these probabilities by accessing to more information about the layers. Formally, if at step t > 0 of the quantization, we have the quantized weight set of the previous step as Q (t−1) = n Q (t−1) 1 , . . . , Q (t−1) M o , each of which with possibly different quantization level from N (0) to N (K−1). Then, we randomly choose one layer based on the probability distribution p, say i, that has the bit precision of N (k) , 0 ≤ k < K − 1, and quantize it to the next level of bit precision [...] Note that for this set we are calculating the mth layer quantization as Q (t) m = Q 1 Q (t−1) m , N(k+1), α (t) i) using any chosen quantization operator. We run this operation of competition for U steps, and at each step u based on this observation, we will update the weights and probability distribution" [p. 3] "The information in this setting is the accuracy of validation based on the model where all layers’ parameters are kept at the same level of quantization as before, except for one that is quantized to the next level of bit precision. Using the resulted network, we calculate the validation loss. Then, we will repeat this step for all layers in the network several times" Khan explicitly teaches generation of a quantized model for each layer until all layers are quantized into a low precision format.)
calculating a second weight sensitivity value based on a difference between the output of the base model and the output of the second weight evaluation model; and(Khan [p. 2] "In order to prevent the quantization loss during execution, Jain et al. [25] introduce a new fixed-point number format. The key focus is on keeping the quantization error as minimum as possible […] Hence, practitioners use a quantization mapping function Q(.; ., .) to map values from a higher bit precision to the lower one:" Khan explicitly teaches that the competition stage is based on minimizing a quantization error (the difference between the base model output and output of the quantized (weight evaluation) model))
determining a mean of the first weight sensitivity value and the second weight sensitivity value as the weight sensitivity value of the parametric layer.(Khan [p. 2] "In order to prevent the quantization loss during execution, Jain et al. [25] introduce a new fixed-point number format. The key focus is on keeping the quantization error as minimum as possible […] Hence, practitioners use a quantization mapping function Q(.; ., .) to map values from a higher bit precision to the lower one:" Eqn. 3 is a mean squared error term for quantization loss.).
However, the combination of Khan, Wu, and Barreto doesn't explicitly teach and calculating an output of the base model, wherein the parameters comprise an input, a weight and an output of a layer and wherein the high precision format is a 16-bit floating point representation of data;.
Wang, in the same field of endeavor, teaches and calculating an output of the base model, wherein the parameters comprise an input, a weight and an output of a layer and wherein the high precision format is a 16-bit floating point representation of data;([¶0018] "“full-precision” refers to floating-point precision, and may include half-precision (16-bit)").
The combination of Khan, Wu, and Barreto as well as Wang are directed towards quantizing convolutional neural networks. Therefore, the combination of Khan, Wu, and Barreto as well as Wang are analogous art in the same field of endeavor. It would have been obvious before the effective filing date of the claimed invention to combine the teachings of the combination of Khan, Wu, and Barreto with the teachings of Wang by quantizing to 16-bit in a mixed precision quantization system. While one of ordinary skill in the art would recognize that 16-bit is commonly used in computer applications, Wang provides as additional motivation for combination that it can ([¶0004] “produce an acceptable level of model accuracy (e.g., with below a threshold amount of information loss, or with a minimum amount of information loss)”).
Regarding claim 3, the combination of Khan, Wu, Barreto, and Wang teaches The method as claimed in claim 2, wherein generating the union sensitivity list based on sensitivity values evaluated for the plurality of layers further comprises evaluating a feature sensitivity value for the parametric layer by: calculating an output of the parametric layer of the base model;(Khan [p. 2] "Consider we have a neural network model w ∈ R d , where it contains M layers (e.g., convolutional or fully connected) each of which has a number of parameters to be learned. Formally, we denote each layer’s parameter tensor by Wm, m ∈ [M]. For instance, if the mth layer is a convolution layer, Wm is a 4-dimensional tensor. We represent the set of all layers’ parameters as W = {W1, . . . ,WM}, where the concatenation of all these parameters in a vector format is w. In a supervised learning task, we want to perform a prediction mapping on a training dataset T , with n training samples. This mapping is from a feature space X to a label space Y, where each sample point is denoted by (xi, yi) ∈ X × Y. Then, in this task, the goal is to learn a classifier, w, that has the minimum empirical loss on the training data for this mapping" [p. 3] "we need to train KM different networks and choose the best one in terms of the overall accuracy on the validation set" See Eqn. 1. Khan explicitly teaches that the loss is a function of the model output compared to validation output as is known in the art.)
generating a first feature evaluation model by quantizing features and weights of layers, of the base model, from a previous parametric layer to the parametric layer into low precision format and calculating an output of the parametric layer of the first feature evaluation model;(Khan [p. 2] "Consider we have a neural network model w ∈ R d , where it contains M layers (e.g., convolutional or fully connected) each of which has a number of parameters to be learned. Formally, we denote each layer’s parameter tensor by Wm, m ∈ [M]. For instance, if the mth layer is a convolution layer, Wm is a 4-dimensional tensor. We represent the set of all layers’ parameters as W = {W1, . . . ,WM}, where the concatenation of all these parameters in a vector format is w. In a supervised learning task, we want to perform a prediction mapping on a training dataset T , with n training samples. This mapping is from a feature space X to a label space Y, where each sample point is denoted by (xi, yi) ∈ X × Y. Then, in this task, the goal is to learn a classifier, w, that has the minimum empirical loss on the training data for this mapping" [p. 3] "we need to train KM different networks and choose the best one in terms of the overall accuracy on the validation set" See also Eqn. 1 and 3.)
calculating a first feature sensitivity value based on a difference between the output of the base model and the output of the first feature evaluation model;(Khan [p. 2] "Consider we have a neural network model w ∈ R d , where it contains M layers (e.g., convolutional or fully connected) each of which has a number of parameters to be learned. Formally, we denote each layer’s parameter tensor by Wm, m ∈ [M]. For instance, if the mth layer is a convolution layer, Wm is a 4-dimensional tensor. We represent the set of all layers’ parameters as W = {W1, . . . ,WM}, where the concatenation of all these parameters in a vector format is w. In a supervised learning task, we want to perform a prediction mapping on a training dataset T , with n training samples. This mapping is from a feature space X to a label space Y, where each sample point is denoted by (xi, yi) ∈ X × Y. Then, in this task, the goal is to learn a classifier, w, that has the minimum empirical loss on the training data for this mapping" [p. 3] "we need to train KM different networks and choose the best one in terms of the overall accuracy on the validation set" See also Eqn. 1 and 3.)
generating a second feature evaluation model by quantizing weights and features of all previous layers till the parametric layer, of the base model, to low precision format;(Khan [p. 2] "We represent the set of all layers’ parameters as W = {W1, . . . ,WM}" [p. 3] " all layers’ parameters are kept at the same level of quantization as before, except for one that is quantized to the next level of bit precision. Using the resulted network, we calculate the validation loss. Then, we will repeat this step for all layers in the network several times")
calculating a second feature sensitivity value based on a difference between the output of the base model and the output of the second feature evaluation model; and(Khan [p. 2] "Consider we have a neural network model w ∈ R d , where it contains M layers (e.g., convolutional or fully connected) each of which has a number of parameters to be learned. Formally, we denote each layer’s parameter tensor by Wm, m ∈ [M]. For instance, if the mth layer is a convolution layer, Wm is a 4-dimensional tensor. We represent the set of all layers’ parameters as W = {W1, . . . ,WM}, where the concatenation of all these parameters in a vector format is w. In a supervised learning task, we want to perform a prediction mapping on a training dataset T , with n training samples. This mapping is from a feature space X to a label space Y, where each sample point is denoted by (xi, yi) ∈ X × Y. Then, in this task, the goal is to learn a classifier, w, that has the minimum empirical loss on the training data for this mapping" [p. 3] "we need to train KM different networks and choose the best one in terms of the overall accuracy on the validation set" See also Eqn. 1 and 3.)
determining a mean of the first feature sensitivity value and the second feature sensitivity value as the feature sensitivity value of the parametric layer.(Khan [p. 2] "In order to prevent the quantization loss during execution, Jain et al. [25] introduce a new fixed-point number format. The key focus is on keeping the quantization error as minimum as possible […] Hence, practitioners use a quantization mapping function Q(.; ., .) to map values from a higher bit precision to the lower one:" Eqn. 3 is a mean squared error term for quantization loss.).
Regarding claims 15 and 16, claims 15 and 16 are directed towards a system for performing the method of claims 2 and 3, respectively. Therefore, the rejections applied to claims 2 and 3 also apply to claims 15 and 16.
Claims 4, 5, 17, and 18 are rejected under U.S.C. §103 as being unpatentable over the combination of Khan, Wu, Barreto, Wang, and in further view of Sather (US20240193426A1).
Regarding claim 4, the combination of Khan, Wu, Barreto, and Wang teaches The method as claimed in claim 3.
However, the combination of Khan, Wu, Barreto, and Wang doesn't explicitly teach wherein generating the union sensitivity list based on the sensitivity values evaluated for the plurality of layers further comprising: normalizing feature sensitivity values and weight sensitivity values corresponding to each parametric layer among the plurality of layers;
evaluating the score for each normalized value; and
generating the union sensitivity list of the scores evaluated for the normalized values.
Sather, in the same field of endeavor, teaches generating the union sensitivity list based on the sensitivity values evaluated for the plurality of layers further comprising: normalizing feature sensitivity values and weight sensitivity values corresponding to each parametric layer among the plurality of layers;([¶0004] "For quantized activation functions, some embodiments use a lookup table (LUT) to implement the activation function (because only a particular number of output activation values are allowed) [...] embodiments that do use batch normalization (e.g., for all fanout layers), output scaling is not used as the output is invariant under rescaling of all inputs by a same factor.")
evaluating the score for each normalized value; and([¶0004] "the output is invariant under rescaling of all inputs by a same factor." all outputs for all inputs interpreted as each normalized value)
generating the union sensitivity list of the scores evaluated for the normalized values;([¶0005] "to ensure that the activation values fall in the available range of quantized activations" The available range of quantized activations is interpreted as the union sensitivity list of the scores evaluated for normalized values which is in line with the interpretation of Khan.).
The combination of Khan, Wu, Barreto, and Wang as well as Sather are directed towards quantizing convolutional neural networks. Therefore, the combination of Khan, Wu, Barreto, and Wang as well as Sather are analogous art in the same field of endeavor. It would have been obvious before the effective filing date of the claimed invention to combine the teachings of the combination of Khan, Wu, Barreto, and Wang with the teachings of Sather by using batch normalization in the quantized convolutional neural network. Sather provides as additional motivation for combination ([¶0021] “the learned affine transformations will be able to update the initial scales in order to improve accuracy and adapt to value and weight quantization.”).
Regarding claim 5, the combination of Khan, Wu, Barreto, Wang, and Sather teaches The method as claimed in claim 4, wherein evaluating the score for each normalized value comprising evaluating a Z-score for each normalized value.(Sather [¶0066] "Given the analytic form of both the mean y(x) and variance s(x)−y2(x) random values from this distribution are sampled during training and used to learn the temperature of the noise, T. For example, given an input x in the interval [xi, xi+1] a sample cane be generated: [See Eqn. 42] where z is a z-score drawn from a standard normal distribution").
Regarding claims 17 and 18, claims 17 and 18 are directed towards a system for performing the method of claims 4 and 5, respectively. Therefore, the rejections applied to claims 4 and 5 also apply to claims 17 and 18.
Claims 9 and 22 are rejected under U.S.C. §103 as being unpatentable over the combination of Khan and Wu and Barreto and Defossez (“Differentiable Model Compression via Pseudo Quantization Noise”, 2021).
Regarding claim 9, the combination of Khan, Wu, and Barreto teaches The method as claimed in claim 8, wherein evaluating the difference value of bits for the plurality of parametric layers comprising: evaluating a number of bits for each channel of the parametric layer; (Khan [p. 3 Competition] "a) Competition: Based on the discussion above, the goal is to find the best quantization level for each layer in the network" [p. 3 Competition] "Formally, if at step t > 0 of the quantization, we have the quantized weight set of the previous step as Q (t−1) = n Q (t−1) 1 , . . . , Q (t−1) M o , each of which with possibly different quantization level from N (0) to N (K−1). Then, we randomly choose one layer based on the probability distribution p, say i, that has the bit precision of N (k) , 0 ≤ k < K − 1, and quantize it to the next level of bit precision" Quantized weight set interpreted as synonymous with sensitivity list corresponding to respective quantization error (sensitivity values). The quantization error is obtained through a difference value (see Eqn. 3).).
However, the combination of Khan, Wu, and Barreto doesn't explicitly teach and
determining a difference between a maximum number of bits and a minimum number of bits for the parametric layer and storing the difference as the difference value of the layer.
Defossez, in the same field of endeavor, teaches and determining a difference between a maximum number of bits and a minimum number of bits for the parametric layer and storing the difference as the difference value of the layer.([p. 4] "Parametrization. In practice, the number of bits used for each group b ∈ Rg ∗+ is obtained from a logit parameter l ∈ Rg, so that we have b =bmin +σ(l)(bmax −bmin), with σ is the sigmoid function, and bmin and bmax the minimal and maximal number of bits to use. The trainable parameter l is initialized so that b = binit. We set binit = 8").
The combination of Khan, Wu, and Barreto as well as Defossez are directed towards neural network quantization. Therefore, the combination of Khan, Wu, and Barreto as well as Defossez are analogous art in the same field of endeavor. It would have been obvious before the effective filing date of the claimed invention to combine the teachings of the combination of Khan, Wu, and Barreto with the teachings of Defossez by using and storing a difference of bits. Defossez provides as additional motivation for combination ([p. 7] “We presented DiffQ, a novel and simple differentiable method for model quantization […] Results suggest that DiffQ is superior to the baseline methods on several benchmarks from various domains”). This motivation for combination also applies to the remaining claims which depend on this combination.
Regarding claim 22, claim 22 is directed towards a system for performing the method of claim 9. Therefore, the rejection applied to claim 9 also applies to claim 22.
Claims 11, 12, 24, and 25 are rejected under U.S.C. §103 as being unpatentable over the combination of Khan, Wu, Barreto, and in further view of Chen (US20210182077A1).
Regarding claim 11, the combination of Khan, Wu, and Barreto teaches The method as claimed in claim 10, wherein selecting the group of layers corresponding to the fourth set of sensitivity values comprises clustering the sensitivity values based on the evaluated scores into another plurality of groups by:(Wu [p. 6 6.1] "We conduct mixed precision search at the block level – all layers in one block use the same precision" [p. 6 §6.1] "For each block, we can choose a precision from {0, 1, 2, 3, 4, 8, 32}. If the precision is 0, we simply skip this block so the input and output are identical. If the precision is 32, we use the full-precision floating point weights. For all other precisions with k-bit, we quantize weights to k-bit fixed-point numbers. See Appendix B for more experiment details").
However, the combination of Khan, Wu, and Barreto doesn't explicitly teach clustering a fifth set of sensitivity values associated with scores of less than or equal to a fourth threshold into a fifth group, wherein the fourth threshold is a negative value of the first threshold; clustering a sixth set of sensitivity values associated with scores of less than a fifth threshold and greater than the fourth threshold into a sixth group, wherein the fifth threshold is a negative value of the second threshold; clustering a seventh set of sensitivity values associated with scores of less than a sixth threshold and greater than the fifth threshold into a seventh group, wherein the sixth threshold is a negative value of the third threshold; and clustering the remaining sensitivity values as an eighth set of sensitivity values of the union sensitivity list into an eighth group.
Chen, in the same field of endeavor, teaches clustering a fifth set of sensitivity values associated with scores of less than or equal to a fourth threshold into a fifth group, wherein the fourth threshold is a negative value of the first threshold; clustering a sixth set of sensitivity values associated with scores of less than a fifth threshold and greater than the fourth threshold into a sixth group, wherein the fifth threshold is a negative value of the second threshold; clustering a seventh set of sensitivity values associated with scores of less than a sixth threshold and greater than the fifth threshold into a seventh group, wherein the sixth threshold is a negative value of the third threshold; and clustering the remaining sensitivity values as an eighth set of sensitivity values of the union sensitivity list into an eighth group.([¶0590] "The coarse-grained pruning unit is configured to: select M weights from the weights of the neural network through a sliding window, where M is an integer greater than 1, and when the M weights satisfy a preset condition, set all or part of the M weights to zero." [¶0592] "the preset condition is: the amount of information of the M weights being less than a first preset threshold." [¶0593] "The first preset threshold is a first threshold, a second threshold, or a third threshold. The amount of information of the M weights being less than the first preset threshold includes: the arithmetic mean of the absolute values of the M weights being less than the first threshold, or the geometric mean of the absolute values of the M weights being less than the second threshold, or the maximum value of the M weights being less than the third threshold" [¶0744] "The present disclosure provides a data quantization method including: grouping weights of a neural network; using a clustering algorithm to cluster each group of weights, dividing a group of weights into m clusters, computing a central weight for each cluster, and replacing weights in each cluster with the central weight, where m is a positive integer" [¶1194] "quantizing the weights of the neural network, which includes: grouping the weights of the neural network, using a clustering algorithm to cluster each group of weights, computing a central weight for each cluster, and replacing weights in each cluster with the corresponding central weight of the cluster; encoding the central weights to obtain a codebook and a weight dictionary; and performing a second retraining on the neural network, where only the codebook is trained during the retraining, and the content of the weight dictionary remains unchanged.").
The combination of Khan, Wu, and Barreto as well as Chen are directed towards quantization of convolutional neural networks. Therefore, the combination of Khan, Wu, and Barreto as well as Chen are reasonably pertinent analogous art. It would have been obvious before the effective filing date of the claimed invention to combine the teachings of the combination of Khan, Wu, and Barreto with the teachings of Chen by clustering weight sensitivity values for quantization. Chen provides as additional motivation for combination ([¶3195] "By fully exploiting the characteristics of weight distribution of the neural network, low-bit quantized weights are obtained, which may greatly improve the processing speed and reduce the weight storage overhead and memory access overhead."). This motivation for combination also applies to the remaining claims which depend on this combination.
Regarding claim 12, the combination of Khan, Wu, Barreto, and Chen teaches The method as claimed in claim 11, wherein further comprising: for each sensitivity value of any of the fifth group, the sixth group and the seventh group, i. determining each sensitivity value corresponds to at least one of a feature sensitivity value and a weight sensitivity value;(Khan [p. 3 Competition] "a) Competition: Based on the discussion above, the goal is to find the best quantization level for each layer in the network" [p. 3 Competition] "Formally, if at step t > 0 of the quantization, we have the quantized weight set of the previous step as Q (t−1) = n Q (t−1) 1 , . . . , Q (t−1) M o , each of which with possibly different quantization level from N (0) to N (K−1). Then, we randomly choose one layer based on the probability distribution p, say i, that has the bit precision of N (k) , 0 ≤ k < K − 1, and quantize it to the next level of bit precision" Quantized weight set interpreted as synonymous with sensitivity list.)
ii. upon determining that the sensitivity value corresponds to the weight sensitivity value of a layer, convert a weight and an input of the layer to the lower precision format; and(Khan [p. 3 Competition] "a) Competition: Based on the discussion above, the goal is to find the best quantization level for each layer in the network" [p. 3 Competition] "Formally, if at step t > 0 of the quantization, we have the quantized weight set of the previous step as Q (t−1) = n Q (t−1) 1 , . . . , Q (t−1) M o , each of which with possibly different quantization level from N (0) to N (K−1). Then, we randomly choose one layer based on the probability distribution p, say i, that has the bit precision of N (k) , 0 ≤ k < K − 1, and quantize it to the next level of bit precision" Quantized weight set interpreted as synonymous with sensitivity list of weight sensitivity values of a layer.)
iii. upon determining that the sensitivity value corresponds to the feature sensitivity value of a layer, convert parameters of layers from a previous parametric layer to the parametric layer to the lower precision format.(Khan [p. 3] "p (u) m is the probability of choosing the mth expert (i.e. layer). Also, π (u) i is the weight of each layer calculated based on the validation loss" Pi is interpreted as a feature sensitivity value calculated as a result of the loss function. See also Algorithm 1 which shows that quantization proceeds upon determining said sensitivity value.).
Regarding claims 24 and 25, claims 24 and 25 are directed towards a system for performing the method of claims 11 and 12, respectively. Therefore, the rejections applied to claims 11 and 12 also apply to claims 24 and 25.
Claim 13 is rejected under U.S.C. §103 as being unpatentable over the combination of Khan and Wu and Barreto and Chen and Lee (US11625577B2).
Regarding claim 13, the combination of Khan, Wu, Barreto, and Chen teaches The method as claimed in claim 11, further comprising, for the eighth group, determining a layer corresponding to each sensitivity value;(Khan [p. 3 Competition] "a) Competition: Based on the discussion above, the goal is to find the best quantization level for each layer in the network" [p. 3 Competition] "Formally, if at step t > 0 of the quantization, we have the quantized weight set of the previous step as Q (t−1) = n Q (t−1) 1 , . . . , Q (t−1) M o , each of which with possibly different quantization level from N (0) to N (K−1). Then, we randomly choose one layer based on the probability distribution p, say i, that has the bit precision of N (k) , 0 ≤ k < K − 1, and quantize it to the next level of bit precision" [p. 3] "p (u) m is the probability of choosing the mth expert (i.e. layer). Also, π (u) i is the weight of each layer calculated based on the validation loss" Khan explicitly teaches that each sensitivity value corresponds to a layer.)
quantizing a layer of group into lower precision format and perform the steps of j to l of claim 10.(Khan [p. 2 §IIIB] "the proposed algorithm iteratively alternates between competition and collaboration stages" [p. 5] "In order to mitigate this issue, we introduce an adaptive recovery scheme where we keep retraining until the network reaches an accuracy threshold" [p. 3 §IIIB(b) Collaboration] "we use the second approach, adaptive recovery, in which we set a threshold for accuracy, and continue the fine-tuning until we reach to that predefined threshold. This means that the number of epochs St will be set adaptively in training. We observed that some quantization steps need only one epoch to recover while some others need more than several epochs to fully recover the threshold. For more details on the collaboration part, the reader is referred to the next section" Khan explicitly teaches iteratively alternating between competition and collaboration until a threshold accuracy is reached based on a determined quantization policy. Accuracy for each respective set of quantizers interpreted as performance value. Accuracy of the ResNet18 and ResNet50 models also interpreted as a performance value.).
However, the combination of Khan, Wu, Barreto, and Chen doesn't explicitly teach sorting the layers based on the difference value of bits of each layer; cluster the layers based on the difference value of bits into a plurality of groups; for each group of layers, sort the layers in an ascending order of the corresponding sensitivity values; and.
Lee, in the same field of endeavor, teaches sorting the layers based on the difference value of bits of each layer; cluster the layers based on the difference value of bits into a plurality of groups; for each group of layers, sort the layers in an ascending order of the corresponding sensitivity values; and([Col. 1 l. 52-65] "The statistic may include a mean square of weight differences for the each of the layers. The method may include sorting the layers in order of a size of the analyzed statistic, wherein the determining of the one or more layers to be quantized may include identifying layers having a relatively small analyzed statistic size from among the sorted layers. [...] The determining of the one or more layers to be quantized may include determining a number of layers from among the sorted layers to be the one or more layers in ascending order of the size of the analyzed statistic." Quantization is interpreted as being based on a difference value of bits).
The combination of Khan, Wu, Barreto, and Chen as well as Lee are directed towards quantizing convolutional neural networks. Therefore, the combination of Khan, Wu, Barreto, and Chen as well as Lee are analogous art in the same field of endeavor. It would have been obvious before the effective filing date of the claimed invention to combine the teachings of the combination of Khan, Wu, Barreto, and Chen with the teachings of Lee by clustering the layers and sorting them in ascending order as part of the quantization process. Lee provides as additional motivation for combination ("a weight range for each layer is sorted in ascending order, and some layers having a small weight range are quantized with a lower-bit (4-bit) precision. However, as illustrated in FIG. 17 , it may be seen that accuracy loss of the 1701 when using a weight range is larger when compared with the other cases 1702 and 1703. This is because, as a weight range decreases, a weight value to be expressed decreases and thus expression with a lower-bit precision is possible, but even when the weight range is small, when the maximum value of the weight is large, an integer bit to express the weight increases accordingly.").
Allowable Subject Matter
Claims 26-27 allowed.
Below are the closest cited references, each of which disclose various aspects of the claimed invention:
Wu ("MIXED PRECISION QUANTIZATION OF CONVNETS VIA DIFFERENTIABLE NEURAL ARCHITECTURE SEARCH", 2018)
Khan ("Learning to Quantize Deep Neural Networks: A Competitive-Collaborative Approach", 2020)
Barreto (“A Unifying Methodology for the Evaluation of Neural Network Models on Novelty Detection Tasks”, 2013)
Sather (US20240193426A1)
Chen (US20210182077A1)
However, none of the prior art references of record, alone or in combination, disclose or suggest the combined features recited in the independent claims, including specifically (for claim 1):
selecting the group of layers corresponding to the first set of sensitivity values comprises: clustering the sensitivity values based on the evaluated scores into a plurality of groups by: clustering the first set of sensitivity values associated with scores of greater than or equal to a first threshold into a first group, wherein the first threshold is three times of a standard deviation of the scores; clustering a second set of sensitivity values associated with scores of greater than a second threshold and less than the first threshold into a second group, wherein the second threshold is 2.5 times of the standard deviation; clustering a third set of sensitivity values associated with scores of greater than a third threshold and less than the second threshold into a third group, wherein the third threshold is two times of the standard deviation; and clustering the remaining sensitivity values as the fourth set of sensitivity values of the union sensitivity list into a fourth group.
While the primary reference Khan teaches a quantization aware training method which evaluates accuracy of a quantized model to conditionally iteratively quantize and train said model, Khan does not explicitly teach "wherein selecting the group of layers corresponding to the first set of sensitivity values comprises: clustering the sensitivity values based on the evaluated scores into a plurality of groups by: clustering the first set of sensitivity values associated with scores of greater than or equal to a first threshold into a first group, wherein the first threshold is three times of a standard deviation of the scores; clustering a second set of sensitivity values associated with scores of greater than a second threshold and less than the first threshold into a second group, wherein the second threshold is 2.5 times of the standard deviation; clustering a third set of sensitivity values associated with scores of greater than a third threshold and less than the second threshold into a third group, wherein the third threshold is two times of the standard deviation; and clustering the remaining sensitivity values as the fourth set of sensitivity values of the union sensitivity list into a fourth group.". While Chen in the same field of endeavor teaches clustering layers for pruning and quantization, Chen does not teach that the clustering of sensitivity values is performed based on a range of the standard deviation. Neither Wu or Sather remedy this deficiency nor would it be obvious to arrive at the claim limitation before the effective filing date of the claimed invention.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Dong (“HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks”, 2020) is directed towards a sensitivity based neural network quantization method.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SIDNEY VINCENT BOSTWICK whose telephone number is (571)272-4720. The examiner can normally be reached M-F 7:30am-5:00pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Miranda Huang can be reached on (571)270-7092. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SIDNEY VINCENT BOSTWICK/Examiner, Art Unit 2124