DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-20 are presented for examination.
Response to Amendment
The prior objections to the claims, specification, and drawings have been obviated by the amendment. Therefore, these objections are withdrawn. The prior 112(b) rejections have also been obviated by the amendment. Therefore, these rejections are withdrawn.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an
abstract idea without significantly more. The analysis of the claims will follow the 2019 Revised Patent Subject Matter Eligibility Guidance (“2019 PEG”).
Claim 1
Step 1: The claim is directed to a method, and is therefore directed to the statutory category of processes.
Step 2A Prong 1: The claim recites:
• calculating an activation quantization sensitivity (AQS) value for each of a plurality of
convolution layers in a neural network, wherein the AQS value indicates sensitivity of
convolution output to quantized convolution input, and wherein the neural network includes a plurality of operation (OP) layers, the OP layers including at least the plurality of convolution layers; This limitation recites a mathematical calculation of calculating a sensitivity value of convolution output to quantized convolution input for each of a plurality of convolution layers in a neural network.
• forming a plurality of quantization groups by grouping one or more convolution layers into a quantization group to be executed by a corresponding set of target hardware, and wherein forming the plurality of quantization groups further comprises grouping the OP layers whose respective input ends are connected to receive a same input; This
limitation encompasses mentally sorting convolutional layers into a plurality of quantization
groups, and mentally grouping OP layers whose respective input ends are connected to receive a same input.
• calculating a group AQS value for each quantization group based on AQS values of the
convolution layers in the quantization group; This limitation recites a mathematical calculation of calculating a group AQS value for each quantization group based on AQS values of the convolution layers in the quantization group.
• selecting bit-widths supported by the target hardware platform for corresponding
quantization groups to optimize, under a given constraint, a sensitivity metric that is
calculated based on each quantization group's group AQS value; This limitation encompasses
mentally selecting a bit-width for each quantization group that is supported by the target hardware
platform and optimizes a sensitivity metric calculated based on each quantization group’s group
AQS value.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The claim further
recites that the method for determining bit-widths is “for mixed-precision neural network computing on a
target hardware platform.” However, this limitation amounts to merely indicating a field of use in which
to apply a judicial exception (MPEP 2106.05(h)).
Step 2B: The claim does not contain significantly more than the judicial exception. The “for mixed-
precision neural network computing on a target hardware platform” limitation amounts to merely
indicating a field of use in which to apply a judicial exception as stated above (MPEP 2106.05(h)). As an
ordered whole, the claim is directed to the abstract idea of performing mathematical calculations to determine optimal bit-widths for groups of layers in a neural network model. Nothing in the claim provides significantly more than this. As such, the claim is not patent eligible.
Claim 2
Step 1: A process, as above.
Step 2A Prong 1: The claim recites:
• performing Integer Linear Programming (ILP) to obtain a mixed-precision quantization
configuration for the neural network that minimizes a total AQS under the given
constraint, wherein the total AQS is a sum of all group AQS values in the neural network; This limitation recites the mathematical concept of performing Integer Linear Programming.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. See analysis of
claim 1.
Step 2B: The claim does not contain significantly more than the judicial exception. See analysis of claim
1.
Claim 3
Step 1: A process, as above.
Step 2A Prong 1: The claim recites:
• wherein the given constraint is one of model size, latency, and total binary operations of
the neural network; This limitation merely further limits the constraint used in performing
Integer Linear Programming, which is a mathematical concept.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. See analysis of
claim 1.
Step 2B: The claim does not contain significantly more than the judicial exception. See analysis of claim
1.
Claim 4
Step 1: A process, as above.
Step 2A Prong 1: The claim recites:
• wherein the given constraint is total binary operations (BOPS), which limits a sum of per-
group BOPS, and wherein per-group BOPS is calculated as a number of MAC operations
multiplied by a bit-width of input activation and a bit-width of weights; This limitation
merely further limits the constraint used in selecting bit-widths for quantization groups. Selecting
bit-widths to optimize a sensitivity metric under a given constraint is still mentally performable
using this constraint.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. See analysis of
claim 1.
Step 2B: The claim does not contain significantly more than the judicial exception. See analysis of claim
1.
Claim 5
Step 1: A process, as above.
Step 2A Prong 1: The claim recites:
• selecting an input activation bit-width and a weight bit-width for each quantization group
to optimize the sensitivity metric under the given constraint; This limitation encompasses
mentally selecting an input activation bit-width and a weight bit-width for each quantization
group that optimizes a sensitivity metric under a given constraint.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. See analysis of
claim 1.
Step 2B: The claim does not contain significantly more than the judicial exception. See analysis of claim
1.
Claim 6
Step 1: A process, as above.
Step 2A Prong 1: The claim recites:
• identifying one or more quantization groups to lower corresponding bit-widths under a
user-defined accuracy criterion; This limitation encompasses mentally identifying quantization
groups to lower corresponding bit-widths based on an accuracy criterion defined by a user.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. See analysis of
claim 1.
Step 2B: The claim does not contain significantly more than the judicial exception. See analysis of claim
1.
Claim 7
Step 1: A process, as above.
Step 2A Prong 1: The claim recites:
• wherein an identified quantization group is one having a highest group quality log (MAC+1) /group AQS among all quantization groups, wherein MAC represents a number of multiply-and-add operations; This limitation merely further limits the “identifying” limitation of claim 6. The “identifying” limitation is still mentally performable, as one can mentally identify a quantization group having a highest group quality.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. See analysis of
claim 1.
Step 2B: The claim does not contain significantly more than the judicial exception. See analysis of claim
1.
Claim 8
Step 1: A process, as above.
Step 2A Prong 1: The claim recites:
• wherein the group AQS value is a maximum AQS value of all convolution layers in the
quantization group; This limitation merely further limits the calculating a group AQS value
limitation of claim 1, which still constitutes a mathematical calculation. Additionally, finding a
maximum value is mentally performable.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. See analysis of
claim 1.
Step 2B: The claim does not contain significantly more than the judicial exception. See analysis of claim
1.
Claim 9
Step 1: A process, as above.
Step 2A Prong 1: The claim recites:
• calculating a first output of a convolution layer using input activation and weights at step t; This limitation recites a mathematical calculation.
• calculating a second output of the convolution layer using the input activation and
weights at step (t-1); This limitation recites a mathematical calculation.
• measuring a difference between the first output and the second output to compute the AQS value of the convolution layer; This limitation recites the mathematical concept of measuring a difference between two values.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. See analysis of
claim 1.
Step 2B: The claim does not contain significantly more than the judicial exception. See analysis of claim
1.
Claim 10
Step 1: A process, as above.
Step 2A Prong 1: The claim recites:
• calculating a first output of a convolution layer using input activation and weights at a first data precision; This limitation recites a mathematical calculation
• calculating a second output of the convolution layer using the input activation and the
weights at a second data precision, the second data precision using a larger bit-width than
the first data precision; This limitation recites a mathematical calculation
• measuring a difference between the first output and the second output to compute the AQS value of the convolution layer; This limitation recites the mathematical concept of measuring a difference between two values
Step 2A Prong 2: This judicial exception is not integrated into a practical application. See analysis of
claim 1.
Step 2B: The claim does not contain significantly more than the judicial exception. See analysis of claim
1.
Claim 11
Step 1: A process, as above.
Step 2A Prong 1: The claim recites:
• wherein grouping the one or more convolution layers into the quantization group further
comprises: grouping input activation and weights of a convolution layer into a first quantization group, the input activation and the weights having a first data precision; and grouping an output of the convolution layer into a second quantization group when the target hardware platform supports different data precisions for the output and the input activation; This limitation merely further limits the grouping limitation of claim 1. The grouping limitation is still mentally performable as one can mentally group input activation and weights of a convolution layer into a first quantization group having a first data precision and mentally group an output of the convolution layer into a second quantization group when the target hardware platform supports different data precisions for the output and the input activation.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. See analysis of
claim 1.
Step 2B: The claim does not contain significantly more than the judicial exception. See analysis of claim
1.
Claims 12-20
Step 1: The claims recite a system, and are therefore directed to the statutory category of machines.
Step 2A Prong 1: Claims 12-20 recite the same judicial exception as claims 1-2 and 4-10, respectively.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The claims further
recite “memory to store a neural network that includes a plurality of operation (OP) layers, the OP layers including at least a plurality of convolution layers” and “processing circuitry coupled to the memory and operative to: [perform the method].” However, these limitations amount to mere instructions to apply the judicial exception on a generic computer (MPEP 2106.05(f)).
Step 2B: The claims do not contain significantly more than the judicial exception. The memory and
processing circuitry limitations amount to mere instructions to apply the judicial exception on a generic
computer (MPEP 2106.05(f)) as stated above. As an ordered whole, the claims are directed to the abstract
idea of performing mathematical calculations to determine optimal bit-widths for a neural network model.
Nothing in the claims provide significantly more than this. As such, the claims are not patent eligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-5, 9-10, 12-15 and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang (CN113962365A) in view of Fan et al. (HFPQ: deep neural network compression by hardware-friendly pruning-quantization) (“Fan”), further in view of Liu et al. (Group Fisher Pruning for Practical Network Compression) (“Liu”), further in view of Yao et al. (HAWQ-V3: Dyadic Neural Network Quantization) (“Yao”).
Regarding claim 1, Zhang discloses “A method for determining bit-widths for mixed-precision neural network computing on a target hardware platform, comprising:
calculating an activation quantization sensitivity (AQS) value for each of a plurality of… layers in a neural network, wherein the AQS value indicates sensitivity of… output to quantized… input (Zhang, [n0011]: “determining the quantization sensitivity parameter corresponding to each network layer in the first i blocks under the quantization configuration, wherein the quantization sensitivity parameter corresponding to a network layer is used to characterize the difference between the output result before quantization and the output result after quantization of the network layer”; the examiner notes that “quantization of the network layer” corresponds to quantized input, see [n0114]: “perform quantization processing on the input image, weight parameters and output image of each network layer”);
forming a plurality of quantization groups by grouping one or more… layers into a quantization group (Zhang, [n0007]: “The neural network includes multiple blocks, each block including at least one network layer. The method includes: acquiring a target quantization configuration set for a group of blocks in the neural network, the group of blocks including at least one block; filtering a candidate quantization configuration set for another group of blocks in the neural network based on the acquired target quantization configuration set for the group of blocks to obtain a target quantization configuration set for the other group of blocks”; the examiner notes that the groups of blocks correspond to “a plurality of quantization groups”) to be executed by a corresponding set of target hardware (Zhang, [n0004]: “One feasible approach to deploying neural networks in resource-constrained scenarios is to quantize the full-precision parameters in the neural network into lower precision, so as to use less bit width to store the parameters, thereby reducing memory requirements. Hardware devices can support quantization methods with various bit widths”);
calculating a group AQS value for… [a] quantization group based on AQS values of the… layers in the quantization group…” (Zhang, [n0012]: “summing the quantization sensitivity parameters of each network layer in the first i blocks under the quantization configuration to obtain the estimated value of the quantization sensitivity parameter of the first i blocks under the quantization configuration”; the examiner notes that the “first i blocks” correspond to a quantization group and “the quantization sensitivity parameter of the first i blocks” corresponds to a group AQS value).
Zhang does not appear to explicitly disclose calculating a group AQS value for each quantization group, that the layers are convolution layers, or that the neural network includes a plurality of operation (OP) layers, the OP layers including at least the plurality of convolution layers.
However, Fan discloses “wherein… [a] neural network includes a plurality of operation (OP) layers, the OP layers including at least… [a] plurality of convolution layers” (Fan, Fig. 3: the examiner notes VGG16 is a neural network containing 13 convolutional layers, which perform the operation of convolution) and “calculating a group… [sensitivity] value for each… group [of convolution layers]” (Fan, Fig.3: Sensitivity analysis of VGG16. The 13 convolutional layers of VGG16 are divided into 5 large blocks. Each block contains two or three convolutional layers. The sensitivity of different blocks or convolutional layers is indicated by the color shade. The darker the color, the larger is the sensitivity. The five blocks in this figure are sorted by sensitivity to Conv2B > Conv3B > Conv1B > Conv5B > Conv4B. For block 3, the sensitivity of the convolutional layer is in the order of Conv5 > Conv6 > Conv7).
Fan and the instant application both relate to neural network compression and are analogous. It would have been obvious to one of ordinary skill in the art, prior to the effective filing date of the claimed invention, to have modified the neural network layers of Zhang to be convolution layers, to have modified the neural network to include a plurality of operation (OP) layers, the OP layers including at least the plurality of convolution layers, and to have modified the calculating a group AQS value step of Zhang to be performed for each quantization group, as disclosed by Fan, and one would have been motivated to do so for the purpose of minimizing accuracy loss of a deep convolution neural network after compression, and to make the neural network better implemented in hardware (see Fan, section 3, paragraphs 1-2).
Neither Zhang nor Fan appear to explicitly disclose “wherein forming the plurality of quantization groups further comprises grouping the OP layers whose respective input ends are connected to receive a same input.”
However, Liu discloses “grouping…OP layers whose respective input ends are connected to receive a same input” (Liu, 3.2. Prune Coupled Channels: “Then given parents of each layer, we can assign layers to different groups where layers in one group have coupled channels to be pruned simultaneously. It contains the following situations: (1) layers which have the same parents should be assigned to one group because their input channels (or equivalently y, output channels of their parents) are coupled and should be pruned together as Fig.3; Examiner notes that Fig.3 (b) shows conv2 and conv5, having connected input ends to receive a same input (both have conv1 as a parent), and thus are assigned to one group).
Liu and the instant application both relate to neural network compression and are analogous. It would have been obvious to one of ordinary skill in the art, prior to the effective filing date of the claimed invention to have modified the combination of Zhang/Fan with the teachings of Liu such that forming the plurality of quantization groups further comprises grouping the OP layers whose respective input ends are connected to receive a same input, and one would have been motivated to do so for the purpose of having the ability to quantize any layers including those with coupled inputs, thus achieving better trade-off between accuracy drop and actual acceleration (see Liu, I. Introduction, Paragraph 5).
Neither Zhang nor Fan nor Liu appear to explicitly disclose “selecting bit-widths supported by the target hardware platform for corresponding quantization groups to optimize, under a given constraint, a sensitivity metric that is calculated based on each quantization group’s group AQS value.”
However, Yao discloses “selecting bit-widths supported by the target hardware platform for corresponding… [layers] to optimize, under a given constraint, a sensitivity metric that is calculated based on each… [layer’s] AQS value” (Yao, 3.4: “We can use an Integer Linear Programming (ILP) problem to formalize the problem definition of finding the bit-precision setting that has optimal trade-off as described next. Assume that we have B choices for quantizing each layer (i.e., 2 for INT4 or INT8). For a model with L layers, the search space of the ILP will be BL. The goal of solving the ILP problem is to find the best bit configuration among these BL possibilities that results in optimal trade-offs between model perturbation Ω, and user-specified constraints such as model size, BOPS, and latency. Each of these bit-precision settings could result in a different model perturbation. To make the problem tractable, we assume that the perturbations for each layer are independent of each other (i.e.,
Ω
=
∑
i
=
1
L
Ω
i
b
i
, where
Ω
i
b
i
is the i-th layer’s perturbation with
b
i
bit). This allows us to precompute the sensitivity of each layer separately, and it only requires BL computations. For the sensitivity metric, we use the Hessian based perturbation proposed in (Dong et al., 2020, Eq. 2.11). The ILP problem tries to find the right bit precision that minimizes this sensitivity, as follows: [see equations 8-11]”; the examiner notes that
∑
i
=
1
L
Ω
i
b
i
corresponds to “a sensitivity metric that is calculated based on each layer’s AQS value” as it is a summation of the quantization sensitivity (AQS) of each layer).
Yao and the instant application both relate to hardware-aware mixed-precision quantization and are analogous. It would have been obvious to one of ordinary skill in the art, prior to the effective filing date of the claimed invention, to have modified the combination of Zhang/Fan/Liu to include the step of “selecting bit-widths supported by the target hardware platform for corresponding quantization groups to optimize, under a given constraint, a sensitivity metric that is calculated based on each quantization group’s group AQS value” as disclosed by Yao, using the quantization groups and group AQS values disclosed by Zhang/Fan/Liu instead of individual layers and individual AQS values, and one would have been motivated to do so for the purpose of balancing the trade-off between model perturbation and other constraints, such as memory footprint and latency (see Yao, Abstract).
Regarding claim 2, the rejection of claim 1 is incorporated. Zhang as modified by Fan and Liu discloses “group AQS values” but does not appear to explicitly disclose the further limitations of the claim.
However, Yao further discloses “performing Integer Linear Programming (ILP) to obtain a mixed-precision quantization configuration for the neural network that minimizes a total AQS under the given constraint, wherein the total AQS is a sum of all… [layer] AQS values in the neural network” (Yao, 3.4: “We can use an Integer Linear Programming (ILP) problem to formalize the problem definition of finding the bit-precision setting that has optimal trade-off as described next. Assume that we have B choices for quantizing each layer (i.e., 2 for INT4 or INT8). For a model with L layers, the search space of the ILP will be BL. The goal of solving the ILP problem is to find the best bit configuration among these BL possibilities that results in optimal trade-offs between model perturbation Ω, and user-specified constraints such as model size, BOPS, and latency. Each of these bit-precision settings could result in a different model perturbation. To make the problem tractable, we assume that the perturbations for each layer are independent of each other (i.e.,
Ω
=
∑
i
=
1
L
Ω
i
b
i
, where
Ω
i
b
i
is the i-th layer’s perturbation with
b
i
bit). This allows us to precompute the sensitivity of each layer separately, and it only requires BL computations. For the sensitivity metric, we use the Hessian based perturbation proposed in (Dong et al., 2020, Eq. 2.11). The ILP problem tries to find the right bit precision that minimizes this sensitivity, as follows: [see equations 8-11]; the examiner notes that
∑
i
=
1
L
Ω
i
b
i
corresponds to “a total AQS” as it is a summation of the quantization sensitivity (AQS) of each layer).
Yao and the instant application both relate to hardware-aware mixed-precision quantization and are analogous. It would have been obvious to one of ordinary skill in the art, prior to the effective filing date of the claimed invention, to have modified the combination of Zhang/Fan/Liu to include the step of “performing Integer Linear Programming (ILP) to obtain a mixed-precision quantization configuration for the neural network that minimizes a total AQS under the given constraint, wherein the total AQS is a sum of all group AQS values in the neural network” as disclosed by Yao, using the group AQS values disclosed by Zhang/Fan/Liu instead of individual AQS values, and one would have been motivated to do so for the purpose of balancing the trade-off between model perturbation and other constraints, such as memory footprint and latency (see Yao, Abstract).
Regarding claim 3, the rejection of claim 2 is incorporated. Zhang as modified by Fan, Liu, and Yao further discloses “wherein the given constraint is one of model size, latency, and total binary operations of the neural network” (Yao, 3.4: “…and user-specified constraints such as model size, BOPS, and latency” and Yao, 3.4: “Note that it is not necessary to set all these constraints at the same time. Typically, which constraint to use depends on the end-user application”).
Yao and the instant application both relate to hardware-aware mixed-precision quantization and are analogous. It would have been obvious to one of ordinary skill in the art, prior to the effective filing date of the claimed invention, to have modified the combination of Zhang/Fan/Liu to include selecting bit-widths to optimize a sensitivity metric under a given constraint, the given constraint being one of model size, latency, and total binary operations of the neural network as disclosed by Yao, and one would have been motivated to do so for the purpose of balancing the trade-off between model perturbation and other constraints, such as memory footprint and latency (see Yao, Abstract).
Regarding claim 4, the rejection of claim 1 is incorporated. Zhang as modified by Fan and Liu discloses quantization groups, but does not appear to disclose the further limitations of the claim.
However, Yao further discloses “wherein the given constraint is total binary operations (BOPS), which limits a sum of per-…[layer] BOPS, and wherein per-…[layer] BOPS is calculated as a number of MAC operations multiplied by a bit-width of input activation and a bit-width of weights (Yao, 3.4: equation 10; and Yao, 3.4: “…
G
i
b
i
is the corresponding BOPS required for computing that layer. The latter measures the total Bit Operations for calculating a layer (van Baalen et al., 2020):
G
i
b
i
=
b
w
i
b
a
i
M
A
C
i
, where
M
A
C
i
is the total Multiply-Accumulate operations for computing the i-th layer, and
b
w
i
,
b
a
i
are the bit precisions used for weight and activation”; the examiner notes that equation 10 depicts limiting a sum of
G
i
b
i
, which corresponds to “per-layer BOPS” as it represents the BOPS required for computing the i-th layer).
Yao and the instant application both relate to hardware-aware mixed-precision quantization and are analogous. It would have been obvious to one of ordinary skill in the art, prior to the effective filing date of the claimed invention, to have modified the combination of Zhang/Fan/Liu to include selecting bit-widths to optimize a sensitivity metric under a given constraint, the given constraint being BOPS, which limits a sum of per-group BOPS, and wherein per-group BOPS is calculated as a number of MAC operations multiplied by a bit-width of input activation and a bit-width of weights, as disclosed by Yao, using the quantization groups disclosed by Zhang/Fan/Liu, and one would have been motivated to do so for the purpose of minimizing the model perturbation while observing an application specific constraint on total bit operations (see Yao, Introduction).
Regarding claim 5, the rejection of claim 1 is incorporated. Zhang as modified by Fan and Liu discloses quantization groups, but does not appear to explicitly disclose the further limitations of the claim.
However, Yao further discloses “selecting an input activation bit-width and a weight bit-width for each…[layer] to optimize the sensitivity metric under the given constraint” (Yao, Section 4, Table 1: see Method- HAWQ-V3, Precision- W4/8A4/8, “Also, ‘WxAy’ means weight with x-bit and activation with y-bit, and 4/8 means mixed precision with 4 and 8 bits”; the examiner notes that Section 3.4 discloses selecting bit-widths to optimize the sensitivity metric under the given constraint- see rejection of claim 1).
Yao and the instant application both relate to hardware-aware mixed-precision quantization and are analogous. It would have been obvious to one of ordinary skill in the art, prior to the effective filing date of the claimed invention, to have modified the combination of Zhang/Fan/Liu to include selecting an input activation bit-width and a weight bit-width for each quantization group to optimize a sensitivity metric under a given constraint, as disclosed by Yao, using the quantization groups disclosed by Zhang/Fan/Liu, and one would have been motivated to do so for the purpose of balancing the trade-off between model perturbation and other constraints, such as memory footprint and latency (see Yao, Abstract).
Regarding claim 9, the rejection of claim 1 is incorporated. Zhang as modified by Fan, Liu, and Yao further discloses “wherein calculating the AQS value further comprises:
calculating a first output of a convolution layer using input activation and weights at step t;
(Zhang, [n0102]: “obtaining the second output result of the network layer when only the network layer is quantized by the quantization configuration while other network layers are not quantized”; note that the examiner is interpreting “step t” to be a step at which the network layer is quantized);
calculating a second output of the convolution layer using the input activation and weights at step (t-1); (Zhang, [n0102]: “obtaining the first output result of the network layer when all network layers are not quantized”; the examiner notes that this output is obtained at step (t-1) because it occurs before quantization of the network layer);
and measuring a difference between the first output and the second output to compute the AQS value of the convolution layer” (Zhang, [n0102]: “the quantization sensitivity parameter corresponding to a network layer is used to characterize the difference between the output result of the network layer before quantization and the output result after quantization”).
Regarding claim 10, the rejection of claim 1 is incorporated. Zhang as modified by Fan, Liu, and Yao further discloses “wherein calculating the AQS value further comprises:
calculating a first output of a convolution layer using input activation and weights at a first data precision (Zhang, [n0102]: “obtaining the second output result of the network layer when only the network layer is quantized by the quantization configuration while other network layers are not quantized”);
calculating a second output of the convolution layer using the input activation and the weights at a second data precision, the second data precision using a larger bit-width than the first data precision (Zhang, [n0102]: “obtaining the first output result of the network layer when all network layers are not quantized”; the examiner notes that non-quantized layers will have a larger bit-width than quantized layers, see [n0004]: “quantize the full-precision parameters in the neural network into lower precision”);
and measuring a difference between the first output and the second output to compute the AQS value of the convolution layer” (Zhang, [n0102]: “the quantization sensitivity parameter corresponding to a network layer is used to characterize the difference between the output result of the network layer before quantization and the output result after quantization”).
Claim 12 is a system claim corresponding to method claim 1. Zhang discloses “A system operative to determine bit-widths for mixed-precision neural network computing on a target hardware platform, comprising: memory to store a neural network; and processing circuitry coupled to the memory and operative to…” (Zhang, [n0027]: “According to a third aspect of the present disclosure, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a neural network quantization method according to any embodiment of the present disclosure”). The further limitations of the claim correspond to those of claim 1, and the remainder of the rejection follows the same rationale as the rejection of claim 1 above.
Regarding claim 13, the rejection of claim 12 is incorporated. Zhang as modified by Fan and Liu discloses quantization groups and “group AQS values,” but does not appear to explicitly disclose the further limitations of the claim.
However, Yao further discloses “performing Integer Linear Programming (ILP) to obtain the bit-width for each… [layer] that optimizes a total AQS under the given constraint, wherein the total AQS is a sum of all… [layer] AQS values in the neural network” (Yao, 3.4: “We can use an Integer Linear Programming (ILP) problem to formalize the problem definition of finding the bit-precision setting that has optimal trade-off as described next. Assume that we have B choices for quantizing each layer (i.e., 2 for INT4 or INT8). For a model with L layers, the search space of the ILP will be BL. The goal of solving the ILP problem is to find the best bit configuration among these BL possibilities that results in optimal trade-offs between model perturbation Ω, and user-specified constraints such as model size, BOPS, and latency. Each of these bit-precision settings could result in a different model perturbation. To make the problem tractable, we assume that the perturbations for each layer are independent of each other (i.e.,
Ω
=
∑
i
=
1
L
Ω
i
b
i
, where
Ω
i
b
i
is the i-th layer’s perturbation with
b
i
bit). This allows us to precompute the sensitivity of each layer separately, and it only requires BL computations. For the sensitivity metric, we use the Hessian based perturbation proposed in (Dong et al., 2020, Eq. 2.11). The ILP problem tries to find the right bit precision that minimizes this sensitivity, as follows: [see equations 8-11]”; the examiner notes that
∑
i
=
1
L
Ω
i
b
i
corresponds to “a total AQS” as it is a summation of the quantization sensitivity (AQS) of each layer).
Yao and the instant application both relate to hardware-aware mixed-precision quantization and are analogous. It would have been obvious to one of ordinary skill in the art, prior to the effective filing date of the claimed invention, to have modified the combination of Zhang/Fan/Liu to include the step of “performing Integer Linear Programming (ILP) to obtain the bit-width for each quantization group that optimizes a total AQS under the given constraint, wherein the total AQS is a sum of all group AQS values in the neural network” as disclosed by Yao, using the quantization groups and group AQS values disclosed by Zhang/Fan/Liu instead individual layers and AQS values, and one would have been motivated to do so for the purpose of balancing the trade-off between model perturbation and other constraints, such as memory footprint and latency (see Yao, Abstract).
Regarding claim 14, the rejection of claim 12 is incorporated. Claim 14 is a system claim corresponding to method claim 4 and is rejected for the same reasons as given in the rejection of claim 4 above.
Regarding claim 15, the rejection of claim 12 is incorporated. Claim 15 is a system claim corresponding to method claim 5 and is rejected for the same reasons as given in the rejection of claim 5 above.
Regarding claim 19, the rejection of claim 12 is incorporated. Claim 19 is a system claim corresponding to method claim 9 and is rejected for the same reasons as given in the rejection of claim 9 above.
Regarding claim 20, the rejection of claim 12 is incorporated. Claim 20 is a system claim corresponding to method claim 10 and is rejected for the same reasons as given in the rejection of claim 10 above.
Claims 6 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Fan, Liu, and Yao, and further in view of Qadeer et al. (US20210174172) (“Qadeer”).
Regarding claim 6, the rejection of claim 1 is incorporated. Zhang as modified by Fan, Liu, and Yao further discloses “identifying one or more quantization groups to lower corresponding bit-widths under a[n]… accuracy criterion” (Zhang, [n0007]: “when the group of blocks is quantized using the quantization configurations in the target quantization configuration set for the group of blocks, the boundary values of the target performance parameters of the neural network do not exceed a preset parameter range corresponding to the target performance parameters” and Zhang, [n0016]: “Optionally, the target performance parameter includes at least one of the following: the output accuracy of the quantized neural network…”; the examiner notes that quantizing the group of blocks reads on lowering corresponding bit-widths, see [n0004]: “quantize the full-precision parameters in the neural network into lower precision”).
Zhang as modified by Fan, Liu, and Yao does not appear to explicitly disclose that the accuracy criterion is “user-defined.”
However, Qadeer discloses a “user-defined accuracy criterion” (Qadeer, [0025]: “Generally, the system can access or receive as input from a user a loss-of-accuracy threshold in association with a floating-point network. The loss-of-accuracy threshold can define an accuracy (as a value of the accuracy measure), a proportion of an original accuracy of the floating-point network that is desired for the quantized network, or an accuracy interval around the original accuracy of the floating-point network”).
Qadeer and the instant application both relate to quantization of neural networks and are analogous. It would have been obvious to one of ordinary skill in the art, prior to the effective filing date of the claimed invention, to have modified the accuracy criterion disclosed by the combination of Zhang/ Fan/Liu/Yao, to be user-defined as disclosed by Qadeer, and one would have been motivated to do so for the purpose of generating a quantized network that satisfies a user’s desired accuracy (see Qadeer, [0027]).
Regarding claim 16, the rejection of claim 12 is incorporated. Claim 16 is a system claim corresponding to method claim 6, and is rejected for the same reasons as given in the rejection of claim 6 above.
Claims 8 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Fan, Liu, and Yao, and further in view of Li et al. (US20230196197) (“Li”).
Regarding claim 8, the rejection of claim 1 is incorporated. Zhang as modified by Fan, Liu, and Yao discloses the group AQS value, and AQS values of convolution layers in the quantization group, but does not appear to explicitly disclose “wherein the group AQS value is a maximum AQS value of all convolution layers in the quantization group.”
However, Li discloses “wherein the group… [sequence number] value is the maximum… [sequence number] value of all… layers in the…group” (Li, [0012]: “In some embodiments, the determining the group with the least cost value from the groups with the group available resources greater than or equal to the layer required resources of the target layer as the group in which the target layer is located includes: determining the maximum sequence number of each group; the maximum sequence number of any of the groups being the maximum value of the sequence numbers of all to-be-calibrated layers in the group, and the sequence number of any of the to-be-calibrated layers being the order of the to-be-calibrated layer in the model according to the preset processing sequence”).
Li and the instant application both relate to quantization of neural networks and are analogous. It would have been obvious to one of ordinary skill in the art, prior to the effective filing date of the claimed invention, to have modified the combination of Zhang/Fan/Liu/Yao to have the group AQS value be a maximum AQS value of all convolution layers in the quantization group, in the manner disclosed by Li, and one would have been motivated to do so for the purpose of reducing the number of calibration operations (the examiner notes that “calibration” corresponds to determining the quantization factor, see [0004]) to increase calculation speed during the calibration of a model (see Li, Abstract).
Regarding claim 18, the rejection of claim 12 is incorporated. Claim 18 is a system claim corresponding to method claim 8, and is rejected for the same reasons as given in the rejection of claim 8 above.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Fan, Liu, and Yao, and further in view of Cheng et al. (Quantization in Layer’s Input is Matter) (“Cheng”).
Regarding claim 11, the rejection of claim 1 is incorporated. Zhang as modified by Fan, Liu, and Yao do not appear to explicitly disclose the further limitations of the claim.
However, Cheng discloses “grouping input activation and weights of a… layer into a first quantization group, the input activation and the weights having a first data precision (Cheng, 5.3 Combination of Inference and Storage, Algorithm 3: “Quantize the model parameters in the jkth layer and the output of the (jk−1)th layer with
q
i
k
precision”; Examiner notes that, e.g. the model parameters of layerj2 correspond to weights of a neural network layer, and the output of layerj1 corresponds to the input activation of the neural network layerj2, which form a quantization group having a first data precision
q
i
2
); and
grouping an output of the… layer into a second quantization group when… [a] target hardware platform supports different data precisions for the output and the input activation (Examiner notes that the output of layerj2 forms a second quantization group with the parameters of layerj3 at a different data precision of
q
i
3
).
Cheng and the instant application both relate to neural network quantization and are analogous. It would have been obvious to one of ordinary skill in the art, prior to the effective filing date of the claimed invention, to have modified the combination of Zhang/Fan/Liu/Yao with the teachings of Cheng such that grouping the one or more convolution layers into the quantization group further comprises “grouping input activation and weights of a convolution layer into a first quantization group, the input activation and the weights having a first data precision; and grouping an output of the convolution layer into a second quantization group when the target hardware platform supports different data precisions for the output and the input activation,” and one would have been motivated to do so for the purpose of improving performance and resource consumption of the quantization algorithm (see Cheng, Conclusion).
Response to Arguments
Applicant's arguments filed May 2, 2026 regarding the rejections under 35 U.S.C. § 101 have been fully considered, but they are not persuasive.
Applicant argues on page 10 regarding claims 1 and 12 that “the calculations of the activation quantization sensitivity (AQS) values are not abstract mathematical calculations” because they are performed in the context of selecting bit-widths of quantization groups for execution on a target hardware platform. Examiner respectfully disagrees. Performing mathematical calculations in a particular context does not preclude them from being mathematical calculations, and mathematical calculations are one of the abstract idea groupings (MPEP 2106.04(a)).
Applicant further argues on page 11 that claims 1 and 12 are directed to a specific improvement in mixed-precision neural network execution on a hardware platform because “[t]he claimed method and system determine how neural network operations are grouped and executed based on hardware-supported bit-widths, thereby improving computational efficiency of a computer system.” However, this amounts to an argument that the abstract idea itself of determining how neural network operations are grouped and executed provides the improvement. The judicial exception alone cannot provide the improvement (MPEP 2106.05(a)). Claims 1 and 12 do not contain additional elements beyond the judicial exception that would integrate the judicial exception into a practical application.
Applicant further argues on page 11 that claims 1 and 12 integrate mathematical calculations into a practical application of “hardware-constrained bit-width selection for neural network execution” as they do not merely calculate a sensitivity metric, but rather “optimize the sensitivity metric to select hardware-supported bit-widths for respective quantization groups.” Examiner respectfully disagrees. Selecting hardware-supported bit-widths for respective quantization groups is a mental process as analyzed in the Office Action, and therefore does not integrate the mathematical calculations into a practical application. Only additional elements beyond the judicial exception can integrate the judicial exception into a practical application (see MPEP 2106.04(II)(A)(2) “[If] there are no additional claim elements besides the judicial exception, or if the additional claim elements merely recite another judicial exception, that is insufficient to integrate the judicial exception into a practical application”).
Applicant's arguments regarding the rejections under 35 U.S.C. § 103 have been fully considered, but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GWYNEVERE A DETERDING whose telephone number is (571)272-7657. The examiner can normally be reached Mon-Fri. 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kamran Afshar can be reached at (571) 272-7796. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/G.A.D./Examiner, Art Unit 2125
/KAMRAN AFSHAR/Supervisory Patent Examiner, Art Unit 2125
Prosecution Insights