DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is responsive to the Application filed on 7/12/2023. Claims 1-11 are pending in the case. Claims 1, 8, and 11 are independent claims.
Claim Rejections - 35 U.S.C. § 101
35 U.S.C. § 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-11 are rejected under 35 U.S.C. § 101 because the claimed invention is directed to an abstract idea without significantly more.
As to claim 1:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a process.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “quantizing, in a forward transfer process, a network parameter represented by a continuous real value” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III). Note there is no minimum number of neurons in the “transfer process.”
Yes, the limitation “calculating a quantization error” is the abstract idea of a mathematical calculation. See MPEP § 2106.04(a)(2)(I)(C).
Yes, the limitation “determining, in a backward transfer process, a gradient of a weight in a neural network model” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III). Note there is no minimum number of neurons in the “transfer process.”
Yes, the limitation “correcting the gradient of the weight based on the calculated quantization error, wherein the correcting comprises correcting a magnitude of the gradient and correcting a direction of the gradient” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
No, the limitation “updating the neural network model according to the corrected gradient” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP §§ 2106.04(d), 2106.05(f)(1).
No, the limitation “updating the neural network model according to the corrected gradient” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2).
The additional elements, taken alone or in combination, fail to integrate the judicial exception into a practical application.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
No, the limitation “updating the neural network model according to the corrected gradient” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP § 2106.05(f)(1).
No, the limitation “updating the neural network model according to the corrected gradient” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP § 2106.05(f)(2).
The additional elements, taken alone or in combination, fail to amount to significantly more than the judicial exception.
As to claim 2:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a process.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “a quantization step is determined according to a quantization interval and a quantization bit width” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Yes, the limitation “the continuous real value is mapped to a discrete quantization value” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Yes, the limitation “the discrete quantization value is limited in a range that is representable by the quantization bit width” is the abstract idea of a mathematical calculation. See MPEP § 2106.04(a)(2)(I)(C).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
The analysis of the parent claim is incorporated.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
The analysis of the parent claim is incorporated.
As to claim 3:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a process.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “an updated value corresponding to a discrete quantization value is calculated” is the abstract idea of a mathematical calculation. See MPEP § 2106.04(a)(2)(I)(C).
Yes, the limitation “the direction of the gradient is corrected according to the updated value of the discrete quantization value and the continuous real value” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Yes, the limitation “the magnitude of the gradient is corrected according to the quantization error in the forward transfer process of the network” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III). Note there is no minimum number of neurons in the “transfer process.”
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
The analysis of the parent claim is incorporated.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
The analysis of the parent claim is incorporated.
As to claim 4:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a process.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “wherein if a direction of the continuous real value pointing to the updated value of the discrete quantization value is consistent with the direction of the gradient, then in the correcting of the gradient, the direction of the gradient is corrected as an opposite direction of a direction of an original gradient, otherwise the direction of the gradient is maintained as the direction of the original gradient” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
The analysis of the parent claim is incorporated.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
The analysis of the parent claim is incorporated.
As to claim 5:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a process.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “wherein if the direction of the gradient is positive and the continuous real value is less than the discrete quantization value while being greater than the updated value of the discrete quantization value, or if the direction of the gradient is negative and the continuous real value is greater than the discrete quantization value while being less than the updated value of the discrete quantization value, then a magnitude of an original gradient is reduced, wherein the direction of the gradient is positive when a value of the gradient is positive” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
The analysis of the parent claim is incorporated.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
The analysis of the parent claim is incorporated.
As to claim 6:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a process.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “wherein if the direction of the gradient is positive and the continuous real value is greater than the discrete quantization value while also being greater than the updated value of the discrete quantization value, or if the direction of the gradient is negative and the continuous real value is less than the discrete quantization value while also being less than the updated value of the discrete quantization value, then a magnitude of an original gradient is increased” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
The analysis of the parent claim is incorporated.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
The analysis of the parent claim is incorporated.
As to claim 7:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a process.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “wherein in the correcting of the gradient, the calculated quantization error is scaled” is the abstract idea of a mathematical calculation. See MPEP § 2106.04(a)(2)(I)(C).
Yes, the limitation “the gradient of the weight is corrected based on the scaled quantization error” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
The analysis of the parent claim is incorporated.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
The analysis of the parent claim is incorporated.
As to claim 8:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a machine.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “quantize, in a forward transfer process, a network parameter represented by a continuous real value” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III). Note there is no minimum number of neurons in the “transfer process.”
Yes, the limitation “calculate a quantization error” is the abstract idea of a mathematical calculation. See MPEP § 2106.04(a)(2)(I)(C).
Yes, the limitation “determine, in a backward transfer process, a gradient of a weight in a neural network model” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III). Note there is no minimum number of neurons in the “transfer process.”
Yes, the limitation “correct the gradient of the weight based on the calculated quantization error, wherein the correcting comprises correcting a magnitude of the gradient and correcting a direction of the gradient” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
No, the limitation “one or more storage media” is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h).
No, the limitation “one or more processors, wherein the one or more processors and the one or more storage media are configured to” is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h).
No, the limitation “update the neural network model according to the corrected gradient” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP §§ 2106.04(d), 2106.05(f)(1).
No, the limitation “update the neural network model according to the corrected gradient” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2).
The additional elements, taken alone or in combination, fail to integrate the judicial exception into a practical application.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
No, the limitation “one or more storage media” is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(h).
No, the limitation “one or more processors, wherein the one or more processors and the one or more storage media are configured to” is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(h).
No, the limitation “update the neural network model according to the corrected gradient” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP § 2106.05(f)(1).
No, the limitation “update the neural network model according to the corrected gradient” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP § 2106.05(f)(2).
The additional elements, taken alone or in combination, fail to amount to significantly more than the judicial exception.
As to claim 9:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a machine.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
The analysis of the parent claim is incorporated.
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
No, the limitation “receiving a data set corresponding to a requirement of a task that the neural network model is capable of performing” is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g).
No, the limitation “performing operations on the data set in layers from top to bottom in the neural network model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP §§ 2106.04(d), 2106.05(f)(1).
No, the limitation “performing operations on the data set in layers from top to bottom in the neural network model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2).
No, the limitation “outputting a result” is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g).
The additional elements, taken alone or in combination, fail to integrate the judicial exception into a practical application.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
No, the limitation “receiving a data set corresponding to a requirement of a task that the neural network model is capable of performing” is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP § 2106.05(g). Furthermore the additional element is directed to receiving or transmitting data over a network, which the courts have recognized as well‐understood, routine, and conventional when they are claimed in a generic manner. See MPEP § 2106.05(d)(II).
No, the limitation “performing operations on the data set in layers from top to bottom in the neural network model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP § 2106.05(f)(1).
No, the limitation “performing operations on the data set in layers from top to bottom in the neural network model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP § 2106.05(f)(2).
No, the limitation “outputting a result” is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP § 2106.05(g). Furthermore the additional element is directed to receiving or transmitting data over a network, which the courts have recognized as well‐understood, routine, and conventional when they are claimed in a generic manner. See MPEP § 2106.05(d)(II).
The additional elements, taken alone or in combination, fail to amount to significantly more than the judicial exception.
As to claim 10:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a machine.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
The analysis of the parent claim is incorporated.
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
No, the limitation “receive a data set corresponding to a requirement of a task that the neural network model is capable of performing” is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g).
No, the limitation “perform operations on the data set in layers from top to bottom in the neural network model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP §§ 2106.04(d), 2106.05(f)(1).
No, the limitation “perform operations on the data set in layers from top to bottom in the neural network model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2).
No, the limitation “output a result” is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g).
The additional elements, taken alone or in combination, fail to integrate the judicial exception into a practical application.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
No, the limitation “receive a data set corresponding to a requirement of a task that the neural network model is capable of performing” is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP § 2106.05(g). Furthermore the additional element is directed to receiving or transmitting data over a network, which the courts have recognized as well‐understood, routine, and conventional when they are claimed in a generic manner. See MPEP § 2106.05(d)(II).
No, the limitation “perform operations on the data set in layers from top to bottom in the neural network model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP § 2106.05(f)(1).
No, the limitation “perform operations on the data set in layers from top to bottom in the neural network model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP § 2106.05(f)(2).
No, the limitation “output a result” is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP § 2106.05(g). Furthermore the additional element is directed to receiving or transmitting data over a network, which the courts have recognized as well‐understood, routine, and conventional when they are claimed in a generic manner. See MPEP § 2106.05(d)(II).
The additional elements, taken alone or in combination, fail to amount to significantly more than the judicial exception.
As to claim 11:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a manufacture.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “quantizing, in a forward transfer process, a network parameter represented by a continuous real value” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III). Note there is no minimum number of neurons in the “transfer process.”
Yes, the limitation “calculating a quantization error” is the abstract idea of a mathematical calculation. See MPEP § 2106.04(a)(2)(I)(C).
Yes, the limitation “determining, in a backward transfer process, a gradient of a weight in a neural network model” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III). Note there is no minimum number of neurons in the “transfer process.”
Yes, the limitation “correcting the gradient of the weight based on the calculated quantization error, wherein the correcting comprises correcting a magnitude of the gradient and correcting a direction of the gradient” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
No, the limitation “non-transitory computer-readable storage medium storing instructions that, when executed by a computer, cause the computer to perform operations” is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h).
No, the limitation “updating the neural network model according to the corrected gradient” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP §§ 2106.04(d), 2106.05(f)(1).
No, the limitation “updating the neural network model according to the corrected gradient” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2).
The additional elements, taken alone or in combination, fail to integrate the judicial exception into a practical application.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
No, the limitation “non-transitory computer-readable storage medium storing instructions that, when executed by a computer, cause the computer to perform operations” is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(h).
No, the limitation “updating the neural network model according to the corrected gradient” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP § 2106.05(f)(1).
No, the limitation “updating the neural network model according to the corrected gradient” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP § 2106.05(f)(2).
The additional elements, taken alone or in combination, fail to amount to significantly more than the judicial exception.
Claim Rejections - 35 U.S.C. § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. §§ 102 and 103 (or as subject to pre-AIA 35 U.S.C. §§ 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. § 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-11 are rejected under 35 U.S.C. § 102(a)(1) as being anticipated by Liu et al. (US 2020/0394523 A1, hereinafter Liu).
As to independent claim 1, Liu discloses a method for a neural network model, the method comprising:
quantizing, in a forward transfer process, a network parameter (“The data to be quantized is quantized by using the data bit width n1 to obtain a quantized fixed-point number,” paragraph 0143 lines 5-7) represented by a continuous real value (“Fx refers to a floating-point value of the data x before quantization,” paragraph 0073 lines 3-4), and calculating a quantization error (“The quantization error diffbit is determined according to the pre-quantized data and the corresponding quantized data, and the quantization error diffbit is compared with the threshold to obtain a comparison result,” paragraph 0143 lines 7-10);
determining, in a backward transfer process, a gradient of a weight in a neural network model (“perform back propagation on a gradient, which means to propagate the gradient from the output layer to the hidden layer, and finally to the input layer, and sequentially adjust weights and biases of each layer in the neural network according to the gradient,” paragraph 0053 lines 5-9);
correcting the gradient of the weight based on the calculated quantization error, wherein the correcting comprises correcting a magnitude of the gradient and correcting a direction of the gradient (“perform back propagation on a gradient, which means to propagate the gradient from the output layer to the hidden layer, and finally to the input layer, and sequentially adjust weights and biases of each layer in the neural network according to the gradient,” paragraph 0053 lines 5-9); and
updating the neural network model according to the corrected gradient (“perform back propagation on a gradient, which means to propagate the gradient from the output layer to the hidden layer, and finally to the input layer, and sequentially adjust weights and biases of each layer in the neural network according to the gradient,” paragraph 0053 lines 5-9).
As to dependent claim 2, Liu further discloses a method wherein in the quantizing,
a quantization step is determined according to a quantization interval (“The formula (1) shows that when the data to be quantized is quantized by using the quantization parameter corresponding to the first situation, a quantization interval is 2s and is marked as C,” paragraph 0073 lines 16-19) and a quantization bit width (“a maximum value A of a floating-point number may be represented by an n-bit fixed-point number as 2s(2n-1−1), then a maximum value in a number field of the data to be quantized may be represented by an n-bit fixed-point number as 2s(2n-1−1), and a minimum value in the number field of the data to be quantized may be represented by an n-bit fixed-point number as −2s(2n-1−1),” paragraph 0073 lines 9-16),
the continuous real value is mapped to a discrete quantization value (“the following formula (1) may be used to quantize the data to obtain quantized data Ix:
I
x
=
r
o
u
n
d
F
x
2
s
,” paragraph 0072 lines 2-5), and
the discrete quantization value is limited in a range that is representable by the quantization bit width (“a maximum value A of a floating-point number may be represented by an n-bit fixed-point number as 2s(2.sup.n-1−1), then a maximum value in a number field of the data to be quantized may be represented by an n-bit fixed-point number as 2s(2n-1−1), and a minimum value in the number field of the data to be quantized may be represented by an n-bit fixed-point number as −2s(2n-1−1),” paragraph 0073 lines 9-16).
As to dependent claim 3, Liu further discloses a method wherein in the correcting of the gradient,
an updated value corresponding to a discrete quantization value is calculated (“The quantization error diffbit is determined according to the pre-quantized data and the corresponding quantized data, and the quantization error diffbit is compared with the threshold to obtain a comparison result,” paragraph 0143 lines 7-10),
the direction of the gradient is corrected (“perform back propagation on a gradient, which means to propagate the gradient from the output layer to the hidden layer, and finally to the input layer, and sequentially adjust weights and biases of each layer in the neural network according to the gradient,” paragraph 0053 lines 5-9) according to the updated value of the discrete quantization value and the continuous real value (“The quantization error diffbit is determined according to the pre-quantized data and the corresponding quantized data, and the quantization error diffbit is compared with the threshold to obtain a comparison result,” paragraph 0143 lines 7-10), and
the magnitude of the gradient is corrected (“perform back propagation on a gradient, which means to propagate the gradient from the output layer to the hidden layer, and finally to the input layer, and sequentially adjust weights and biases of each layer in the neural network according to the gradient,” paragraph 0053 lines 5-9) according to the quantization error (“The quantization error diffbit is determined according to the pre-quantized data and the corresponding quantized data, and the quantization error diffbit is compared with the threshold to obtain a comparison result,” paragraph 0143 lines 7-10) in the forward transfer process of the network (“perform a forward processing on a signal, which means to transmit the signal from the input layer to the output layer through the hidden layer,” paragraph 0053 lines 2-4).
As to dependent claim 4, Liu further discloses a method wherein if a direction of the continuous real value pointing to the updated value of the discrete quantization value is consistent with the direction of the gradient, then in the correcting of the gradient, the direction of the gradient is corrected as an opposite direction of a direction of an original gradient, otherwise the direction of the gradient is maintained as the direction of the original gradient (“perform back propagation on a gradient, which means to propagate the gradient from the output layer to the hidden layer, and finally to the input layer, and sequentially adjust weights and biases of each layer in the neural network according to the gradient,” paragraph 0053 lines 5-9).
As to dependent claim 5, Liu further discloses a method wherein if the direction of the gradient is positive and the continuous real value is less than the discrete quantization value while being greater than the updated value of the discrete quantization value, or if the direction of the gradient is negative and the continuous real value is greater than the discrete quantization value while being less than the updated value of the discrete quantization value, then a magnitude of an original gradient is reduced, wherein the direction of the gradient is positive when a value of the gradient is positive (“perform back propagation on a gradient, which means to propagate the gradient from the output layer to the hidden layer, and finally to the input layer, and sequentially adjust weights and biases of each layer in the neural network according to the gradient,” paragraph 0053 lines 5-9).
As to dependent claim 6, Liu further discloses a method wherein if the direction of the gradient is positive and the continuous real value is greater than the discrete quantization value while also being greater than the updated value of the discrete quantization value, or if the direction of the gradient is negative and the continuous real value is less than the discrete quantization value while also being less than the updated value of the discrete quantization value, then a magnitude of an original gradient is increased (“perform back propagation on a gradient, which means to propagate the gradient from the output layer to the hidden layer, and finally to the input layer, and sequentially adjust weights and biases of each layer in the neural network according to the gradient,” paragraph 0053 lines 5-9).
As to dependent claim 7, Liu further discloses a method wherein in the correcting of the gradient, the calculated quantization error is scaled (“both the point position parameter and the scaling coefficients are related to the data bit width. Different data bit width may lead to different point position parameters and scaling coefficients,” paragraph 0099 lines 6-9), and the gradient of the weight is corrected based on the scaled quantization error (“perform back propagation on a gradient, which means to propagate the gradient from the output layer to the hidden layer, and finally to the input layer, and sequentially adjust weights and biases of each layer in the neural network according to the gradient,” paragraph 0053 lines 5-9).
As to independent claim 8, Liu discloses an apparatus for a neural network model, the apparatus comprising:
one or more storage media (“Memory,” figure 9 part 120); and
one or more processors (“Processor,” figure 9 part 110), wherein the one or more processors and the one or more storage media are configured to
quantize, in a forward transfer process, a network parameter (“The data to be quantized is quantized by using the data bit width n1 to obtain a quantized fixed-point number,” paragraph 0143 lines 5-7) represented by a continuous real value (“Fx refers to a floating-point value of the data x before quantization,” paragraph 0073 lines 3-4), and calculate a quantization error (“The quantization error diffbit is determined according to the pre-quantized data and the corresponding quantized data, and the quantization error diffbit is compared with the threshold to obtain a comparison result,” paragraph 0143 lines 7-10);
determine, in a backward transfer process, a gradient of a weight in a neural network model (“perform back propagation on a gradient, which means to propagate the gradient from the output layer to the hidden layer, and finally to the input layer, and sequentially adjust weights and biases of each layer in the neural network according to the gradient,” paragraph 0053 lines 5-9);
correct the gradient of the weight based on the calculated quantization error, wherein the correcting comprises correcting a magnitude of the gradient and correcting a direction of the gradient (“perform back propagation on a gradient, which means to propagate the gradient from the output layer to the hidden layer, and finally to the input layer, and sequentially adjust weights and biases of each layer in the neural network according to the gradient,” paragraph 0053 lines 5-9); and
update the neural network model according to the corrected gradient (“perform back propagation on a gradient, which means to propagate the gradient from the output layer to the hidden layer, and finally to the input layer, and sequentially adjust weights and biases of each layer in the neural network according to the gradient,” paragraph 0053 lines 5-9).
As to dependent claim 9, Liu further discloses a method comprising:
receiving a data set corresponding to a requirement of a task that the neural network model is capable of performing (“As a first layer in the neural network, the input layer receives input signals (values) and transmits the signals (values) to a next layer,” paragraph 0046 lines 6-9);
performing operations on the data set in layers from top to bottom in the neural network model (“In the neural network shown in FIG. 1, each of the neurons in the five hidden layers is fully connected, and each of the neurons in each hidden layer is connected with each neuron in the next layer,” paragraph 0047 lines 8-11); and
outputting a result (“The output layer receives the output from the last hidden layer,” paragraph 0048 lines 3-4).
As to dependent claim 10, Liu further discloses an apparatus wherein the one or more processors and the one or more storage media are further configured to:
receive a data set corresponding to a requirement of a task that the neural network model is capable of performing (“As a first layer in the neural network, the input layer receives input signals (values) and transmits the signals (values) to a next layer,” paragraph 0046 lines 6-9);
perform operations on the data set in layers from top to bottom in the neural network model (“In the neural network shown in FIG. 1, each of the neurons in the five hidden layers is fully connected, and each of the neurons in each hidden layer is connected with each neuron in the next layer,” paragraph 0047 lines 8-11); and
output a result (“The output layer receives the output from the last hidden layer,” paragraph 0048 lines 3-4).
As to independent claim 11, Liu discloses a non-transitory computer-readable storage medium storing instructions that, when executed by a computer, cause the computer to perform operations (“a computer-readable medium storing computer-readable program codes (such as software or firmware) which can be executed by the (micro)processor,” paragraph 0161 lines 4-6) comprising:
quantizing, in a forward transfer process, a network parameter (“The data to be quantized is quantized by using the data bit width n1 to obtain a quantized fixed-point number,” paragraph 0143 lines 5-7) represented by a continuous real value (“Fx refers to a floating-point value of the data x before quantization,” paragraph 0073 lines 3-4), and calculating a quantization error (“The quantization error diffbit is determined according to the pre-quantized data and the corresponding quantized data, and the quantization error diffbit is compared with the threshold to obtain a comparison result,” paragraph 0143 lines 7-10);
determining, in a backward transfer process, a gradient of a weight in a neural network model (“perform back propagation on a gradient, which means to propagate the gradient from the output layer to the hidden layer, and finally to the input layer, and sequentially adjust weights and biases of each layer in the neural network according to the gradient,” paragraph 0053 lines 5-9);
correcting the gradient of the weight based on the calculated quantization error, wherein the correcting comprises correcting a magnitude of the gradient and correcting a direction of the gradient (“perform back propagation on a gradient, which means to propagate the gradient from the output layer to the hidden layer, and finally to the input layer, and sequentially adjust weights and biases of each layer in the neural network according to the gradient,” paragraph 0053 lines 5-9); and
updating the neural network model according to the corrected gradient (“perform back propagation on a gradient, which means to propagate the gradient from the output layer to the hidden layer, and finally to the input layer, and sequentially adjust weights and biases of each layer in the neural network according to the gradient,” paragraph 0053 lines 5-9).
Conclusion
The prior art made of record and not relied upon is considered pertinent to Applicant’s disclosure:
US 2021/0216867 A1 disclosing quantization error and backpropagation
Applicant is required under 37 C.F.R. § 1.111(c) to consider these references fully when responding to this action.
It is noted that any citation to specific pages, columns, lines, or figures in the prior art references and any interpretation of the references should not be considered to be limiting in any way. A reference is relevant for all it contains and may be relied upon for all that it would have reasonably suggested to one having ordinary skill in the art. In re Heck, 699 F.2d 1331, 1332-33, 216 U.S.P.Q. 1038, 1039 (Fed. Cir. 1983) (quoting In re Lemelson, 397 F.2d 1006, 1009, 158 U.S.P.Q. 275, 277 (C.C.P.A. 1968)).
In the interests of compact prosecution, Applicant is invited to contact the examiner via electronic media pursuant to USPTO policy outlined MPEP § 502.03. All electronic communication must be authorized in writing. Applicant may wish to file an Internet Communications Authorization Form PTO/SB/439. Applicant may wish to request an interview using the Interview Practice website: http://www.uspto.gov/patent/laws-and-regulations/interview-practice.
Applicant is reminded Internet e-mail may not be used for communication for matters under 35 U.S.C. § 132 or which otherwise require a signature. A reply to an Office action may NOT be communicated by Applicant to the USPTO via Internet e-mail. If such a reply is submitted by Applicant via Internet e-mail, a paper copy will be placed in the appropriate patent application file with an indication that the reply is NOT ENTERED. See MPEP § 502.03(II).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Ryan Barrett whose telephone number is 571 270 3311. The examiner can normally be reached 9:00am to 5:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, Applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor Michelle Bechtold can be reached at 571 431 0762. The fax phone number for the organization where this application or proceeding is assigned is 571 273 8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Ryan Barrett/
Primary Examiner, Art Unit 2148