Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 10/03/2023 was filed before the mailing date of the first office action. Examiner notes that the document titled “Post-training Quantization” was not provided in a legible format and has not been considered. The rest of the submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered as denoted by the examiner.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 8 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Claim 8 recites the limitation “quantizing the plurality of weights from the at least one updated layer of the updated MLM with the original MLM, the quantized MLM, the updated MLM and a representative dataset collected on an Edge device”. It is not clear what it means to quantize weights from a layer of an updated model with all of an original model, a quantized model, an updated model, and a representative dataset collected on an edge device. For purposes of examination, Examiner is interpreting that weights from a layer of an updated model (i.e., an updated layer) may be quantized.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101. Claims 1-10 are directed to a method, claims 11-15 are directed to a separate method, and claims 16-20 are directed to an additional method; therefore, claims 1-20 fall within one of the four statutory categories (i.e., process, machine, manufacture, or composition of matter). However, claims 1-20 fall within the judicial exception of an abstract idea, specifically the abstract ideas of “Mental Processes” (including observation, evaluation, and opinion) and “Mathematical Concepts (including mathematical calculations and relationships)”.
Claim 1:
Claim 1 is directed to a method; therefore, the claim does fall within one of the four statutory categories (i.e., process, machine, manufacture, or composition of matter).
Claim 1 recites the following abstract ideas:
Step 2A Prong 1:
quantizing a plurality of weights and a respective activation function from each of a plurality of respective layers of an original Machine Learning Model (MLM) to generate a quantized MLM comprising a plurality of frozen quantized weights, each layer comprising a summation of a plurality of inputs, each input multiplied by a respective one of the plurality of weights, and the summation gated by the respective activation function (mental step directed to observation, evaluation – a person could quantize, or map, observed values related to weights and an activation function from a layer of a machine learning model from one format to another to generate a quantized model in their mind, potentially assisted by pen and paper (see MPEP 2106.04(a)(2)(III)). Examiner notes that MPEP 2106.04(a)(2)(I)(A) also discusses wherein a conversion between data formats such as binary coded decimal and pure binary was interpreted as a mathematical relationship, and that the broadest reasonable interpretation of quantization includes mapping, or converting, values between formats in a similar manner);
quantizing the plurality of weights from at least one updated layer of an updated MLM to generate a plurality of updated quantized weights, (mental step directed to observation, evaluation – a person could quantize, or map, observed values related to weights of an updated layer of a machine learning model from one format to another to generate a plurality of quantized weights in their mind, potentially assisted by pen and paper. Examiner notes that MPEP 2106.04(a)(2)(I)(A) also discusses wherein a conversion between data formats such as binary coded decimal and pure binary was interpreted as a mathematical relationship, and that the broadest reasonable interpretation of quantization includes mapping, or converting, values between formats in a similar manner);
quantizing the respective activation function of the at least one updated layer of the updated MLM from a difference between the original MLM and the updated MLM, to generate a respective quantized activation function (mental step directed to observation, evaluation – a person could quantize, or map, observed values related to an activation function of an updated layer of a machine learning model from one format based on an observed or mentally determined difference between an original and updated model in their mind, potentially assisted by pen and paper. Examiner notes that MPEP 2106.04(a)(2)(I)(A) also discusses wherein a conversion between data formats such as binary coded decimal and pure binary was interpreted as a mathematical relationship, and that the broadest reasonable interpretation of quantization includes mapping, or converting, values between formats in a similar manner); and
generating a new quantized MLM from the frozen quantized weights, the updated quantized weights and the respective quantized activation function (mental step directed to observation, evaluation – a person could quantize, or map, observed values related to frozen weights, updated weights, and the activation function of a given layer of a machine learning model from one format in their mind to update, or generate a new model, potentially assisted by pen and paper. Examiner notes that MPEP 2106.04(a)(2)(I)(A) also discusses wherein a conversion between data formats such as binary coded decimal and pure binary was interpreted as a mathematical relationship, and that the broadest reasonable interpretation of quantization includes mapping, or converting, values between formats in a similar manner).
Claim 1 recites the following additional elements:
extracting the plurality of frozen quantized weights of at least one frozen layer from the layers of the quantized MLM; and wherein the at least one updated layer of the updated MLM is generated by updating the respective layer of the original MLM.
Step 2A Prong 2:
Extracting frozen quantized weights from at least one frozen layer of a quantized model is interpreted as insignificant extra-solution activity directed to mere data gathering by transmitting data over a network. Updating a layer of an original model is interpreted as insignificant extra-solution activity directed to mere data gathering by receiving new data over a network. These additional elements, when considered as a whole with the aforementioned abstract ideas, do not integrate those abstract ideas into a practical application (see MPEP 2106.05(g)).
Step 2B:
Extracting frozen quantized weights from at least one frozen layer of a quantized model is interpreted as well-understood, routine, conventional activity directed to transmitting data over a network. Updating a layer of an original model is interpreted as well-understood, routine, conventional activity directed to receiving new data over a network. These additional elements, when considered as a whole with the aforementioned abstract ideas, do not amount to significantly more than those abstract ideas (see MPEP 2106.05(d)(II)).
Claim 2 recites determining the difference between the original MLM and the updated MLM by determining an updated scale comprising adding a ratio to an original scale of the original MLM, wherein the ratio is determined by a delta-difference between a delta-max and a delta-min, divided by one less than an integer maximum of an integer range used for quantizing the respective activation function of the updated MLM (mental step directed to observation, evaluation – a person could determine the difference between an original and updated model in their mind by determining an updated scale by adding ration to an original scale and determine that ratio based on a delta-difference between a delta-max and a delta-min divided by one less than an integer maximum of an integer range. Examiner notes that the broadest reasonable interpretation of this limitation also includes an mathematical equation in light of at least paragraph [0023] of Applicant’s specification. Examiner also notes that wherein the integer range is “used for” quantizing an activation function of an updated MLM is interpreted as the intended use of this integer range).
Claim 3 recites determining the delta-max by subtracting an original maximum of an original floating point range of the activation function of each layer of the original MLM from an updated maximum of an updated floating point range of the activation function of each layer of the updated MLM (mental step directed to observation, evaluation – a person could determine an a delta-max value in their mind by subtracting an original maximum of a floating point range of an activation function from an updated maximum of an updated floating point range. Examiner notes that the broadest reasonable interpretation of this limitation also includes an mathematical equation in light of at least paragraph [0020] of Applicant’s specification).
Claim 4 recites determining the delta-min by subtracting an original minimum of an original floating point range of the activation function of each layer of the original MLM from an updated minimum of an updated floating point range of the activation function of each layer of the updated MLM (mental step directed to observation, evaluation – a person could determine an a delta-min value in their mind by subtracting an original minimum of a floating point range of an activation function from an updated minimum of an updated floating point range. Examiner notes that the broadest reasonable interpretation of this limitation also includes an mathematical equation in light of at least paragraph [0020] of Applicant’s specification).
Claim 5 recites determining the difference between the original MLM and the updated MLM by determining an updated integer zero point by adding an offset to an original integer zero point of the original MLM, wherein the offset is determined by subtracting an updated minimum of an updated floating point range of the activation function of each layer of the updated MLM divided by an updated scale, from an original minimum of an original floating point range of the activation function of each layer of the original MLM divided by an original scale of the original MLM (mental step directed to observation, evaluation – a person could determine the difference between an original and updated model in their mind by determining an updated integer zero point by adding an offset to an original integer zero point and determine this offset by dividing an updated minimum of an updated floating point range of an activation function by an updated scale and subtracting this value from an original minimum of a floating point range divided by an original scale. Examiner notes that the broadest reasonable interpretation of this limitation also includes an mathematical equation in light of at least paragraph [0023] of Applicant’s specification).
Claim 6 recites wherein the original integer zero point is determined by subtracting the original minimum of the original floating point range of the activation function of each layer of the original MLM, divided by the original scale, from an integer minimum of an integer range used for quantizing the respective activation function of the quantized MLM (mental step directed to observation, evaluation – a person could determine an original integer zero point in their mind by dividing an original minimum of an original floating point range by an original scale and subtracting that value from an original minimum of an integer range. Examiner notes that the broadest reasonable interpretation of this limitation also includes an mathematical equation in light of at least paragraph [0022] of Applicant’s specification. Examiner also notes that wherein the integer range is “used for” quantizing an activation function of a quantized MLM is interpreted as the intended use of this integer range).
Claim 7 recites wherein the original scale is determined by subtracting the original minimum of the original floating point range from the original maximum of the original floating point range, divided by the integer minimum of the integer range subtracted from an integer maximum of the integer range (mental step directed to observation, evaluation – a person could determine an original scale in their mind by subtracting an original minimum of an original floating point range from an original maximum of that floating point range and dividing that value by the integer minimum of an integer range subtracted from an integer maximum of that integer range. Examiner notes that the broadest reasonable interpretation of this limitation also includes an mathematical equation in light of at least paragraph [0022] of Applicant’s specification).
Claim 8 recites quantizing the plurality of weights from the at least one updated layer of the updated MLM with the original MLM, the quantized MLM, the updated MLM and a representative dataset collected on an Edge device (Examiner notes that this limitation is interpreted in light of the 112(b) rejection of claim 8 such that that weights from a layer of an updated model (i.e., an updated layer) may be quantized. Given this interpretation, this limitation is interpreted as a mental step directed to observation, evaluation – a person could quantize, or map, observed values related to weights of an updated layer of a machine learning model from one format to another in their mind, potentially assisted by pen and paper. Examiner notes that MPEP 2106.04(a)(2)(I)(A) also discusses wherein a conversion between data formats such as binary coded decimal and pure binary was interpreted as a mathematical relationship, and that the broadest reasonable interpretation of quantization includes mapping, or converting, values between formats in a similar manner),
wherein the representative dataset is a subset of a dataset used to train the updated MLM. As the claims do not recite any particular machine learning model nor particular steps for training an updated model, training or updating a model with a representative subset of a dataset is interpreted as an additional element directed to generic computer activity in the technological environment in which the claimed abstract ideas are performed and utilizing data stored in memory, or collected, on an edge device. This limitation does not integrate the claimed abstract ideas into a practical application or amount to significantly more than the claimed abstract ideas (see MPEP 2106.05(d) and MPEP 2106.05(h)).
Claim 9 recites wherein generating the quantized MLM further comprises quantizing the plurality of weights from each one of a plurality of tensor inputs of the original MLM (mental step directed to observation, evaluation – a person could quantize, or map, observed values related to weights from a plurality of tensor inputs of an original model from one format to another in their mind, potentially assisted by pen and paper. Examiner notes that MPEP 2106.04(a)(2)(I)(A) also discusses wherein a conversion between data formats such as binary coded decimal and pure binary was interpreted as a mathematical relationship, and that the broadest reasonable interpretation of quantization includes mapping, or converting, values between formats in a similar manner).
Claim 10 recites wherein generating the quantized MLM further comprises quantizing the plurality of weights from each one of a plurality of channels of the original MLM (mental step directed to observation, evaluation – a person could quantize, or map, observed values related to weights of a plurality of model channels from one format to another in their mind, potentially assisted by pen and paper. Examiner notes that MPEP 2106.04(a)(2)(I)(A) also discusses wherein a conversion between data formats such as binary coded decimal and pure binary was interpreted as a mathematical relationship, and that the broadest reasonable interpretation of quantization includes mapping, or converting, values between formats in a similar manner).
Claim 11:
Claim 11 is directed to a method; therefore, the claim does fall within one of the four statutory categories (i.e., process, machine, manufacture, or composition of matter).
Claim 11 recites the following abstract ideas:
Step 2A Prong 1:
wherein the quantized MLM is generated by quantizing an original MLM (mental step directed to observation, evaluation – a person could quantize, or map, observed values related to an original machine learning model from one format to another to generate a quantized model in their mind, potentially assisted by pen and paper (see MPEP 2106.04(a)(2)(III)). Examiner notes that MPEP 2106.04(a)(2)(I)(A) also discusses wherein a conversion between data formats such as binary coded decimal and pure binary was interpreted as a mathematical relationship, and that the broadest reasonable interpretation of quantization includes mapping, or converting, values between formats in a similar manner);
quantizing a plurality of updated weights from an updated MLM to generate a plurality of updated quantized weights (mental step directed to observation, evaluation – a person could quantize, or map, observed values related to weights of an updated layer of a machine learning model from one format to another to generate a plurality of quantized weights in their mind, potentially assisted by pen and paper. Examiner notes that MPEP 2106.04(a)(2)(I)(A) also discusses wherein a conversion between data formats such as binary coded decimal and pure binary was interpreted as a mathematical relationship, and that the broadest reasonable interpretation of quantization includes mapping, or converting, values between formats in a similar manner);
quantizing a respective activation function of each updated layer from the updated MLM from a difference between the original MLM and the updated MLM to generate a quantized activation function, wherein the difference comprises an updated scale and an updated integer zero point (mental step directed to observation, evaluation – a person could quantize, or map, observed values related to an activation function of an updated layer of a machine learning model from one format based on an observed or mentally determined difference between an original and updated model in their mind, potentially assisted by pen and paper. Examiner notes that MPEP 2106.04(a)(2)(I)(A) also discusses wherein a conversion between data formats such as binary coded decimal and pure binary was interpreted as a mathematical relationship, and that the broadest reasonable interpretation of quantization includes mapping, or converting, values between formats in a similar manner); and
generating a new quantized MLM from the frozen quantized weights, the updated quantized weights and the respective quantized activation function of each updated layer (mental step directed to observation, evaluation – a person could quantize, or map, observed values related to frozen weights, updated weights, and the activation function of a given layer of a machine learning model from one format in their mind to update, or generate a new model, potentially assisted by pen and paper. Examiner notes that MPEP 2106.04(a)(2)(I)(A) also discusses wherein a conversion between data formats such as binary coded decimal and pure binary was interpreted as a mathematical relationship, and that the broadest reasonable interpretation of quantization includes mapping, or converting, values between formats in a similar manner).
Claim 11 recites the following additional elements:
extracting a plurality of frozen quantized weights from each frozen layer of a quantized Machine Learning Model (MLM), and wherein the updated MLM is generated by updating at least one layer of the original MLM to form at least one updated layer.
Step 2A Prong 2:
Extracting frozen quantized weights from at least one frozen layer of a quantized model is interpreted as insignificant extra-solution activity directed to mere data gathering by transmitting data over a network. Updating a layer of an original model is interpreted as insignificant extra-solution activity directed to mere data gathering by receiving new data over a network. These additional elements, when considered as a whole with the aforementioned abstract ideas, do not integrate those abstract ideas into a practical application (see MPEP 2106.05(g)).
Step 2B:
Extracting frozen quantized weights from at least one frozen layer of a quantized model is interpreted as well-understood, routine, conventional activity directed to transmitting data over a network. Updating a layer of an original model is interpreted as well-understood, routine, conventional activity directed to receiving new data over a network. These additional elements, when considered as a whole with the aforementioned abstract ideas, do not amount to significantly more than those abstract ideas (see MPEP 2106.05(d)(II)).
Claim 12 is a method claim and its limitation is included in claim 2. Claim 12 is rejected for the same reasons as claim 2.
Claim 13 recites determining the delta-max by subtracting an original maximum of an original floating point range of the activation function of each layer of the original MLM from an updated maximum of an updated floating point range of the activation function of each layer of the updated MLM (mental step directed to observation, evaluation – a person could determine an a delta-max value in their mind by subtracting an original maximum of a floating point range of an activation function from an updated maximum of an updated floating point range. Examiner notes that the broadest reasonable interpretation of this limitation also includes an mathematical equation in light of at least paragraph [0020] of Applicant’s specification)
and determining the delta-min by subtracting an original minimum of the original floating point range of the activation function of each layer of the original MLM from an updated minimum of the updated floating point range of the activation function of each layer of the updated MLM (mental step directed to observation, evaluation – a person could determine an a delta-min value in their mind by subtracting an original minimum of a floating point range of an activation function from an updated minimum of an updated floating point range. Examiner notes that the broadest reasonable interpretation of this limitation also includes an mathematical equation in light of at least paragraph [0020] of Applicant’s specification).
Claim 14 is a method claim and its limitation is included in claim 5. Claim 14 is rejected for the same reasons as claim 5.
Claim 15 recites wherein the original integer zero point is determined by subtracting from an integer minimum of an integer range used for quantizing the respective activation function of the quantized MLM, a quantized minimum of a quantized floating point range of the activation function of each layer of the quantized MLM, divided by the original scale (mental step directed to observation, evaluation – a person could determine an original integer zero point in their mind by dividing an original minimum of an original floating point range by an original scale and subtracting that value from an original minimum of an integer range. Examiner notes that the broadest reasonable interpretation of this limitation also includes an mathematical equation in light of at least paragraph [0022] of Applicant’s specification. Examiner also notes that wherein the integer range is “used for” quantizing an activation function of a quantized MLM is interpreted as the intended use of this integer range),
and the original scale is determined by subtracting the original minimum of the original floating point range from the original maximum of the original floating point range, divided by the integer minimum of the integer range subtracted from an integer maximum of the integer range (mental step directed to observation, evaluation – a person could determine an original scale in their mind by subtracting an original minimum of an original floating point range from an original maximum of that floating point range and dividing that value by the integer minimum of an integer range subtracted from an integer maximum of that integer range. Examiner notes that the broadest reasonable interpretation of this limitation also includes an mathematical equation in light of at least paragraph [0022] of Applicant’s specification).
Claim 16:
Claim 16 is directed to a method; therefore, the claim does fall within one of the four statutory categories (i.e., process, machine, manufacture, or composition of matter).
Claim 16 recites the following abstract ideas:
Step 2A Prong 1:
wherein the quantized model is generated by quantizing an original model (mental step directed to observation, evaluation – a person could quantize, or map, observed values related to an original machine learning model from one format to another to generate a quantized model in their mind, potentially assisted by pen and paper (see MPEP 2106.04(a)(2)(III)). Examiner notes that MPEP 2106.04(a)(2)(I)(A) also discusses wherein a conversion between data formats such as binary coded decimal and pure binary was interpreted as a mathematical relationship, and that the broadest reasonable interpretation of quantization includes mapping, or converting, values between formats in a similar manner);
quantizing a plurality of updated weights from an updated model to generate a plurality of updated quantized weights (mental step directed to observation, evaluation – a person could quantize, or map, observed values related to weights of an updated layer of a machine learning model from one format to another to generate a plurality of quantized weights in their mind, potentially assisted by pen and paper. Examiner notes that MPEP 2106.04(a)(2)(I)(A) also discusses wherein a conversion between data formats such as binary coded decimal and pure binary was interpreted as a mathematical relationship, and that the broadest reasonable interpretation of quantization includes mapping, or converting, values between formats in a similar manner);
quantizing a respective activation function of each updated layer from the updated model from a difference between the original model and the updated model to generate a quantized activation function, wherein the difference comprises an updated scale and an updated integer zero point, the updated scale modifying an original scale used to quantize the original model with a ratio proportional to a floating point range difference between the updated model and the original model, and the updated integer zero point modifying an original integer zero point used to quantize the original model with a scaled difference between the floating point range minimums of the updated model and the original model (mental step directed to observation, evaluation – a person could quantize, or map, observed values related to an activation function of an updated layer of a machine learning model from one format based on an observed or mentally determined difference between an original and updated model in their mind, potentially assisted by pen and paper. Examiner notes that MPEP 2106.04(a)(2)(I)(A) also discusses wherein a conversion between data formats such as binary coded decimal and pure binary was interpreted as a mathematical relationship, and that the broadest reasonable interpretation of quantization includes mapping, or converting, values between formats in a similar manner); and
generating a new quantized model from the frozen quantized weights, the updated quantized weights and the respective quantized activation function of each updated layer (mental step directed to observation, evaluation – a person could quantize, or map, observed values related to frozen weights, updated weights, and the activation function of a given layer of a machine learning model from one format in their mind to update, or generate a new model, potentially assisted by pen and paper. Examiner notes that MPEP 2106.04(a)(2)(I)(A) also discusses wherein a conversion between data formats such as binary coded decimal and pure binary was interpreted as a mathematical relationship, and that the broadest reasonable interpretation of quantization includes mapping, or converting, values between formats in a similar manner).
Claim 16 recites the following additional elements:
extracting a plurality of frozen quantized weights from each frozen layer of a quantized model, and wherein the updated model is generated by updating at least one layer of the original model to form at least one updated layer.
Step 2A Prong 2:
Extracting frozen quantized weights from at least one frozen layer of a quantized model is interpreted as insignificant extra-solution activity directed to mere data gathering by transmitting data over a network. Updating a layer of an original model is interpreted as insignificant extra-solution activity directed to mere data gathering by receiving new data over a network. These additional elements, when considered as a whole with the aforementioned abstract ideas, do not integrate those abstract ideas into a practical application (see MPEP 2106.05(g)).
Step 2B:
Extracting frozen quantized weights from at least one frozen layer of a quantized model is interpreted as well-understood, routine, conventional activity directed to transmitting data over a network. Updating a layer of an original model is interpreted as well-understood, routine, conventional activity directed to receiving new data over a network. These additional elements, when considered as a whole with the aforementioned abstract ideas, do not amount to significantly more than those abstract ideas (see MPEP 2106.05(d)(II)).
Claim 17 recites training the updated model with a new dataset stored on an Edge device. As the claims do not recite any particular machine learning model nor particular steps for training an updated model, training this model is interpreted as an additional element directed to generic computer activity in the technological environment in which the claimed abstract ideas are performed and utilizing data stored in memory on an edge device. This limitation does not integrate the claimed abstract ideas into a practical application or amount to significantly more than the claimed abstract ideas (see MPEP 2106.05(d) and MPEP 2106.05(h)).
Claim 18 is a method claim and its limitation is included in claim 8. Claim 18 is rejected for the same reasons as claim 8.
Claim 19 recites wherein the updated model adapts to the new dataset through transfer learning. As the claims do not recite any particular machine learning model nor particular steps for adapting a model through transfer learning, this transfer learning step is interpreted as an additional element directed to generic computer activity in the technological environment in which the claimed abstract ideas are performed. This limitation does not integrate the claimed abstract ideas into a practical application or amount to significantly more than the claimed abstract ideas (see MPEP 2106.05(d) and MPEP 2106.05(h)).
Claim 20 recites inferencing with a neural processing unit and the new quantized model to perform transfer learning on an Edge device. The neural processing unit and edge device are interpreted as additional elements directed to generic computer components merely utilized in the technological environment in which the claimed abstract ideas are performed. As the claims do not recite any particular machine learning model nor particular steps for model inference or transfer learning, inferencing and transfer learning are interpreted as an additional elements directed to generic computer activity in the technological environment in which the claimed abstract ideas are performed. These limitations do not integrate the claimed abstract ideas into a practical application or amount to significantly more than the claimed abstract ideas (see MPEP 2106.05(d) and MPEP 2106.05(h)).
Viewed as a whole, these additional claim elements do not provide meaningful limitations to transform the abstract idea into a patent eligible application of the abstract idea such that the claims amount to significantly more than the abstract idea itself. Therefore, the claims are rejected under 35 U.S.C. 101.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5, 9-15, and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Sher et al (US 20240193422 A1, herein Sher) in view of Demaj et al (US 20220164664 A1, herein Demaj).
Regarding claim 1, Sher teaches a method (para. [0075] recites “FIG. 5 illustrates a process 500 for layer-by-layer training” (i.e. a method, or process)) for partial quantization to achieve full quantized model comprising: quantizing a plurality of weights and a respective activation function from each of a plurality of respective layers of an original Machine Learning Model (MLM) to generate a quantized MLM comprising a plurality of frozen quantized weights (para. [0055] recites “CNN quantization may comprise or consist of quantization of the weights and/or activations of the CNN. Each of these weights and activations are represented as arrays of real values”. Para. [0075] recites “In process 500, the layers to be quantized are quantized and frozen. It should be understood that, when a layer is "frozen," the weights of the layer are set at their current value, such that they are not modified any further during the training process” (i.e., weights and activation functions from layers of a machine learning model can be quantized and the weights can be frozen)), each layer comprising a summation of a plurality of inputs, each input multiplied by a respective one of the plurality of weights, and the summation gated by the respective activation function (para. [0061] recites “FIG. 2 illustrates a standard convolutional layer 200 in a full-precision CNN, according to an example. In convolutional layer 200, a convolution 220 with weights 215 is applied to an input 205. An activation function 230 is then applied to the output of convolution 220 to produce an activation or output 235, which may become the input 205 to a subsequent layer. Input 205, weights 215, the output of convolution 220, and output 235 may each comprise an array of one or a plurality of floating-point values” (i.e., each layer comprises inputs multiplied by weights and gated by activation functions));
quantizing the respective activation function of the at least one updated layer of the updated MLM [from a difference between the original MLM and the updated MLM], to generate a respective quantized activation function (para. [0055] recites “CNN quantization may comprise or consist of quantization of the weights and/or activations of the CNN. Each of these weights and activations are represented as arrays of real values”. Fig. 5 and para. [0075] recite “Process 500 may be used to quantize one or more layers of a CNN that has been previously trained with floating-point weights”. Para. [0076]-[0077] recite “If another layer remains to be quantized (i.e., "Yes" in subprocess 520), process 500 proceeds to subprocess 530. Otherwise, if no more layers remain to be quantized (i.e., "No" in subprocess 520), process 500 proceeds to subprocess 570. In subprocess 530, input 205 of the current layer being quantized is quantized (e.g., by input quantization module 305). In subprocess 540, the model is retrained” (i.e., quantizing an activation function from a layer of a model that has been updated, or retrained, in an iterative training process));
extracting the plurality of frozen quantized weights of at least one frozen layer from the layers of the quantized MLM (para. [0066] recites “optimum parameters 307 are saved for future quantization operations on inputs 205. Subsequently, weights quantization module 315 solves the MMSE problem to obtain the optimum parameters 317, and saves these optimum parameters 317 for future quantization operations on weights 215. It should be understood that, during operation of the QNN, weights quantization module 315 is unnecessary, since the final quantized integer values of weights 215 may simply be saved and retrieved during operation” (i.e., previously frozen weights can be retrieved, or extracted));
quantizing the plurality of weights from at least one updated layer of an updated MLM to generate a plurality of updated quantized weights, wherein the at least one updated layer of the updated MLM is generated by updating the respective layer of the original MLM; and generating a new quantized MLM from the frozen quantized weights, the updated quantized weights and the respective quantized activation function (fig. 5 and para. [0076]-[0077] recite “the layers are quantized in order from the input to the output of the CNN. If another layer remains to be quantized (i.e., "Yes" in subprocess 520), process 500 proceeds to subprocess 530. Otherwise, if no more layers remain to be quantized (i.e., "No" in subprocess 520), process 500 proceeds to subprocess 570. In subprocess 530, input 205 of the current layer being quantized is quantized (e.g., by input quantization module 305). In subprocess 540, the model is retrained” (i.e., the model can be retrained, or updated, and an quantized model can be generated based on the quantized weights and activation functions of the model layers)).
However, while Sher teaches that activation functions of layers of a machine learning model can be quantized (see at least paragraph [0055]), Sher does not explicitly teach quantizing [the respective activation function of the at least one updated layer of] the updated MLM from a difference between the original MLM and the updated MLM, [to generate a respective quantized activation function].
Demaj teaches quantizing [the respective activation function of the at least one updated layer of] the updated MLM from a difference between the original MLM and the updated MLM, [to generate a respective quantized activation function] (para. [0067] recites “The method then includes steps 24 to 36 for adapting the quantization of the corrected weights of the layer to the values of these corrected weights. Para. [0068] recites “At step 25, the processing unit 106 calculates quantization step sizes of the corrected weights of the layer” (i.e., determining the difference between an original model and the updated quantized model. Examiner notes that that Demaj teaches an updated zero point modifying an original zero point in at least paragraph [0082] and an updated scale modifying an original scale in at least paragraph [0083])).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine these teachings by to modify the model quantization method from Sher with the quantization method from Demaj. Sher and Demaj are both directed to neural network quantization methods, and Sher states in at least paragraph [0054] that “different quantization schemes can be utilized”. One of ordinary skill in the art would accordingly recognize that a different quantization scheme such as the method from Demaj could be utilized in the quantization method from Sher.
Regarding claim 2, the combination of Sher and Demaj teaches the method of claim 1 as mentioned above, further comprising determining the difference between the original MLM and the updated MLM by determining an updated scale comprising adding a ratio to an original scale of the original MLM, wherein the ratio is determined by a delta-difference between a delta-max and a delta-min, divided by one less than an integer maximum of an integer range used for quantizing the respective activation function of the updated MLM (Sher para. [0055] recites “CNN quantization may comprise or consist of quantization of the weights and/or activations of the CNN. Each of these weights and activations are represented as arrays of real values” (i.e., activation functions can be quantized). Demaj para. [0083] recites “At step 35, processing unit 106 updates the value of the quantization step size according to the following formula:
PNG
media_image1.png
90
384
media_image1.png
Greyscale
where Scalec(n+ 1) is the value of the updated quantization step size of the weights of the channel, minqc(n) is the minimum value of the corrected weights of the channel, maxqc(n) is the maximum value of the weights of the channel, Scale(n) is the value of the quantization step size of the initial weights of the channel, and minq and maxq are respectively the minimum and maximum values that the weights can take”. Demaj para. [0069] recites “when the weights are in an asymmetric 8-bit unsigned format, the minimum value minq is equal to 0 and the maximum value maxq is equal to 255. When the weights are in a symmetric 8-bit signed format, the minimum value minq is equal to -127 and the maximum value maxq is equal to 127” (i.e., determining an updated scale value based on the differences between original and updated maximum and minimum floating point values, and a value one less than the integer range (ex., 255 in an asymmetric unsigned format). Examiner also notes that wherein the integer range is “used for” quantizing an activation function of an updated MLM is interpreted as the intended use of this integer range)).
Regarding claim 3, the combination of Sher and Demaj teaches the method of claim 2 as mentioned above, further comprising determining the delta-max by subtracting an original maximum of an original floating point range of the activation function of each layer of the original MLM from an updated maximum of an updated floating point range of the activation function of each layer of the updated MLM (Sher para. [0055] recites “CNN quantization may comprise or consist of quantization of the weights and/or activations of the CNN. Each of these weights and activations are represented as arrays of real values” (i.e., activation functions can be quantized). Demaj para. [0067] recites “The method then includes steps 24 to 36 for adapting the quantization of the corrected weights of the layer to the values of these corrected weights. At step 24, the processing unit 106 determines a minimum weight and a maximum weight from at least one set of the corrected weights of the layer”. Demaj para. [0068] recites “At step 25, the processing unit 106 calculates quantization step sizes of the corrected weights of the layer. In embodiments, the processing unit 106 calculates a quantization step size for the set of the corrected weights of the layer using the minimum weight and maximum weight of the set of the corrected weights of the layer” (i.e., determining the difference between an original maximum floating point value and an corrected, or updated maximum floating point value)).
Regarding claim 4, the combination of Sher and Demaj teaches the method of claim 2 as mentioned above, further comprising determining the delta-min by subtracting an original minimum of an original floating point range of the activation function of each layer of the original MLM from an updated minimum of an updated floating point range of the activation function of each layer of the updated MLM (Sher para. [0055] recites “CNN quantization may comprise or consist of quantization of the weights and/or activations of the CNN. Each of these weights and activations are represented as arrays of real values” (i.e., activation functions can be quantized). Demaj para. [0067] recites “The method then includes steps 24 to 36 for adapting the quantization of the corrected weights of the layer to the values of these corrected weights. At step 24, the processing unit 106 determines a minimum weight and a maximum weight from at least one set of the corrected weights of the layer”. Demaj para. [0068] recites “At step 25, the processing unit 106 calculates quantization step sizes of the corrected weights of the layer. In embodiments, the processing unit 106 calculates a quantization step size for the set of the corrected weights of the layer using the minimum weight and maximum weight of the set of the corrected weights of the layer” (i.e., determining the difference between an original minimum floating point value and an corrected, or updated minimum floating point value)).
Regarding claim 5, the combination of Sher and Demaj teaches the method of claim 1 as mentioned above, further comprising determining the difference between the original MLM and the updated MLM by determining an updated integer zero point by adding an offset to an original integer zero point of the original MLM, wherein the offset is determined by subtracting an updated minimum of an updated floating point range of the activation function of each layer of the updated MLM divided by an updated scale, from an original minimum of an original floating point range of the activation function of each layer of the original MLM divided by an original scale of the original MLM (Sher para. [0055] recites “CNN quantization may comprise or consist of quantization of the weights and/or activations of the CNN. Each of these weights and activations are represented as arrays of real values” (i.e., activation functions can be quantized). Demaj para. [0082] recites “At step 34, the processing unit 106 updates the zero-point value stored in memory 104. The processing unit 106 calculates the new zero-point using the following formula:
PNG
media_image2.png
74
506
media_image2.png
Greyscale
where Zp(n+1) is the updated zero-point value, minq and maxq are respectively the minimum and maximum values that the weights can take, min(n)=Scale(n)·minqc(n)-Zp(n) and max(n)=Scale(n)·maxqc·(n)-Zp (n), Zp(n) being the zero-point value stored in the memory 104, minqc(n) being the minimum value of the corrected weights of the channel, maxqc(n) being the maximum value of the weights of the channel” (i.e., determining an updated zero point value based on an original and updated floating point range minimum, and an updated scale)).
Regarding claim 9, the combination of Sher and Demaj teaches the method of claim 1 as mentioned above, wherein generating the quantized MLM further comprises quantizing the plurality of weights from each one of a plurality of tensor inputs of the original MLM (Demaj para. [0026] recites “The set can be the set of the weights of the layer when the layer is quantized uniformly or the set of the weights of a same channel of the layer when the layer is quantized per channel” (i.e., quantizing weights can be done per channel. Examiner notes that one of ordinary skill in the art would recognize that tensors can be comprised of multiple channels; therefore, quantizing channels would mean quantizing corresponding tensor inputs)).
Regarding claim 10, the combination of Sher and Demaj teaches the method of claim 1 as mentioned above, wherein generating the quantized MLM further comprises quantizing the plurality of weights from each one of a plurality of channels of the original MLM (Demaj para. [0081] recites “Then, the values of the weights, of the quantization step size, and of the zero-point stored in the memory 104 are replaced at step 33 by the newly-calculated values. Furthermore, if the layer is to be quantized per channel, then the processing unit 106 performs quantization of the weights for each channel of the layer in steps 34 and 37” (i.e., quantizing weights for a plurality of channels of the model)).
Regarding claim 11, Sher teaches a method (para. [0075] recites “FIG. 5 illustrates a process 500 for layer-by-layer training” (i.e. a method, or process)) for partial quantization to achieve full quantized model comprising: extracting a plurality of frozen quantized weights from each frozen layer of a quantized Machine Learning Model (MLM), wherein the quantized MLM is generated by quantizing an original MLM (fig. 5 and para. [0075] recite “Process 500 may be used to quantize one or more layers of a CNN that has been previously trained with floating-point weights. In process 500, the layers to be quantized are quantized and frozen. It should be understood that, when a layer is "frozen," the weights of the layer are set at their current value, such that they are not modified any further during the training process” (i.e., quantizing a model and freezing model weights). Para. [0066] recites “optimum parameters 307 are saved for future quantization operations on inputs 205. Subsequently, weights quantization module 315 solves the MMSE problem to obtain the optimum parameters 317, and saves these optimum parameters 317 for future quantization operations on weights 215. It should be understood that, during operation of the QNN, weights quantization module 315 is unnecessary, since the final quantized integer values of weights 215 may simply be saved and retrieved during operation” (i.e., previously frozen weights can be retrieved, or extracted));
quantizing a respective activation function of each updated layer from the updated MLM [from a difference between the original MLM and the updated MLM] to generate a quantized activation function; quantizing a plurality of updated weights from an updated MLM to generate a plurality of updated quantized weights (para. [0055] recites “CNN quantization may comprise or consist of quantization of the weights and/or activations of the CNN. Each of these weights and activations are represented as arrays of real values”. Para. [0075] recites “In process 500, the layers to be quantized are quantized and frozen. It should be understood that, when a layer is "frozen," the weights of the layer are set at their current value, such that they are not modified any further during the training process” (i.e., weights and activation functions from layers of a machine learning model can be quantized and the weights can be frozen)),
wherein the updated MLM is generated by updating at least one layer of the original MLM to form at least one updated layer; and generating a new quantized MLM from the frozen quantized weights, the updated quantized weights and the respective quantized activation function of each updated layer (fig. 5 and para. [0076]-[0077] recite “the layers are quantized in order from the input to the output of the CNN. If another layer remains to be quantized (i.e., "Yes" in subprocess 520), process 500 proceeds to subprocess 530. Otherwise, if no more layers remain to be quantized (i.e., "No" in subprocess 520), process 500 proceeds to subprocess 570. In subprocess 530, input 205 of the current layer being quantized is quantized (e.g., by input quantization module 305). In subprocess 540, the model is retrained” (i.e., the model can be retrained, or updated, and an quantized model can be generated based on the quantized weights and activation functions of the model layers)).
However, while Sher teaches that activation functions of layers of a machine learning model can be quantized (see at least paragraph [0055]), Sher does not explicitly teach quantizing [a respective activation function of each updated layer from] the updated MLM from a difference between the original MLM and the updated MLM [to generate a quantized activation function], wherein the difference comprises an updated scale and an updated integer zero point.
Demaj teaches quantizing [a respective activation function of each updated layer from] the updated MLM from a difference between the original MLM and the updated MLM [to generate a quantized activation function], wherein the difference comprises an updated scale and an updated integer zero point (para. [0067] recites “The method then includes steps 24 to 36 for adapting the quantization of the corrected weights of the layer to the values of these corrected weights. Para. [0068] recites “At step 25, the processing unit 106 calculates quantization step sizes of the corrected weights of the layer” (i.e., determining the difference between an original model and the updated quantized model. Examiner notes that that Demaj teaches an updated zero point in at least paragraph [0082] and an updated scale in at least paragraph [0083])).
See claim 1 for motivation to combine.
Claim 12 is a method claim and its limitation is included in claim 2. Claim 12 is rejected for the same reasons as claim 2.
Regarding claim 13, the combination of Sher and Demaj teaches the method of claim 12 as mentioned above, further comprising determining the delta-max by subtracting an original maximum of an original floating point range of the activation function of each layer of the original MLM from an updated maximum of an updated floating point range of the activation function of each layer of the updated MLM and determining the delta-min by subtracting an original minimum of the original floating point range of the activation function of each layer of the original MLM from an updated minimum of the updated floating point range of the activation function of each layer of the updated MLM (Sher para. [0055] recites “CNN quantization may comprise or consist of quantization of the weights and/or activations of the CNN. Each of these weights and activations are represented as arrays of real values” (i.e., activation functions can be quantized). Demaj para. [0067] recites “The method then includes steps 24 to 36 for adapting the quantization of the corrected weights of the layer to the values of these corrected weights. At step 24, the processing unit 106 determines a minimum weight and a maximum weight from at least one set of the corrected weights of the layer”. Demaj para. [0068] recites “At step 25, the processing unit 106 calculates quantization step sizes of the corrected weights of the layer. In embodiments, the processing unit 106 calculates a quantization step size for the set of the corrected weights of the layer using the minimum weight and maximum weight of the set of the corrected weights of the layer” (i.e., determining the difference between an original minimum value and an corrected, or updated minimum value, and determining the difference between an original maximum value and a corrected, or updated maximum value)).
Claim 14 is a method claim and its limitation is included in claim 5. Claim 14 is rejected for the same reasons as claim 5.
Regarding claim 16, Sher teaches a method (para. [0075] recites “FIG. 5 illustrates a process 500 for layer-by-layer training” (i.e. a method, or process)) for partial quantization to achieve full quantized model comprising: extracting a plurality of frozen quantized weights from each frozen layer of a quantized model, wherein the quantized model is generated by quantizing an original model (fig. 5 and para. [0075] recite “Process 500 may be used to quantize one or more layers of a CNN that has been previously trained with floating-point weights. In process 500, the layers to be quantized are quantized and frozen. It should be understood that, when a layer is "frozen," the weights of the layer are set at their current value, such that they are not modified any further during the training process” (i.e., quantizing a model and freezing model weights). Para. [0066] recites “optimum parameters 307 are saved for future quantization operations on inputs 205. Subsequently, weights quantization module 315 solves the MMSE problem to obtain the optimum parameters 317, and saves these optimum parameters 317 for future quantization operations on weights 215. It should be understood that, during operation of the QNN, weights quantization module 315 is unnecessary, since the final quantized integer values of weights 215 may simply be saved and retrieved during operation” (i.e., previously frozen weights can be retrieved, or extracted));
quantizing a respective activation function of each updated layer from the updated model [from a difference between the original model and the updated model] to generate a quantized activation function; quantizing a plurality of updated weights from an updated model to generate a plurality of updated quantized weights (para. [0055] recites “CNN quantization may comprise or consist of quantization of the weights and/or activations of the CNN. Each of these weights and activations are represented as arrays of real values”. Para. [0075] recites “In process 500, the layers to be quantized are quantized and frozen. It should be understood that, when a layer is "frozen," the weights of the layer are set at their current value, such that they are not modified any further during the training process” (i.e., weights and activation functions from layers of a machine learning model can be quantized and the weights can be frozen)),
wherein the updated model is generated by updating at least one layer of the original model to form at least one updated layer; and generating a new quantized model from the frozen quantized weights, the updated quantized weights and the respective quantized activation function of each updated layer (fig. 5 and para. [0076]-[0077] recite “the layers are quantized in order from the input to the output of the CNN. If another layer remains to be quantized (i.e., "Yes" in subprocess 520), process 500 proceeds to subprocess 530. Otherwise, if no more layers remain to be quantized (i.e., "No" in subprocess 520), process 500 proceeds to subprocess 570. In subprocess 530, input 205 of the current layer being quantized is quantized (e.g., by input quantization module 305). In subprocess 540, the model is retrained” (i.e., the model can be retrained, or updated, and an quantized model can be generated based on the quantized weights and activation functions of the model layers)).
However, while Sher teaches that activation functions of layers of a machine learning model can be quantized (see at least paragraph [0055]), Sher does not explicitly teach quantizing [a respective activation function of each updated layer from] the updated model from a difference between the original model and the updated model [to generate a quantized activation function], wherein the difference comprises an updated scale and an updated integer zero point, the updated scale modifying an original scale used to quantize the original model with a ratio proportional to a floating point range difference between the updated model and the original model, and the updated integer zero point modifying an original integer zero point used to quantize the original model with a scaled difference between the floating point range minimums of the updated model and the original model.
Demaj teaches quantizing [a respective activation function of each updated layer from] the updated model from a difference between the original model and the updated model [to generate a quantized activation function], wherein the difference comprises an updated scale and an updated integer zero point, the updated scale modifying an original scale used to quantize the original model with a ratio proportional to a floating point range difference between the updated model and the original model, and the updated integer zero point modifying an original integer zero point used to quantize the original model with a scaled difference between the floating point range minimums of the updated model and the original model (para. [0067] recites “The method then includes steps 24 to 36 for adapting the quantization of the corrected weights of the layer to the values of these corrected weights. Para. [0068] recites “At step 25, the processing unit 106 calculates quantization step sizes of the corrected weights of the layer” (i.e., determining the difference between an original model and the updated quantized model. Examiner notes that that Demaj teaches an updated zero point modifying an original zero point in at least paragraph [0082] and an updated scale modifying an original scale in at least paragraph [0083])).
See claim 1 for motivation to combine.
Claims 6-8, 15, and 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over Sher et al (US 20240193422 A1, herein Sher) in view of Demaj et al (US 20220164664 A1, herein Demaj), in further view of Gao et al (“Cloud-Edge Inference under Communication Constraints: Data Quantization and Early Exit”, herein Gao).
Regarding claim 6, the combination of Sher and Demaj teaches the method of claim 5 as mentioned above, and quantizing respective activation functions for layers of a MLM (Sher para. [0055] recites “CNN quantization may comprise or consist of quantization of the weights and/or activations of the CNN. Each of these weights and activations are represented as arrays of real values” (i.e., activation functions can be quantized)).
However, the combination of Sher and Demaj does not teach wherein the original integer zero point is determined by subtracting the original minimum of the original floating point range [of the activation function of each layer of the original MLM], divided by the original scale, from an integer minimum of an integer range [used for quantizing the respective activation function of the quantized MLM].
Gao teaches wherein the original integer zero point is determined by subtracting the original minimum of the original floating point range [of the activation function of each layer of the original MLM], divided by the original scale, from an integer minimum of an integer range [used for quantizing the respective activation function of the quantized MLM] (Gao section II A recites “We utilize the method of quantifying the data to be uploaded for data compression. Denote the maximum value as rmax and minimum value as rmin of partition point data, respectively (i.e., floating point range values). Then we determine qmax and qmin according to the quantization method (i.e., integer range values). The quantization scale S and quantization zero Z are derived as
S
=
r
m
a
x
-
r
m
i
n
q
m
a
x
-
q
m
i
n
and
Z
=
[
q
m
a
x
-
r
m
a
x
S
]” (Examiner notes that it would be obvious to one of ordinary skill in the art that determining the original zero point Z could use the formula from Gao based on the original floating point minimum, the original scale, and the integer minimum rather than the maximum range values. Examiner notes that wherein the integer range is “used for” quantizing an activation function of a quantized MLM is interpreted as the intended use of this integer range)).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine these teachings by modifying the scale and zero point calculation method from Demaj (which modifies Sher) to utilize the scale and zero point calculation method from Gao. Gao and Demaj are both directed to methods of model quantization based on a scale and zero point. One of ordinary skill in the art would recognize that the quantization scale and zero point as described in equations from section II of Gao could be updated using the equations from at least paragraphs [0082]-[0083] of Demaj.
Regarding claim 7, the combination of Sher, Demaj, and Gao teaches the method of claim 6 as mentioned above, wherein the original scale is determined by subtracting the original minimum of the original floating point range from the original maximum of the original floating point range, divided by the integer minimum of the integer range subtracted from an integer maximum of the integer range (Gao section II A recites “We utilize the method of quantifying the data to be uploaded for data compression. Denote the maximum value as rmax and minimum value as rmin of partition point data, respectively (i.e., floating point range values). Then we determine qmax and qmin according to the quantization method (i.e., integer range values). The quantization scale S and quantization zero Z are derived as
S
=
r
m
a
x
-
r
m
i
n
q
m
a
x
-
q
m
i
n
and
Z
=
[
q
m
a
x
-
r
m
a
x
S
]” (i.e., determining the original scale S based on the original floating point minimum and maximum and the integer minimum and maximum)).
Regarding claim 8, the combination of Sher and Demaj teaches the method of claim 1 as mentioned above, further comprising quantizing the plurality of weights from the at least one updated layer of the updated MLM with the original MLM, the quantized MLM, the updated MLM and a representative dataset collected on an Edge device, (Examiner notes that this limitation is interpreted in light of the 112(b) rejection of claim 8 such that that weights from a layer of an updated model (i.e., an updated layer) may be quantized. Given this interpretation, at least [0077] of Sher teaches quantizing weights from an updated layer of a model)).
However, the combination of Sher and Demaj does not teach wherein the representative dataset is a subset of a dataset used to train the updated MLM.
Gao teaches wherein the representative dataset is a subset of a dataset used to train the updated MLM (Gao section 1 para. 6 recites “we design the DNN partition for cloud-edge inference by jointly optimizing the quantization and exit-net to process partition point data”. Gao section II A recites “Quantization is applied in DNN partitioning as a way of data compression, which can reduce the amount of data that needs to be transmitted. After the data has been uploaded, we can inverse quantify the data and continue with the subsequent inference” (i.e., using the quantized model to perform transfer learning to an edge device for inference. Examiner notes that at least fig. 3 of Gao shows training a model on an edge device using a partition, or subset of training data)).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine these teachings by applying the quantization method from Sher in the cloud-edge computing system from Gao. Sher and Gao are both directed to model quantization methods, and Sher states in at least paragraph [0054] that “different quantization schemes can be utilized”. One of ordinary skill in the art would accordingly recognize that the quantization scheme from Sher could be adapted for cloud-edge partitioning using a system such as the one taught by Gao.
Regarding claim 15, the combination of Sher and Demaj teaches the method of claim 14 as mentioned above, and quantizing respective activation functions for layers of a MLM (Sher para. [0055] recites “CNN quantization may comprise or consist of quantization of the weights and/or activations of the CNN. Each of these weights and activations are represented as arrays of real values” (i.e., activation functions can be quantized)).
However, the combination of Sher and Demaj does not teach wherein the original integer zero point is determined by subtracting from an integer minimum of an integer range [used for quantizing the respective activation function of the quantized MLM], a quantized minimum of a quantized floating point range [of the activation function of each layer of the quantized MLM], divided by the original scale and the original scale is determined by subtracting the original minimum of the original floating point range from the original maximum of the original floating point range, divided by the integer minimum of the integer range subtracted from an integer maximum of the integer range.
Gao teaches wherein the original integer zero point is determined by subtracting from an integer minimum of an integer range [used for quantizing the respective activation function of the quantized MLM], a quantized minimum of a quantized floating point range of the activation function of each layer of the quantized MLM, divided by the original scale (Gao section II A recites “We utilize the method of quantifying the data to be uploaded for data compression. Denote the maximum value as rmax and minimum value as rmin of partition point data, respectively (i.e., floating point range values). Then we determine qmax and qmin according to the quantization method (i.e., integer range values). The quantization scale S and quantization zero Z are derived as
S
=
r
m
a
x
-
r
m
i
n
q
m
a
x
-
q
m
i
n
and
Z
=
[
q
m
a
x
-
r
m
a
x
S
]” (Examiner notes that it would be obvious to one of ordinary skill in the art that determining the original zero point Z could use the formula from Gao based on the original floating point minimum, the original scale, and the integer minimum rather than the maximum range values. Examiner also notes that wherein the integer range is “used for” quantizing an activation function of a quantized MLM is interpreted as the intended use of this integer range)),
and the original scale is determined by subtracting the original minimum of the original floating point range from the original maximum of the original floating point range, divided by the integer minimum of the integer range subtracted from an integer maximum of the integer range (Gao section II A recites “We utilize the method of quantifying the data to be uploaded for data compression. Denote the maximum value as rmax and minimum value as rmin of partition point data, respectively (i.e., floating point range values). Then we determine qmax and qmin according to the quantization method (i.e., integer range values). The quantization scale S and quantization zero Z are derived as
S
=
r
m
a
x
-
r
m
i
n
q
m
a
x
-
q
m
i
n
and
Z
=
[
q
m
a
x
-
r
m
a
x
S
]” (i.e., determining the original scale S based on the original floating point minimum and maximum and the integer minimum and maximum)).
See claim 6 for motivation to combine.
Regarding claim 17, the combination of Sher and Demaj teaches the method of claim 16 as mentioned above.
However, the combination of Sher and Demaj does not teach training the updated model with a new dataset stored on an Edge device.
Gao teaches training the updated model with a new dataset stored on an Edge device (Gao section 1 para. 6 recites “we design the DNN partition for cloud-edge inference by jointly optimizing the quantization and exit-net to process partition point data”. Gao section II A recites “Quantization is applied in DNN partitioning as a way of data compression, which can reduce the amount of data that needs to be transmitted. After the data has been uploaded, we can inverse quantify the data and continue with the subsequent inference” (i.e., using the quantized model to perform transfer learning to an edge device for inference. Examiner notes that at least fig. 3 of Gao shows training and updating a model on an edge device)).
See claim 8 for motivation to combine.
Claim 18 is a method claim and its limitation is included in claim 8. Claim 18 is rejected for the same reasons as claim 8.
Regarding claim 19, the combination of Sher and Demaj teaches the method of claim 16 as mentioned above.
However, the combination of Sher and Demaj does not teach wherein the updated model adapts to the new dataset through transfer learning.
Gao teaches wherein the updated model adapts to the new dataset through transfer learning (Gao section 1 para. 6 recites “we design the DNN partition for cloud-edge inference by jointly optimizing the quantization and exit-net to process partition point data”. Gao section II A recites “Quantization is applied in DNN partitioning as a way of data compression, which can reduce the amount of data that needs to be transmitted. After the data has been uploaded, we can inverse quantify the data and continue with the subsequent inference” (i.e., using the quantized model to perform transfer learning to an edge device for inference. Examiner notes that at least fig. 3 of Gao shows updating a model on an edge device)).
See claim 8 for motivation to combine.
Regarding claim 20, the combination of Sher and Demaj teaches the method of claim 16 as mentioned above.
However, the combination of Sher and Demaj does not teach inferencing with a neural processing unit and the new quantized model to perform transfer learning on an Edge device.
Gao teaches inferencing with a neural processing unit and the new quantized model to perform transfer learning on an Edge device (Gao section 1 para. 6 recites “we design the DNN partition for cloud-edge inference by jointly optimizing the quantization and exit-net to process partition point data”. Gao section II A recite “Quantization is applied in DNN partitioning as a way of data compression, which can reduce the amount of data that needs to be transmitted. After the data has been uploaded, we can inverse quantify the data and continue with the subsequent inference” (i.e., using the quantized model to perform transfer learning to an edge device for inference)).
See claim 8 for motivation to combine.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20200293870 A1 (Isikdogan et al) teaches a method for facilitating partially-frozen neural networks as a combination of frozen and trainable weights.
US 20210248456 A1 (Guo et al) teaches methods for optimizing tensor quantization in one or more layers of an AI model.
US 20230214713 A1 (Choi et al) teaches a method for quantizing neural network models in a federated learning system.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LEAH M FEITL whose telephone number is (571) 272-8350. The examiner can normally be reached on M-F 0900-1700 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Viker Lamardo can be reached on (571) 270-5871. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/L.M.F./ Examiner, Art Unit 2147
/VIKER A LAMARDO/Supervisory Patent Examiner, Art Unit 2147
Prosecution Insights