DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-14, 20 provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-5, 7-10, 13, 15-20 of corresponding application No. 18/739,838 in view of Yu et al. (US 20250225395 A1, hereinafter “Yu”). The corresponding application (18/739,838) discloses similar limitations, shown in the table below, to the instant application (18/742,499).
The differences between both applications appear to be the fifth and last limitation stated in claim 1, 20 of the instant application (18/742,499) stating "obtaining first output values of the first NN model with respect to a first retraining data” and “based on the first output values of the first NN model.” The limitations are taught by Yu (0067, 0075, 0088-0089, 0096 see 35 USC 103 rejection below). It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the similar limitations taught in the corresponding application (18/739,838) with obtaining output values of a first NN model and retraining using the first NN model’s outputs by Yu in order to improve accuracy of a quantized model. ([Yu, 0003, 0065, 0083])
For Claim 2, the instant application (18/742,499) mentions “difference between a first output value of a first graph module of the first NN model and a second output value of a second graph module of the second NN model, and wherein the first graph module of the first NN model corresponds to the second graph module of the second NN model” and the corresponding application (18/739,838) mentions “a difference between an actual result value Ytruth and an output value Yout of each of the plurality of graph modules.” The actual result value Ytruth corresponds to a first output value of a first graph module of the first NN model and an output value Yout corresponds to a second output value of a second graph module of the second NN model. It is obvious that the first graph module of first NN model corresponds to the second graph module of the second NN model because it gives a way to measure how different or similar the two NN models by comparing their corresponding outputs between modules and utilizing them to train the second NN model to have similar outputs as the first NN model, especially since second NN model is a quantization of the first NN model.
This is a provisional nonstatutory double patenting rejection.
Instant Application: 18/742,499
Corresponding Application: 18/739,838
Claim 1:
A method comprising: adding a plurality of markers to each of a plurality of graph modules in a first neural network (NN) model in a form of a directed acyclic graph (DAG); generating calibration data by collecting input values and output values of each of the plurality of graph modules by using the plurality of markers; determining, based on the calibration data, a scale value and an offset value applicable to the first NN model; generating, based on the scale value and the offset value, a second NN model including at least one parameter including at least one weight parameter in an integer format through quantization; obtaining first output values of the first NN model with respect to a first retraining data; and updating, based on the first output values of the first NN model, the at least one weight parameter included in the second NN model by performing a quantization-aware retraining on the second NN model.
Claim 1:
A method comprising: adding a plurality of markers to each of a plurality of graph modules in a first neural network (NN) model in a form of a directed acyclic graph (DAG); generating calibration data by collecting input values and output values of each of the plurality of graph modules by using the plurality of markers; determining, based on the calibration data, a scale value and an offset value applicable to the first NN model; generating, based on the scale value and the offset value, a second NN model including at least one parameter including at least one weight parameter in an integer format through quantization; and updating the at least one parameter included in the second NN model by performing a quantization-aware retraining on the second NN model.
Claim 2:
The method of claim 1, wherein the updating the at least one weight parameter included in the second NN model further comprises: updating the at least one weight parameter of each of the plurality of graph modules included in the second NN model by using a gradient descent technique so that a loss resulting from changing parameters of the first NN model in the quantization is minimized for each of the plurality of graph modules, wherein the loss represents a difference between a first output value of a first graph module of the first NN model and a second output value of a second graph module of the second NN model, and wherein the first graph module of the first NN model corresponds to the second graph module of the second NN model.
Claim 2:
The method of claim 1, wherein the updating the at least one parameter included in the second NN model by performing the quantization-aware retraining on the second NN model further comprises: updating the at least one parameter of each of the plurality of graph modules included in the second NN model by using a gradient descent technique so that a loss resulting from changing parameters of the first NN model due to the quantization is minimized for each of the plurality of graph modules, wherein the loss represents a difference between an actual result value Ytruth and an output value Yout of each of the plurality of graph modules.
Claim 3:
The method of claim 1, wherein in the performing the quantization-aware retraining on the second NN model further comprises: updating the at least one weight parameter by subtracting a loss resulting from changing parameters of the first NN model due to the quantization.
Claim 3:
The method of claim 2, wherein in the performing the quantization-aware retraining on the second NN model further comprises: updating the at least one parameter by subtracting, from the at least one parameter, the loss resulting from changing the parameters of the first NN model due to the quantization.
Claim 4:
The method of claim 1, wherein in the performing the quantization-aware retraining on the second NN model further comprises: determining, based on at least one user option or retraining completion time, a degree of the updating the at least one weight parameter.
Claim 4:
The method of claim 3, wherein in the performing the quantization-aware retraining on the second NN model further comprises: determining, based on at least one user option or retraining completion time, a degree of the updating the at least one parameter.
Claim 5:
The method of claim 2, wherein the quantization-aware retraining of the second NN model is terminated when the loss reaches a predetermined threshold or exceeds a predetermined execution time.
Claim 5:
The method of claim 2, wherein the quantization-aware retraining of the second NN model is terminated when the loss reaches a predetermined threshold or exceeds a predetermined execution time.
Claim 6:
The method of claim 1, wherein the updating the at least one weight parameter included in the second NN model by performing the quantization-aware retraining on the second NN model further comprises: adding a loss change calculation function to a forward calculation of each of a plurality of graph modules included in the second NN model, corresponding to a quantization module added to each of the plurality of graph modules included in the second NN model; and verifying output values of each of the plurality of graph modules included in the second NN model during a backward computation for changes in each of the at least one weight parameter.
Claim 7:
The method of claim 1, wherein the updating the at least one parameter included in the second NN model by performing the quantization-aware retraining on the second NN model further comprises: adding a loss change calculation function to a forward computation of each of the plurality of graph modules included in the second NN model, corresponding to a quantization module added to each of the plurality of graph modules; and verifying output values of each of the plurality of graph modules during a backward computation for changes in each of the at least one parameter.
Claim 7:
The method of claim 6, wherein the loss change calculation function has results of the forward calculation unaffected by a loss change calculation and preserves original equations removed by round and clip operations included in the quantization module during the backward computation.
Claim 8:
The method of claim 7, wherein the loss change calculation function has results of the forward computation unaffected by a loss change calculation and preserves original equations removed by round and clip operations included in the quantization module during the backward computation.
Claim 8:
The method of claim 6, wherein the loss change calculation function is represented by a first detach function for an input feature map parameter and a second detach function for a weight parameter:
PNG
media_image1.png
58
184
media_image1.png
Greyscale
and
PNG
media_image2.png
49
147
media_image2.png
Greyscale
, where x denotes the input feature map parameter of each of the plurality of graph modules included in the second NN model, sx denotes a scale value for the input feature map parameter, o denotes an offset value for the input feature map parameter, w denotes the weight parameter of each of the plurality of graph modules included in the second NN model, and sw denotes a scale value for the weight parameter.
Claim 9:
The method of claim 7, wherein the loss change calculation function is represented by a first detach function for an input feature map parameter and a second detach function for a weight parameter:
PNG
media_image1.png
58
184
media_image1.png
Greyscale
and
PNG
media_image2.png
49
147
media_image2.png
Greyscale
, where x denotes the input feature map parameter of each of the plurality of graph modules included in the second NN model, sx denotes a scale value for the input feature map parameter, o denotes an offset value for the input feature map parameter, w denotes the weight parameter of each of the plurality of graph modules included in the second NN model, and sw denotes a scale value for the weight parameter.
Claim 9:
The method of claim 8, wherein the updating the at least one weight parameter included in the second NN model by performing the quantization-aware retraining on the second NN model further comprises: replacing
PNG
media_image3.png
70
856
media_image3.png
Greyscale
of each of the plurality of graph modules to which the quantization module is added with
PNG
media_image4.png
71
858
media_image4.png
Greyscale
using the loss change calculation function.
Claim 10:
The method of claim 9, wherein the updating the at least one parameter included in the second NN model by performing the quantization-aware retraining on the second NN model further comprises: replacing
PNG
media_image3.png
70
856
media_image3.png
Greyscale
of each of the plurality of graph modules to which the quantization module is added with
PNG
media_image4.png
71
858
media_image4.png
Greyscale
using the loss change calculation function.
Claim 10:
The method of claim 1, wherein the scale value and the offset value is obtained by an equation below,
PNG
media_image5.png
84
618
media_image5.png
Greyscale
where max denotes a maximum value among the input values and output values collected for the calibration data, min denotes a minimum value among the input values and output values collected for the calibration data, and bitwidth denotes a target quantization bitwidth.
Claim 13:
The method of claim 1, wherein the scale value and the offset value is obtained by an equation below,
PNG
media_image5.png
84
618
media_image5.png
Greyscale
where max denotes a maximum value among the input values and output values collected for the calibration data, min denotes a minimum value among the input values and output values collected for the calibration data, and bitwidth denotes a target quantization bitwidth.
Claim 11:
The method of claim 1, wherein a convolution operation in the first NN model is expressed as:
PNG
media_image6.png
45
520
media_image6.png
Greyscale
where feature_infp denotes an input feature map parameter in a form of floating-point, weightfp denotes a weight parameter in a form of floating-point, of denotes an offset value for an input feature map, sf denotes a scale value for the input feature map, sw denotes a scale value for a weight, and
PNG
media_image7.png
37
25
media_image7.png
Greyscale
denotes a round and clip operation.
Claim 14:
The method of claim 1, wherein a convolution operation in the first NN model is expressed as:
PNG
media_image6.png
45
520
media_image6.png
Greyscale
where feature_infp denotes an input feature map parameter in a form of floating-point, weightfp denotes a weight parameter in a form of floating-point, of denotes an offset value for an input feature map, sf denotes a scale value for the input feature map, sw denotes a scale value for a weight, and
PNG
media_image7.png
37
25
media_image7.png
Greyscale
denotes a round and clip operation.
Claim 12:
The method of claim 1, wherein a convolution operation in the second NN model is expressed as:
PNG
media_image8.png
46
691
media_image8.png
Greyscale
where feature_outint denotes an output feature map parameter in a form of an integer, feature_inint denotes an input feature map parameter in a form of an integer and weightint denotes a weight parameter in a form of an integer.
Claim 15:
The method of claim 1, wherein a convolution operation in the second NN model is expressed as:
PNG
media_image8.png
46
691
media_image8.png
Greyscale
where feature_outint denotes an output feature map parameter in a form of an integer, feature_inint denotes an input feature map parameter in a form of an integer and weightint denotes a weight parameter in a form of an integer.
Claim 13:
The method of claim 1, wherein the at least one weight parameter and at least one input feature map parameter of the first NN model are in a floating-point format with a length of 16 bits to 32 bits.
Claim 16:
The method of claim 1, wherein at least one weight parameter and at least one input feature map parameter of the first NN model are in a floating-point format with a length of 16 bits to 32 bits.
Claim 14:
The method of claim 1, wherein the second NN model includes the at least one weight parameter and an input feature map parameter in an integer (INT) format with a length of 2 bits to 8 bits.
Claim 17:
The method of claim 1, wherein the second NN model includes the at least one weight parameter and at least one input feature map parameter in an integer (INT) format with a length of 2 bits to 8 bits.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 15, and 20 rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1
According to the first part of the analysis, in the instant case, Claims 1 and 15 is directed to a method. Thus, each of the claims falls within one of the four statutory categories (i.e., process, machine, manufacture, or composition of matter).
However, Claims 20 is directed to an article of manufacture (“non-volatile computer readable storage medium”) that is signal per se. The specification does not provide a clear definition of whether the claimed medium is limited to statutory or non-transitory elements. Therefore, under the broadest reasonable interpretation, the claim element “non-volatile computer readable storage medium” is not limited to statutory elements and can be considered non-statutory. Claim 20 is rejected because it does not fall within one of the four statutory categories (i.e., process, machine, manufacture, or composition of matter).
Regarding Claim 1
2A Prong 1: adding a plurality of markers to each of a plurality of graph modules in a first neural network (NN) model in a form of a directed acyclic graph (DAG); (This step for drawing a DAG and adding markers in the DAG structure of a model is practically performable in the human mind and is understood to be a recitation of a mental process with the aid of pen and paper (i.e., evaluation).)
generating calibration data by collecting input values and output values of each of the plurality of graph modules by using the plurality of markers(This step for generating calibration data is based on mathematical equations [Page 43, Equation 1; Page 44, Equation 2; Page 46, Equation 3].)
obtaining first output values of the first NN model with respect to a first retraining data; (This step for obtaining output values a is based on mathematical equations [Page 60, Lines 20-27].)
determining, based on the calibration data, a scale value and an offset value applicable to the first NN model; (This step for determining a scale value and an offset value is based on mathematical equations [Page 43, Line 30].)
generating, based on the scale value and the offset value, a second NN model including at least one parameter including at least one weight parameter in an integer format through quantization; (This step for generating a second NN is based on mathematical equations Para [Page 43, Line 30; Page 44, Line 27; Page 46, Line 2].)
and updating, based on the first output values of the first NN model, the at least one weight parameter included in the second NN model by performing a quantization-aware retraining on the second NN model. (This step for updating weight parameter in a NN model by quantization-aware retraining is based on mathematical equations [Page 6, Lines 9-27].)
2A Prong 2 & 2B: There are no additional elements
Regarding Claim 15
2A Prong 1:
generating, based on a first neural network (NN) model including at least one floating-point parameter, a second NN model including at least one integer parameter by performing quantization; (This step for generating a second NN based on a first NN is based on mathematical equations [Page 43, Line 30; Page 44, Line 27; Page 46, Line 2].)
and performing quantization-aware retraining of the second NN model by using the retraining data, based on output values of the first NN model for the retraining data. (This step for performing quantization-aware retraining is based on mathematical equations [Page 59, Line 21; Page 60, Lines 20-27; Page 61, Lines 2-10; Page 62-63].)
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
performing retraining, based on label values of retraining data, on the first NN model; (Training a machine learning model is understood as mere instructions to implement an abstract idea (e.g., generate inferences) on a computer - see MPEP 2106.05(f).))
The additional elements as disclosed above alone or in combination do not integrate the judicial exception into practical application as they are generic computer functions that are implemented to perform the disclosed abstract idea above.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements: performing retraining, based on label values of retraining data, on the first NN model; (Training a machine learning model is understood as mere instructions to implement an abstract idea (e.g., generate inferences) on a computer - see MPEP 2106.05(f).))
The additional elements as disclosed above in combination of the abstract idea are not sufficient to amount to significantly more than the judicial exception as they are generic computer functions that are implemented to perform the disclosed abstract idea above.
Regarding Claim 20: see the rejection of Claim 1 above. Same rationale applies.
2A Prong 2 & 2B: The claim recites another additional element “A non-volatile computer-readable storage medium storing instructions, the instructions, when executed by one or more processors, causing the one or more processors to perform steps comprising” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claim 2
2A Prong 1:
wherein the updating the at least one weight parameter included in the second NN model further comprises: updating the at least one weight parameter of each of the plurality of graph modules included in the second NN model by using a gradient descent technique so that a loss resulting from changing parameters of the first NN model in the quantization is minimized for each of the plurality of graph modules, wherein the loss represents a difference between a first output value of a first graph module of the first NN model and a second output value of a second graph module of the second NN model, and wherein the first graph module of the first NN model corresponds to the second graph module of the second NN model. (This step for updating a weight parameter using gradient descent is based on mathematical equation [Page 59, Equation 4])
2A Prong 2 & 2B: The claim does not recite any additional elements.
Regarding Claim 3
2A Prong 1: updating the at least one weight parameter by subtracting a loss resulting from changing parameters of the first NN model due to the quantization. ((This step for updating weight parameter by subtracting a loss is based on mathematical equation [Page 59, Equation 4])
2A Prong 2 & 2B: wherein in the performing the quantization-aware retraining on the second NN model further comprises: (Training a machine learning model is understood as mere instructions to implement an abstract idea (e.g., generate inferences) on a computer - see MPEP 2106.05(f).))
Regarding Claim 4
2A Prong 1: determining, based on at least one user option or retraining completion time, a degree of the updating the at least one weight parameter. (This step for determining a degree of updating a weight parameter based on user option or retraining completion time is practically performable in the human mind and is understood to be a recitation of a mental process (i.e., evaluation).)
2A Prong 2 & 2B: wherein in the performing the quantization-aware retraining on the second NN model further comprises: (Training a machine learning model is understood as mere instructions to implement an abstract idea (e.g., generate inferences) on a computer - see MPEP 2106.05(f).))
Regarding Claim 5
2A Prong 1: None
2A Prong 2 & 2B: wherein the quantization-aware retraining of the second NN model is terminated when the loss reaches a predetermined threshold or exceeds a predetermined execution time. (The specification of when to terminate retraining used is understood to be a field of use limitation – See MPEP 2106.05(h).)
Regarding Claim 6
2A Prong 1: wherein the updating the at least one weight parameter included in the second NN model by performing the quantization-aware retraining on the second NN model further comprises: adding a loss change calculation function to a forward calculation of each of a plurality of graph modules included in the second NN model, corresponding to a quantization module added to each of the plurality of graph modules included in the second NN model; (This step for updating weight parameters comprises adding a loss change calculation function is based on mathematical equations [Page 6, Lines 12-19])
and verifying output values of each of the plurality of graph modules included in the second NN model during a backward computation for changes in each of the at least one weight parameter. (This step for verifying output values during backward computation for changes is based on mathematical equations [Page 60, Lines 25-27; Page 63, Equation 6])
2A Prong 2 & 2B: The claim does not recite any additional elements.
Regarding Claim 7
2A Prong 1: wherein the loss change calculation function has results of the forward calculation unaffected by a loss change calculation and preserves original equations removed by round and clip operations included in the quantization module during the backward computation. (This step for a loss change calculation function is based on mathematical equations [Page 6, Lines 12-19])
2A Prong 2 & 2B: The claim does not recite any additional elements.
Regarding Claim 8
2A Prong 1: wherein the loss change calculation function is represented by a first detach function for an input feature map parameter and a second detach function for a weight parameter:
PNG
media_image1.png
58
184
media_image1.png
Greyscale
and
PNG
media_image2.png
49
147
media_image2.png
Greyscale
, where x denotes the input feature map parameter of each of the plurality of graph modules included in the second NN model, sx denotes a scale value for the input feature map parameter, o denotes an offset value for the input feature map parameter, w denotes the weight parameter of each of the plurality of graph modules included in the second NN model, and sw denotes a scale value for the weight parameter. (This step for loss change function is based on mathematical equations listed above)
2A Prong 2 & 2B: The claim does not recite any additional elements.
Regarding Claim 9
2A Prong 1: wherein the updating the at least one weight parameter included in the second NN model by performing the quantization-aware retraining on the second NN model further comprises: replacing
PNG
media_image9.png
55
670
media_image9.png
Greyscale
of each of the plurality of graph modules to which the quantization module is added with
PNG
media_image10.png
57
704
media_image10.png
Greyscale
using the loss change calculation function. (This step for updating a weight parameter is based on mathematical equations listed above)
2A Prong 2 & 2B: The claim does not recite any additional elements.
Regarding Claim 10
2A Prong 1: wherein the scale value and the offset value is obtained by an equation below,
PNG
media_image11.png
64
121
media_image11.png
Greyscale
where max denotes a maximum value among the input values and output values collected for the calibration data, min denotes a minimum value among the input values and output values collected for the calibration data, and bitwidth denotes a target quantization bitwidth. (This step for obtaining a scale value and offset is based on mathematical equations listed above)
2A Prong 2 & 2B: The claim does not recite any additional elements.
Regarding Claim 11
2A Prong 1: wherein a convolution operation in the first NN model is expressed as:
PNG
media_image6.png
45
520
media_image6.png
Greyscale
where feature_infp denotes an input feature map parameter in a form of floating-point, weightfp denotes a weight parameter in a form of floating-point, of denotes an offset value for an input feature map, sf denotes a scale value for the input feature map, sw denotes a scale value for a weight, and
PNG
media_image7.png
37
25
media_image7.png
Greyscale
denotes a round and clip operation. (This step for a convolution operation is based on mathematical equation listed above)
2A Prong 2 & 2B: The claim does not recite any additional elements.
Regarding Claim 12
2A Prong 1: wherein a convolution operation in the second NN model is expressed as:
PNG
media_image12.png
39
569
media_image12.png
Greyscale
where feature_outint denotes an output feature map parameter in a form of an integer, feature_inint denotes an input feature map parameter in a form of an integer and weightint denotes a weight parameter in a form of an integer. (This step for a convolution operation is based on mathematical equation listed above)
2A Prong 2 & 2B: The claim does not recite any additional elements.
Regarding Claim 13
2A Prong 1: None
2A Prong 2 & 2B: wherein the at least one weight parameter and at least one input feature map parameter of the first NN model are in a floating-point format with a length of 16 bits to 32 bits. (The specification of parameters used is understood to be a field of use limitation – See MPEP 2106.05(h).)
Regarding Claim 14
2A Prong 1: None
2A Prong 2 & 2B: wherein the second NN model includes the at least one weight parameter and an input feature map parameter in an integer (INT) format with a length of 2 bits to 8 bits. (The specification of parameters used is understood to be a field of use limitation – See MPEP 2106.05(h).)
Regarding Claim 16
2A Prong 1: when a difference between an output value of each of a plurality of graph modules of the first NN model and an output value of each of the plurality of graph modules of the second NN model that is corresponding to the each of the plurality of graph modules of the first NN model is minimal, updating at least one weight parameter of the each of the plurality of graph modules of the second NN model to at least one weight parameter as is in a current state. (This step for updating a weight parameter when the different between output values is minimal is practically performable in the human mind and is understood to be a recitation of a mental process (i.e., evaluation).)
2A Prong 2 & 2B: wherein the performing the quantization-aware retraining of the second NN model further comprises: (Training a machine learning model is understood as mere instructions to implement an abstract idea (e.g., generate inferences) on a computer - see MPEP 2106.05(f).))
Regarding Claim 17
2A Prong 1: None
2A Prong 2 & 2B: wherein a first weight parameter and a first input feature map parameter of the first NN model are in a floating-point format with a length of 16 bits to 32 bits, and wherein a second weight parameter and a second input feature map parameter of the second NN model are in an integer (INT) format with a length of 2 bits to 8 bits. (The specification of parameters used is understood to be a field of use limitation – See MPEP 2106.05(h).)
Regarding Claim 18
2A Prong 1: adding a loss change calculation function to a forward calculation of each of the plurality of graph modules included in the second NN model, corresponding to a quantization module added to each of the plurality of graph modules included in the second NN model; (This step for adding a loss change calculation function is based on mathematical equations [Page 6, Lines 12-19])
and verifying output values of each of the plurality of graph modules included in the second NN model during a backward computation for changes in each of the at least one weight parameter, (This step for verifying output values during backward computation for changes is based on mathematical equations [Page 60, Lines 25-27; Page 63, Equation 6])
wherein the loss change calculation function has results of the forward calculation unaffected by a loss change calculation and preserves original equations removed by round and clip operations included in the quantization module during the backward computation. (This step for a loss change calculation function is based on mathematical equations [Page 6, Lines 12-19])
2A Prong 2 & 2B: wherein the performing the quantization-aware retraining of the second NN model to update at least one weight parameter included in the second NN model further comprises: (Training a machine learning model is understood as mere instructions to implement an abstract idea (e.g., generate inferences) on a computer - see MPEP 2106.05(f).))
Regarding Claim 19
2A Prong 1: wherein the loss change calculation function is represented by a first detach function for an input feature map parameter and a second detach function for a weight parameter:
PNG
media_image13.png
48
171
media_image13.png
Greyscale
and
PNG
media_image14.png
49
146
media_image14.png
Greyscale
, where x denotes the input feature map parameter of each of the plurality of graph modules included in the second NN model, sx denotes a scale value for the input feature map parameter, o denotes an offset value for the input feature map parameter, w denotes the weight parameter of each of the plurality of graph modules included in the second NN model, and sw denotes a scale value for the weight parameter,
and wherein the updating the at least one weight parameter included in the second NN model by performing the quantization-aware retraining on the second NN model further comprises: replacing
PNG
media_image9.png
55
670
media_image9.png
Greyscale
of each of the plurality of graph modules to which the quantization module is added with
PNG
media_image10.png
57
704
media_image10.png
Greyscale
using the loss change calculation function. (This step for loss change calculation function and updating a weight parameter is based on mathematical equations listed above)
2A Prong 2 & 2B: The claim does not recite any additional elements.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 15-17 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Yu et al. (US 20250225395 A1, hereinafter "Yu").
Regarding Claim 15
Yu discloses: A method comprising: generating, based on a first neural network (NN) model including at least one floating-point parameter, a second NN model including at least one integer parameter by performing quantization; ([Yu, 0083, 0091, 0096] discloses generating sparse floating-point model (i.e first NN model) and sparse quantized model (i.e second NN model) by quantization, where the quantized model is in integer format)
performing retraining, based on label values of retraining data, on the first NN model; ([Yu, 0092-0093] performing modifications (i.e retraining) on sparse floating-point model (i.e first NN model), based on performance metrics, such as classifcation accuracy and mean sqaured error, which includes using label values of input data)
and performing quantization-aware retraining of the second NN model by using the retraining data, based on output values of the first NN model for the retraining data. ([Yu, 0067, 0072, 0075, 0088, 0096] performing quantization-aware retraining on the sparse quantized model by using input data (i.e retraining data), based on comparing output values of the sparse floating-point model (i.e first NN model) and the sparse quantized model (i.e second NN model) to calculate losses. Specs [Page 65, Para 1-2] applicant describes quantization-aware retraining as general retraining with quantization considered and using losses)
Regarding Claim 16
when a difference between an output value of each of a plurality of graph modules of the first NN model and an output value of each of the plurality of graph modules of the second NN model that is corresponding to the each of the plurality of graph modules of the first NN model is minimal, updating at least one weight parameter of the each of the plurality of graph modules of the second NN model to at least one weight parameter as is in a current state. ([Yu, 0062, 0066-0067, 0072, 0075, Fig. 2, 0104-0106] discloses modifying and adjusting (i.e updating) the weight parameters of the sparse quantized model (i.e second NN model) by calculating loss resulted from quantizing (i.e changing parameters) of the sparse floating-point model (i.e first NN model) so that loss is minimized for each layer (i.e graph modules) in the sparse quantized model. The loss includes using loss functions for feature distillations and partial loss of calibration data, which determines a difference output of layers (i.e first graph module) of sparse floating-point model (i.e first NN model) and output of layers (i.e second graph module) of sparse quantized model, see [0094] for an example loss function)
Regarding Claim 17
Yu discloses: wherein a first weight parameter and a first input feature map parameter of the first NN model are in a floating-point format with a length of 16 bits to 32 bits, ([Yu, 0083] discloses sparse floating-point model (i.e first NN model) in higher-precision (i.e floating-point) data format with length half binary16 or FP32 (i.e 16 bits to 32 bits). [0066, 0072] discloses the model having weights parameters and feature maps (i.e input feature map parameter))
and wherein a second weight parameter and a second input feature map parameter of the second NN model are in an integer (INT) format with a length of 2 bits to 8 bits. ([Yu, 0083] discloses sparse quantized model (i.e second NN model) in lower-precision (i.e integer-point) data format with length 8-bit integer or 16-bit integer (i.e 16 bits to 32 bits). [0066, 0072] discloses the model having weights parameters and feature maps (i.e input feature map parameter))
Claim Rejections - 35 USC § 103
The following is a quotation of pre-AIA 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under pre-AIA 35 U.S.C. 103(a) are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-3, 13-14, 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jeong et al. (US 20240256843 A1, hereinafter “Jeong”), in view of Jang et al. (US 20220206698 A1, hereinafter “Jang”) and in view of Yu et al. (US 20250225395 A1, hereinafter “Yu”).
Regarding Claim 1
Jeong discloses: A method comprising: generating calibration data by collecting input values and output values of each of the plurality of graph modules ([Jeong, 0072; Fig. 25, 0200] discloses generating feature map data (i.e calibration data) by collecting input feature maps (i.e input values) and output feature maps (i.e output values) of each graph layers (i.e modules))
determining, based on the calibration data, a scale value and an offset value applicable to the first NN model; ([Jeong, 0074, 0089-0090; Fig. 25, 0201] discloses, based on the feature map data (i.e calibration data), a scale value and shifting or zero-point (i.e offset) value applicable to the first NN model. See [0088, Equation 3] for the affine transformation using both scale value and shifting or zero-point value in quantitation)
generating, based on the scale value and the offset value, a second NN model including at least one parameter including at least one weight parameter in an integer format through quantization; ([Jeong, 0082, 0087, 0089-0090; Fig. 25, 0202-0208] discloses generating, based on scale value and shifting or zero-point (i.e offset) value, as seen in [0088, Equation 3] for affine transformation in quantitation, a third or quantized NN model (i.e second NN model) through quantization where the weight parameters are converted to integer)
Jeong does not explicitly disclose: adding a plurality of markers to each of a plurality of graph modules in a first neural network (NN) model in a form of a directed acyclic graph (DAG); the plurality of markers;
However, Jang discloses: adding a plurality of markers to each of a plurality of graph modules in a first neural network (NN) model in a form of a directed acyclic graph (DAG); ([Jang, 0080-0081] discloses adding checkpoints (i.e marker) to at least one layer in the neural network (i.e first neural network model), which is a DAG as seen in Fig. 4.)
the generating calibration data by collecting ([Jang, Fig. 4, 0080-0081, 0087-0088] discloses generating intermediate values by collecting immediate result values from calculating results (i.e output values) of each graph layer by using checkpoints)
Jeong and Jang are analogous art to the present invention because they are from the same field of endeavor directed to machine learning.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the use calibration data to generate a quantized model disclosed by Jeong with using markers to collect calibration data between layers by Jang. One of ordinary skill in the art would have been motivated to make this modification in order to improve learning rate and utilization by using markers. ([Jang, 0092])
Jeong in view of Jang does not explicitly disclose: obtaining first output values of the first NN model with respect to a first retraining data;
and updating, based on the first output values of the first NN model, the at least one weight parameter included in the second NN model by performing a quantization-aware retraining on the second NN model.
However, Yu discloses: generating([Yu, 0064, 0075, 0083] discloses generating a sparse quantized model (i.e second NN model) where the weight parameters in integer format through quantization)
obtaining first output values of the first NN model with respect to a first retraining data; ([Yu, 0067, 0075, 0088-0089, 0096] discloses obtaining output values of sparse floating-point model (i.e first NN model) with respect to input data (i.e first retraining data) in order to compare with output values of the sparse quantized model (i.e second NN model))
and updating, based on the first output values of the first NN model, the at least one weight parameter included in the second NN model by performing a quantization-aware retraining on the second NN model. ([Yu, 0067, 0072, 0075, 0088, 0096] updating, based on comparing output values of the sparse floating-point model (i.e first NN model) and the sparse quantized model (i.e second NN model) to calculate losses, weights in the sparse quantized model by performing quantization-aware retraining on the sparse quantized model. Specs [Page 65, Para 1-2] applicant describes quantization-aware retraining as retraining with quantization considered and using losses)
Jeong, Jang, and Yu are analogous art to the present invention because they are from the same field of endeavor directed to machine learning.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the method of generating a quantized neural network disclosed by Jeong in view of Jang with retraining the quantized model by Yu. One of ordinary skill in the art would have been motivated to make this modification in order to improve accuracy of a quantized model. ([Yu, 0003, 0065, 0083])
Regarding Claim 2
Jeong in view of Jang and Yu discloses: updating the at least one weight parameter of each of the plurality of graph modules included in the second NN model by using a gradient descent technique so that a loss resulting from changing parameters of the first NN model in the quantization is minimized for each of the plurality of graph modules, wherein the loss represents a difference between a first output value of a first graph module of the first NN model and a second output value of a second graph module of the second NN model, and wherein the first graph module of the first NN model corresponds to the second graph module of the second NN model. ([Yu, 0062, 0066-0067, 0072, 0075, Fig. 2, 0104-0106] discloses modifying and adjusting (i.e updating) the weight parameters to improve accuracy of the sparse quantized model (i.e second NN model) by calculating loss resulted from quantizing (i.e changing parameters) of the sparse floating-point model (i.e first NN model) is minimized for each layer (i.e graph modules) in the sparse quantized model. The loss includes using loss functions for feature distillations and partial loss of calibration data, which determines a difference between output of layers (i.e first graph module) of sparse floating-point model (i.e first NN model) and output of layers (i.e second graph module) of sparse quantized model, see [0094] for an example loss function. [0164] discloses adjusting weights with loss functions can be done with stochastic gradient descent)
Regarding Claim 3
Jeong in view of Jang and Yu discloses: updating the at least one weight parameter by subtracting a loss resulting from changing parameters of the first NN model due to the quantization ([Yu, Fig. 7, Steps 712, 714, 716, 718; 0129, 0131-0132] discloses updating sparse quantized model by minimizing (i.e subtracting) calibration loss function resulting from modifying precision values (i.e changing parameters) of sparse floating-point NN model (i.e first NN model) due to quantization)
Regarding Claim 13
Jeong in view of Jang and Yu discloses: wherein the at least one weight parameter and at least one input feature map parameter of the first NN model are in a floating-point format with a length of 16 bits to 32 bits. ([Yu, 0083] discloses sparse floating-point model (i.e first NN model) in higher-precision (i.e floating-point) data format with length half binary16 or FP32 (i.e 16 bits to 32 bits). [0066, 0072] discloses the model having weights parameters and feature maps (i.e input feature map parameter))
Regarding Claim 14
Jeong in view of Jang and Yu discloses: wherein the second NN model includes the at least one weight parameter and an input feature map parameter in an integer (INT) format with a length of 2 bits to 8 bits. ([Yu, 0083] discloses sparse quantized model (i.e second NN model) in lower-precision (i.e integer-point) data format with length 8-bit integer or 16-bit integer (i.e 16 bits to 32 bits). [0066, 0072] discloses the model having weights parameters and feature maps (i.e input feature map parameter))
Regarding Claim 20
Claim 20 is a “non-volatile computer-readable storage medium” claim having similar limitations of method of Claim 1, therefore it is rejected under the same rational as of Claim #. Additionally, Claim 20 includes additional limitations below that rejected under Yu.
Yu teaches: A non-volatile computer-readable storage medium storing instructions, the instructions, when executed by one or more processors, causing the one or more processors to perform steps comprising: ([Yu, Fig. 9A-9B, 0146, 0153] discloses non-volatile storage containing code and processors)
Claim(s) 4-5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jeong in view of Jang, Yu, and Sikka et al. (US 20220300801 A1, hereinafter “Sikka”)
Regarding Claim 4
Jeong in view of Jang and Yu discloses: determining([Jang, 0003, 0048, 0059] discloses determining a degree or gradient of change (i.e updating) for updating weight parameters for learning and training a model)
Jeong in view of Jang and Yu does not explicitly disclose: based on at least one user option or retraining completion time
However, Sikka discloses: determining, based on at least one user option or retraining completion time, a degree of the updating the at least one weight parameter. ([Sikka, 0073] discloses determining, based on predetermined amount of time or number of iterations for retraining (i.e retraining completion time), number (i.e degree) of updating the network parameters (i.e weight parameter))
Jeong, Jang, Yu, and Sikka are analogous art to the present invention because they are from the same field of endeavor directed to machine learning.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the determining degree of updating weight parameters disclosed by Jeong in view of Jang and Yu with determining the updating using completion time by Sikka. One of ordinary skill in the art would have been motivated to make this modification in order to know when to stop updating network parameters. ([Sikka, 0073])
Regarding Claim 5
Jeong in view of Jang, Yu, and Sikka discloses: wherein the quantization-aware retraining of the second NN model is terminated when the loss reaches a predetermined threshold or exceeds a predetermined execution time. ([Sikka, 0073] discloses quantization-aware retraining of a quantized network (i.e second NN model) is terminated when the loss function stays the same (i.e reaches a predetermined threshold) or achieves (i.e exceeds) a predetermined amount of time (i.e execution time))
Claim(s) 6-7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jeong in view of Jang, Yu, and Zhang et al. (“Root quantization: a self-adaptive supplement STE”, hereinafter “Zhang”)
Regarding Claim 6
Jeong in view of Jang and Yu discloses: wherein the updating the at least one weight parameter included in the second NN model by performing the quantization-aware retraining on the second NN model further comprises: ([Yu, 0067, 0072, 0075, 0088, 0096] updating, based on comparing output values of the sparse floating-point model (i.e first NN model) and the sparse quantized model (i.e second NN model) to calculate losses, weights in the sparse quantized model by performing quantization-aware retraining on the sparse quantized model. Specs [Page 65, Para 1-2] applicant describes quantization-aware retraining as retraining with quantization considered and using losses)
Jeong in view of Jang and Yu does not explicitly disclose: adding a loss change calculation function to a forward calculation of each of a plurality of graph modules included in the second NN model, corresponding to a quantization module added to each of the plurality of graph modules included in the second NN model; and verifying output values of each of the plurality of graph modules included in the second NN model during a backward computation for changes in each of the at least one weight parameter.
However, Zhang discloses: adding a loss change calculation function to a forward calculation of each of a plurality of graph modules included in the second NN model, corresponding to a quantization module added to each of the plurality of graph modules included in the second NN model; ([Zhang, Page 6270, Algorithm 1] discloses gradient equations (i.e loss change calculation function) to a forward propagation (i.e forward calculation) of weights of a NN model, corresponding to quantization equations 4 (i.e quantization module) in [Page 6268, Col 2] applied to every weight of the model. The weights correspond to layers (i.e graph modules) to a NN model. A quantization module is a component that applies the quantization of a model)
and verifying output values of each of the plurality of graph modules included in the second NN model during a backward computation for changes in each of the at least one weight parameter. ([Zhang, Page 6270, Algorithm 1] discloses computing and propagating weight gradients (i.e changes in weight parameter) during backward propagation (i.e backward computation))
Jeong, Jang, Yu, and Zhang are analogous art to the present invention because they are from the same field of endeavor directed to machine learning.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the quantization-aware retraining disclosed by Jeong in view of Jang and Yu with adding a loss change calculation function by Zhang. One of ordinary skill in the art would have been motivated to make this modification in order to approximate quantization error in gradients and fine-tune weights of a quantized model. ([Zhang, Abstract])
Regarding Claim 7
wherein the loss change calculation function has results of the forward calculation unaffected by a loss change calculation and preserves original equations removed by round and clip operations included in the quantization module during the backward computation. ([Zhang, Page 6268, Equations 4; Page 6269, Equation 5; Page 6269, Section 3.3 Para 4; Page 6270, Algorithm 1] discloses the backward propagation (i.e loss change calculation) has a detach function on the
w
n
^
where during the backward propagation, the Equations 4 for
w
n
^
are omitted, which preserves the original equations 4 that includes round and clamp, a type of clip, operations included in quantization (i.e quantization module), so those round and clamp operations are removed during backward propagation)
Claim(s) 8-9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jeong in view of Yu, Jang, Zhang, Esser et al. (“Learned Step Size Quantization”, hereinafter “Esser”), and Subramanian et al. (“Practical Quantization in PyTorch”, hereinafter “Subramanian”)
Regarding Claim 8
Jeong in view of Jang, Yu, and Zhang does not explicitly disclose: wherein the loss change calculation function is represented by a first detach function for an input feature map parameter and a second detach function for a weight parameter:
d
e
t
a
c
h
x
-
o
s
x
-
x
-
o
s
x
and
d
e
t
a
c
h
w
s
w
-
w
s
w
, where x denotes the input feature map parameter of each of the plurality of graph modules included in the second NN model, sx denotes a scale value for the input feature map parameter, o denotes an offset value for the input feature map parameter, w denotes the weight parameter of each of the plurality of graph modules included in the second NN model, and sw denotes a scale value for the weight parameter.
However, Esser discloses: wherein the loss change calculation function is represented by a first detach function for an input feature map parameter and a second detach function for a weight parameter:
d
e
t
a
c
h
x
s
x
-
x
s
x
and
d
e
t
a
c
h
w
s
w
-
w
s
w
, where x denotes the input feature map parameter of each of the plurality of graph modules included in the second NN model, sx denotes a scale value for the input feature map parameter, o denotes an offset value for the input feature map parameter, w denotes the weight parameter of each of the plurality of graph modules included in the second NN model, and sw denotes a scale value for the weight parameter. ([Esser, Page 12, Functions 2-3] discloses detach(yOut- yGrad) in Function 2, where yOut = round (x) and yGrad = x. Variable x can be weights or an activation tensor (i.e input). [Page 12, Function 3] discloses that this variable x divided by scale s, therefore producing equation
d
e
t
a
c
h
x
s
x
-
x
s
x
and
d
e
t
a
c
h
w
s
w
-
w
s
w
by substitution)
Jeong, Jang, Yu, Zhang, and Esser are analogous art to the present invention because they are from the same field of endeavor directed to machine learning.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the method to retrain a second NN model through quantization-aware and loss change function disclosed by Jeong in view of Jang, Yu, and Zhang with using detach equations for loss change function by Esser. One of ordinary skill in the art would have been motivated to make this modification in order to have memory efficient training framework for quantizing weights and activations. ([Esser, Page 11-12, Section B])
Jeong in view of Jang, Yu, Zhang, and Esser does not explicitly disclose:
x
-
o
s
x
However, Subramanian discloses:
x
-
o
s
x
([Subramanian, Page 3, Mapping Function] discloses
r
o
u
n
d
r
S
+
Z
, which using distributive property of multiplication, is a mathematical equivalent of
x
-
o
s
x
. Using the equivalent, zero-point Z corresponds to
-
o
s
x
, input r corresponds to input x, and scale s corresponds to
s
x
.)
Jeong, Jang, Yu, Zhang, Esser, and Subramanian are analogous art to the present invention because they are from the same field of endeavor directed to machine learning.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the use of detach function for loss change function disclosed by Jeong in view of Jang, Yu, Zhang, and Esser with offset in quantizing input by Subramanian. One of ordinary skill in the art would have been motivated to make this modification in order to use affine quantization for activations only but not for weights because that is a common practice in neural network quantization. ([Subramanian, Page 5, Affine and Symmetric Quantization Schemes, Para 1])
Regarding Claim 9
Jeong in view of Yu, Jang, Zhang, and Esser discloses:
x
s
x
and
w
s
w
in the first equation y ([Esser, Page 12, Functions 2-3] discloses yOut = round (x) and yGrad = x. Variable x can be weights or an activation tensor (i.e input). [Page 12, Function 3] discloses that this variable x is divided by scale s, therefore producing equations
x
s
x
and
w
s
w
by substitution)
d
e
t
a
c
h
x
s
x
-
x
s
x
+
x
s
x
and
d
e
t
a
c
h
w
s
w
-
w
s
w
+
w
s
w
in the second equation y ([Esser, Page 12, Functions 2-3] discloses detach(yOut- yGrad) + yGrad in Function 2, where yOut = round (x) and yGrad = x. Variable x can be weights or an activation tensor (i.e input). [Page 12, Function 3] discloses that this variable x is first divided by scale s: v = v/s, therefore producing quantization equations
d
e
t
a
c
h
x
s
x
-
x
s
x
+
x
s
x
and
d
e
t
a
c
h
w
s
w
-
w
s
w
+
w
s
w
for both weights and activation tensors)
Jeong in view of Jang, Yu, Zhang, and Esser does not explicitly disclose:
x
-
o
s
x
; replacing y =
x
-
o
s
x
*
s
x
+
o
*
(
w
s
w
*
s
w
)
of each of each of the plurality of graph modules to which the quantization module is added with y =
(
d
e
t
a
c
h
x
-
o
s
x
-
x
-
o
s
x
+
x
-
o
s
x
*
s
x
+
o
) * (
d
e
t
a
c
h
w
s
w
-
w
s
w
+
w
s
w
*
s
x
))
However, Subramanian discloses:
x
-
o
s
x
([Subramanian, Page 3, Mapping Function] discloses
r
o
u
n
d
r
S
+
Z
, which using distributive property of multiplication, is a mathematical equivalent of
x
-
o
s
x
. Using the equivalent, zero-point Z corresponds to
-
o
s
x
, input r corresponds to input x, and scale s corresponds to
s
x
.)
replacing y =
x
-
o
s
x
*
s
x
+
o
*
(
w
s
w
*
s
w
)
of each of each of the plurality of graph modules to which the quantization module is added with y =
(
d
e
t
a
c
h
x
-
o
s
x
-
x
-
o
s
x
+
x
-
o
s
x
*
s
x
+
o
) * (
d
e
t
a
c
h
w
s
w
-
w
s
w
+
w
s
w
*
s
x
)) ([Subramanian, Page 17, Fig. 7 & Para 1] discloses quantization, convolution, and dequantization through fake quantization modules added. As seen in Fig. 7, the forward expression for y =
i
n
p
u
t
s
*
w
e
i
g
h
t
s
, so to get dequantized y, multiple dequantized inputs and weights. Expressions inside for the
x
-
o
s
x
and
(
w
s
w
)
are disclosed by Esser and Subramanian above. [Subramanian, Page 3, Mapping Function] discloses using a dequantization inverse function by reverting scaling and offset from
s
x
and
o
added in quantization, which produces equations
x
-
o
s
x
*
s
x
+
o
and
(
w
s
w
*
s
w
)
to get their dequantization values, so the putting the forward calculation together with these expressions yields Equation 2 dequantized y =
x
-
o
s
x
*
s
x
+
o
*
(
w
s
w
*
s
w
)
. For equation 2, using detached quantized equations disclosed in Claim 8, they can dequantized by inversing the scales and offsets to obtain dequantized y =
(
d
e
t
a
c
h
x
-
o
s
x
-
x
-
o
s
x
+
x
-
o
s
x
*
s
x
+
o
) * (
d
e
t
a
c
h
w
s
w
-
w
s
w
+
w
s
w
*
s
x
)))
Jeong, Jang, Yu, Zhang, Esser, and Subramanian are analogous art to the present invention because they are from the same field of endeavor directed to machine learning.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the use of detach function for loss change function disclosed by Jeong in view of Jang, Yu, Zhang, and Esser with using offsets and dequantization by Subramanian. One of ordinary skill in the art would have been motivated to make this modification in order to have to use affine quantization for activations only and optimizing and improving accuracy of a model using the dequantize, quantize, and convolution training strategy of Subramanian. ([Subramanian, Page 5, Affine and Symmetric Quantization Schemes, Para 1; Page 17, Fig. 7 & Para 1])
Claim(s) 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jeong in view of Jang, Yu, and Miret et al. (“Neuroevolution-Enhanced Multi-Objective Optimization for
Mixed-Precision Quantization”, hereinafter “Miret”)
Regarding Claim 10
Jeong in view of Jang and Yu discloses: wherein the scale value and the offset value is obtained by an equation below,
PNG
media_image11.png
64
121
media_image11.png
Greyscale
where max denotes a maximum value among the input values and output values collected for the calibration data, min denotes a minimum value among the input values and output values collected for the calibration data, and bitwidth denotes a target quantization bitwidth. ([Jeong, 0048-0047, Equation 2, 0072] discloses a scale value and offset value, where the scale value is obtained by equation
PNG
media_image15.png
78
530
media_image15.png
Greyscale
where max is maximum value of real number value xf collected for calibration data, min is minimum value of real number value xf collected for calibration data, and n is the number of bits (i.e bitwidth) for integer quantization range (i.e target quantization bitwidth). [0075-0076] discloses maximum values and minimum values are related to feature map data (i.e calibration data) collected)
Jeong in view of Jang and Yu does not explicitly disclose: wherein the scale value and the offset value is obtained by an equation below,
PNG
media_image11.png
64
121
media_image11.png
Greyscale
where max denotes a maximum value among the input values and output values collected for the calibration data, min denotes a minimum value among the input values and output values collected for the calibration data, and bitwidth denotes a target quantization bitwidth.
However, Miret discloses: wherein the scale value and the offset value is obtained by an equation below,
PNG
media_image11.png
64
121
media_image11.png
Greyscale
where max denotes a maximum value among the input values and output values collected for the calibration data, min denotes a minimum value among the input values and output values collected for the calibration data, and bitwidth denotes a target quantization bitwidth. ([Miret, Page 4, Section 3.1 & Equation 1] discloses scale equation s and offset equation z
PNG
media_image16.png
32
222
media_image16.png
Greyscale
, where xmax is the maximum value of q(x;b) values calculated for each layer (i.e input values and output values calibration data), xmin is the minimum value of q(x;b) values calculated for each layer (i.e input values and output values calibration data), and bitwidth denotes the selected bit width for quantization (i.e targe quantization bitwidth). The calibration data are the q(x;b) values calculated through inputting activation tensors of a layer into function q)
Jeong, Jang, Yu and Miret are analogous art to the present invention because they are from the same field of endeavor directed to machine learning.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the use of scale values and offset values disclosed by Jeong in view of Jang and Yu with using the offset value equation by Miret. One of ordinary skill in the art would have been motivated to make this modification in order to perform uniform affine quantization for effective quantization. ([Miret, Page 4, Section 3.1])
Claim(s) 11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jeong in view of Jang, Yu, and in view of Bhalgat ("LSQ+: Improving low-bit quantization through learnable offsets and better initialization ", hereinafter “Bhalgat”)
Regarding Claim 11
Jeong in view of Jang and Yu does not explicitly disclose: wherein a convolution operation in the first NN model is expressed as:
PNG
media_image6.png
45
520
media_image6.png
Greyscale
where feature_infp denotes an input feature map parameter in a form of floating-point, weightfp denotes a weight parameter in a form of floating-point, of denotes an offset value for an input feature map, sf denotes a scale value for the input feature map, sw denotes a scale value for a weight, and
PNG
media_image7.png
37
25
media_image7.png
Greyscale
denotes a round and clip operation.
However, Bhalgat discloses: wherein a convolution operation in the first NN model is expressed as:
PNG
media_image6.png
45
520
media_image6.png
Greyscale
where feature_infp denotes an input feature map parameter in a form of floating-point, weightfp denotes a weight parameter in a form of floating-point, of denotes an offset value for an input feature map, sf denotes a scale value for the input feature map, sw denotes a scale value for a weight, and
PNG
media_image7.png
37
25
media_image7.png
Greyscale
denotes a round and clip operation. ([Bhalgat, Page 2980, Col 1 & Equations 2-5] discloses a convolution operation for a first NN model.
w
^
x
^
is the feature_outfp that is calculated through equation 5, where
x
^
=
=
x
-
x
s
+
B
,
x
-
=
c
l
a
m
p
(
x
-
β
s
,
n
,
p
)
, and B is the offset of. Substituting those equations into equation 5 in yields the equation above:
w
^
x
^
=
w
-
x
s
w
x
-
x
s
x
+
B
=
w
-
x
s
w
(
c
l
a
m
p
x
-
β
s
,
n
,
p
x
s
x
+
B
)
.
w
-
is the quantized weight corresponding to weight / scale)
Jeong, Jang, Yu and Bhalgat are analogous art to the present invention because they are from the same field of endeavor directed to machine learning.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the first NN model disclosed by Jeong in view of Jang and Yu with the convolution operation in the first NN model by Bhalgat. One of ordinary skill in the art would have been motivated to make this modification in order to improve the efficient and accurate quantization schemes for activations by using the disclosed convolution operation. ([Bhalgat, Page 2979, Section 3-3.1])
Claim(s) 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jeong in view of Jang, Yu, and in view of Krishnamoorthi ("Quantizing deep convolutional networks for efficient inference: A whitepaper", hereinafter “Krishnamoorthi”)
Regarding Claim 12
Jeong in view of Jang and Yu does not explicitly disclose: wherein a convolution operation in the second NN model is expressed as:
PNG
media_image12.png
39
569
media_image12.png
Greyscale
where feature_outint denotes an output feature map parameter in a form of an integer, feature_inint denotes an input feature map parameter in a form of an integer and weightint denotes a weight parameter in a form of an integer.
However, Krishnamoorthi discloses: wherein a convolution operation in the second NN model is expressed as:
PNG
media_image12.png
39
569
media_image12.png
Greyscale
where feature_outint denotes an output feature map parameter in a form of an integer, feature_inint denotes an input feature map parameter in a form of an integer and weightint denotes a weight parameter in a form of an integer. ([Krishnamoorthi, Page 20, Step 3] discloses convolution operation for y = conv(Q(wcorrected),x) for training a quantized model (i.e second NN model), where conv is ⊗, Q(wcorrected) is the weight int that has been corrected for the model, and x is the input feature map)
Jeong, Jang, Yu and Krishnamoorthi are analogous art to the present invention because they are from the same field of endeavor directed to machine learning.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the method of quantization-aware retraining disclosed by Jeong in view of Jang and Yu with using a convolution operation by Krishnamoorthi. One of ordinary skill in the art would have been motivated to make this modification in order to improve the accuracy of the quantized model to a float-point accuracy. ([Krishnamoorthi, Page 15, Section 3.2.2; Page 20, Step 2])
Claim(s) 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yu in view of Zhang et al. (“Root quantization: a self-adaptive supplement STE”, hereinafter “Zhang”)
Regarding Claim 18
Yu does not explicitly disclose: adding a loss change calculation function to a forward calculation of each of the plurality of graph modules included in the second NN model, corresponding to a quantization module added to each of the plurality of graph modules included in the second NN model; and verifying output values of each of the plurality of graph modules included in the second NN model during a backward computation for changes in each of the at least one weight parameter, wherein the loss change calculation function has results of the forward calculation unaffected by a loss change calculation and preserves original equations removed by round and clip operations included in the quantization module during the backward computation.
However, Zhang discloses: adding a loss change calculation function to a forward calculation of each of the plurality of graph modules included in the second NN model, corresponding to a quantization module added to each of the plurality of graph modules included in the second NN model; ([Zhang, Page 6270, Algorithm 1] discloses gradient equations (i.e loss change calculation function) to a forward propagation (i.e forward calculation) of weights of a NN model, corresponding to quantization equations 4 (i.e quantization module) in [Page 6268, Col 2] applied to every weight of the model. The weights correspond to layers (i.e graph modules) to a NN model. A quantization module is a component that applies the quantization of a model)
and verifying output values of each of the plurality of graph modules included in the second NN model during a backward computation for changes in each of the at least one weight parameter, ([Zhang, Page 6270, Algorithm 1] discloses computing and propagating weight gradients (i.e changes in weight parameter) during backward propagation (i.e backward computation))
wherein the loss change calculation function has results of the forward calculation unaffected by a loss change calculation and preserves original equations removed by round and clip operations included in the quantization module during the backward computation. ([Zhang, Page 6268, Equations 4; Page 6269, Equation 5; Page 6269, Section 3.3 Para 4; Page 6270, Algorithm 1] discloses the backward propagation (i.e loss change calculation) has a detach function on the
w
n
^
where during the backward propagation, the Equations 4 for
w
n
^
are omitted, which preserves the original equations 4 that includes round and clamp (i.e clip) operations included in quantization (i.e quantization module), so those round and clamp operations are removed during backward propagation)
Yu and Zhang are analogous art to the present invention because they are from the same field of endeavor directed to machine learning.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the quantization-aware retraining disclosed by Yu with adding a loss change calculation function by Zhang. One of ordinary skill in the art would have been motivated to make this modification in order to approximate quantization error in gradients and fine-tune weights of a quantized model. ([Zhang, Abstract])
Claim(s) 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yu, in view of Zhang, Esser et al. (“Learned Step Size Quantization”, hereinafter “Esser”), and Subramanian et al. (“Practical Quantization in PyTorch”, hereinafter “Subramanian”)
Regarding Claim 19
Yu in view of Zhang does not explicitly disclose: wherein the loss change calculation function is represented by a first detach function for an input feature map parameter and a second detach function for a weight parameter:
d
e
t
a
c
h
x
s
x
-
x
s
x
and
d
e
t
a
c
h
w
s
w
-
w
s
w
, where x denotes the input feature map parameter of each of the plurality of graph modules included in the second NN model, sx denotes a scale value for the input feature map parameter, o denotes an offset value for the input feature map parameter, w denotes the weight parameter of each of the plurality of graph modules included in the second NN model, and sw denotes a scale value for the weight parameter, and wherein the updating the at least one weight parameter included in the second NN model by performing the quantization-aware retraining on the second NN model further comprises: replacing y =
x
-
o
s
x
*
s
x
+
o
*
(
w
s
w
*
s
w
)
of each of the plurality of graph modules to which the quantization module is added with y =
(
d
e
t
a
c
h
x
-
o
s
x
-
x
-
o
s
x
+
x
-
o
s
x
*
s
x
+
o
) * (
d
e
t
a
c
h
w
s
w
-
w
s
w
+
w
s
w
*
s
x
)) using the loss change calculation function.
However, Esser discloses: wherein the loss change calculation function is represented by a first detach function for an input feature map parameter and a second detach function for a weight parameter:
d
e
t
a
c
h
x
s
x
-
x
s
x
and
d
e
t
a
c
h
w
s
w
-
w
s
w
, where x denotes the input feature map parameter of each of the plurality of graph modules included in the second NN model, sx denotes a scale value for the input feature map parameter, o denotes an offset value for the input feature map parameter, w denotes the weight parameter of each of the plurality of graph modules included in the second NN model, and sw denotes a scale value for the weight parameter. ([Esser, Page 12, Functions 2-3] discloses detach(yOut- yGrad) in Function 2, where yOut = round (x) and yGrad = x. Variable x can be weights or an activation tensor (i.e input). [Page 12, Function 3] discloses that this variable x divided by scale s, therefore producing equation
d
e
t
a
c
h
x
s
x
-
x
s
x
and
d
e
t
a
c
h
w
s
w
-
w
s
w
by substitution)
x
s
x
and
w
s
w
in the third equation y =
x
-
o
s
x
*
s
x
+
o
*
(
w
s
w
*
s
w
)
([Esser, Page 12, Functions 2-3] discloses yOut = round (x) and yGrad = x. Variable x can be weights or an activation tensor (i.e input). [Page 12, Function 3] discloses that this variable x is divided by scale s, therefore producing equations
x
s
x
and
w
s
w
by substitution)
d
e
t
a
c
h
x
s
x
-
x
s
x
+
x
s
x
and
d
e
t
a
c
h
w
s
w
-
w
s
w
+
w
s
w
in the fourth equation y =
(
d
e
t
a
c
h
x
-
o
s
x
-
x
-
o
s
x
+
x
-
o
s
x
*
s
x
+
o
) * (
d
e
t
a
c
h
w
s
w
-
w
s
w
+
w
s
w
*
s
x
)) ([Esser, Page 12, Functions 2-3] discloses detach(yOut- yGrad) + yGrad in Function 2, where yOut = round (x) and yGrad = x. Variable x can be weights or an activation tensor (i.e input). [Page 12, Function 3] discloses that this variable x is first divided by scale s: v = v/s, therefore producing quantization equations
d
e
t
a
c
h
x
s
x
-
x
s
x
+
x
s
x
and
d
e
t
a
c
h
w
s
w
-
w
s
w
+
w
s
w
for both weights and activation tensors)
Yu, Zhang, and Esser are analogous art to the present invention because they are from the same field of endeavor directed to machine learning.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the method to retrain a second NN model through quantization-aware and loss change function disclosed by Yu in view of Zhang with using detach equations for loss change function by Esser. One of ordinary skill in the art would have been motivated to make this modification in order to have memory efficient training framework for quantizing weights and activations. ([Esser, Page 11-12, Section B])
Yu in view of Zhang and Esser does not explicitly disclose:
x
-
o
s
x
; replacing y =
x
-
o
s
x
*
s
x
+
o
*
(
w
s
w
*
s
w
)
of each of each of the plurality of graph modules to which the quantization module is added with y =
(
d
e
t
a
c
h
x
-
o
s
x
-
x
-
o
s
x
+
x
-
o
s
x
*
s
x
+
o
) * (
d
e
t
a
c
h
w
s
w
-
w
s
w
+
w
s
w
*
s
x
))
However, Subramanian discloses:
x
-
o
s
x
([Subramanian, Page 3, Mapping Function] discloses
r
o
u
n
d
r
S
+
Z
, where zero-point (i.e offset) Z corresponds to
-
o
s
x
, input r corresponds to input x, and scale s is
s
x
. Substituting Z =
-
o
s
x
and using distributive property of multiplication, we get the mathematical equivalent to
x
-
o
s
x
)
replacing y =
x
-
o
s
x
*
s
x
+
o
*
(
w
s
w
*
s
w
)
of each of each of the plurality of graph modules to which the quantization module is added with y =
(
d
e
t
a
c
h
x
-
o
s
x
-
x
-
o
s
x
+
x
-
o
s
x
*
s
x
+
o
) * (
d
e
t
a
c
h
w
s
w
-
w
s
w
+
w
s
w
*
s
x
)) ([Subramanian, Page 17, Fig. 7 & Para 1] discloses quantization, convolution, and dequantization through fake quantization modules added. As seen in Fig. 7, the forward expression for y =
i
n
p
u
t
s
*
w
e
i
g
h
t
s
, so to get dequantized y, multiple dequantized inputs and weights. Expressions inside for the
x
-
o
s
x
and
(
w
s
w
)
are disclosed by Esser and Subramanian above. [Subramanian, Page 3, Mapping Function] discloses using a dequantization inverse function by reverting scaling and offset from
s
x
and
o
added in quantization, which produces equations
x
-
o
s
x
*
s
x
+
o
and
(
w
s
w
*
s
w
)
to get their dequantization values, so the putting the forward calculation together with these expressions yields Equation 2 dequantized y =
x
-
o
s
x
*
s
x
+
o
*
(
w
s
w
*
s
w
)
. For equation 2, using detached quantized equations disclosed in Claim 8, they can dequantized by inversing the scales and offsets to obtain dequantized y =
(
d
e
t
a
c
h
x
-
o
s
x
-
x
-
o
s
x
+
x
-
o
s
x
*
s
x
+
o
) * (
d
e
t
a
c
h
w
s
w
-
w
s
w
+
w
s
w
*
s
x
)))
Yu in view of Zhang, Esser, and Subramanian are analogous art to the present invention because they are from the same field of endeavor directed to machine learning.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the use of detach function for loss change function disclosed by Jeong in view of Yu in view of Zhang, and Esser with using offsets and dequantization by Subramanian. One of ordinary skill in the art would have been motivated to make this modification in order to have to use affine quantization for activations only and optimizing and improving accuracy of a model using the dequantize, quantize, and conv training strategy of Subramanian. ([Subramanian, Page 5, Affine and Symmetric Quantization Schemes, Para 1; Page 17, Fig. 7 & Para 1])
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Amanda D. Nguyen whose telephone number is (571)270-1854. The examiner can normally be reached M-F, 7:30am to 5:00 pm ET First Fridays off, 2nd Friday 7:30 am - 4:00 pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abdullah Al Kawsar can be reached at (571)270-3169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AMANDA D NGUYEN/Examiner, Art Unit 2127
/ABDULLAH AL KAWSAR/Supervisory Patent Examiner, Art Unit 2127
