DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Claims 1-20 are pending of which claims 1, 8 and 15 are in independent form.
Claims 1-20 are rejected under 35 U.S.C. 103.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Applicant’s arguments, see “Remarks”, filed 3/26/2026, with respect to 35 USC 112(f) and 35 USC 101 (Abstract Idea) have been fully considered and are persuasive. The said rejections of claims 1-20 has been withdrawn.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 2, and 6 are rejected under 35 U.S.C. 103 as being unpatentable over IGUCHI; Noritaka et al. (US 20210409769 A1) [Iguchi] in view of Li; Duanshun et al. (US 20220261616 A1) [Li].
Regarding claim 1, Iguchi discloses, an electronic device, the device comprising: a shifter circuit configured to perform a shift operation [based on a codebook supporting a plurality of quantization levels preset for data bits of a data set] (it is further possible that a first bit number is different from a second bit number, the first bit number being a number of bits shifted by each of the right bit shift and the left bit shift performed in the first decoding scheme, the second bit number being a number of bits shifted by each of the right bit shift and the left bit shift performed in the second decoding scheme, and in the first decoding scheme, the transforming of the first quantization parameter to the first scale value includes applying the first table to the first quantization parameter to determine the first scale value, and in the second decoding scheme, the transforming of the first quantization parameter to the first scale value includes applying the first table to the first quantization parameter to determine a third scale value, and performing bit shift on the third scale value by a number of bits corresponding to a difference between the first bit number and the second bit number ¶ [0202]-[0203]. Attribute information encoder 7921 includes left bit shifter 7941, transformer 7942, right bit shifter 7943, scale value calculator 7944, quantizer 7945, and entropy encoder 7946 ¶ [1295]. Also see ¶ [0200], [1295], [1299], [1301], [1304], [1312], [1377]. Examiner specifies that first table including a first scale has been interpreted as codebook); and
a decoder circuit (attribute information decoder 7931 ¶ [01299], attribute information decoders ¶ [1303], [1309]) configured to control the shifter (inverse quantizer 7953C performs both the inverse quantization process performed by inverse quantizer 7953A and the left bit shift performed by left bit shifter 7954A shown in FIG. 167 by multiplying the quantized coefficient obtained by entropy decoder 7951A by a scale value (Scale3) generated by scale value calculator 7952B. Similarly, inverse quantizer 7953D performs both the inverse quantization process performed by inverse quantizer 7953B and the left bit shift performed by left bit shifter 7954B shown in FIG. 167 by multiplying the quantized coefficient obtained by entropy decoder 7951B by a scale value (Scale4) generated by scale value calculator 7952B ¶ [1312]) by setting quantization scales (Scale3/Scale4 generated by scale value calculator ¶ [1312], Quantizer 7945A/7945B performs quantization using scale value (Scale3/4) ¶ [1336]) of the data bits differently [for preset groups of a neural network](Scale3 versus Scale4 ¶ [1312], different quantization operations using Scale3 and Scale4 ¶ [1336]. Quantization scale…in layer I ¶ [0693], [0796], Also see ¶ [0202]);
wherein the shifter is configured to quantize (Quantizer 7945A performs both the right bit shift performed by right bit shifter 7943 and the quantization process performed by quantizer 7945 shown in FIG. 165 by dividing the shifted coefficient value obtained by transformer 7942A by the scale value (Scale3) generated by scale value calculator 7952B. Similarly, quantizer 7945B performs both the right bit shift performed by right bit shifter 7943 and the quantization process performed by quantizer 7945 shown in FIG. 165 by dividing the shifted coefficient value obtained by transformer 7942B by the scale value (Scale4) generated by scale value calculator 7952B ¶ [1336]. Also see ¶ [1295], [1299], [1304], [1310], [1312], [1345]-[1348], [1365], [1367]-[1368], [1377]), output the data bits (Quantizer 7945A performs both the right bit shift performed by right bit shifter 7943 and the quantization process performed by quantizer 7945 … by dividing the shifted coefficient value obtained by transformer 7942A by the scale value (Scale3) generated by scale value calculator 7952B ¶ [1336]) by control of the decoder (quantized coefficient obtained by entropy decoder 7951A by a scale value (Scale3) generated by scale value calculator 7952B ¶ [1312], prediction controller 1409 controls whether to decode a decoding target volume using intra prediction or inter prediction. For example, prediction controller 1409 selects intra prediction or inter prediction in accordance with information that is appended to the bitstream and indicates the prediction mode to be used. Note that prediction controller 1409 may continuously select intra prediction when it has been decided in advance to decode the decoding target space using intra space ¶ [0603], [0741]. Also see ¶ [0198], [0200], [0202], [0204], [0726], [0727], [0737]) based on a control signal from the decoder (This reconstructed volume is outputted as decoded three-dimensional data ¶ [0600], [0741], [1201], That is, inverse quantizer 7953C performs both the inverse quantization process performed by inverse quantizer 7953A and the left bit shift performed by left bit shifter 7954A shown in FIG. 167 by multiplying the quantized coefficient obtained by entropy decoder 7951A by a scale value (Scale3) generated by scale value calculator 7952B. Similarly, inverse quantizer 7953D performs both the inverse quantization process performed by inverse quantizer 7953B and the left bit shift performed by left bit shifter 7954B shown in FIG. 167 by multiplying the quantized coefficient obtained by entropy decoder 7951B by a scale value (Scale4) generated by scale value calculator 7952B ¶ [1312], [1343]) that specifies different quantization scales [for the preset groups] (different scales for higher and lower layers ¶ [0662], layer specific quantization scale ¶ [0693], [0796], Scale3 and Scale4 ¶ [1312], [1336]).
However, Iguchi does not explicitly facilitate based on a codebook supporting a plurality of quantization levels preset for data bits of a data set; for preset groups of a neural network; for the preset groups.
Li discloses, based on a codebook supporting a plurality of quantization levels preset for data bits of a data set (the cluster centers identified during clustering may be used to form the codebook… A quantized weight tensor may be represented with a codebook ¶ [0173], scalar quantization may cluster the n elements into k clusters ¶ [0171], Scalar quantization and vector quantization may be effective quantization in NN compression ¶ [0157]);
for preset groups of a neural network; for the preset groups (obtain an NN model including an NN layer associated with a weight matrix ¶ [0124], Clustering-based quantization, as disclosed herein, may address tensor arrangement of CNN layers and/or may reduce the impact of outliers on clustering (for example, during scalar quantization and/or vector quantization of NN weights) ¶ [0160], Scalar quantization may cluster the n elements into k clusters ¶ [0171], clustering-based quantization or inverse quantization for NN ¶ [0193]).
It would have been obvious to one ordinary skilled in the art at the time of the filing of the present invention to combine the teachings of the cited references because Li’s system would have allowed Iguchi to facilitate based on a codebook supporting a plurality of quantization levels preset for data bits of a data set; for preset groups of a neural network; for the preset groups. The motivation to combine is apparent in the Iguchi’s reference, because there is a need to improve reducing the storage and/or transmission bandwidth needed for models.
Regarding claim 2, the combination of Iguchi and Li discloses, wherein the quantization scales are determined based on a scale parameter for minimizing a quantization error when quantizing the data set with an approximate weight in a preset operation (Iguchi: an error (quantization error) which may be generated by quantization reduces as the quantization scale is smaller. In the other case where the quantization scale is larger, the resulting quantization error is larger ¶ [0661]-[0662], [0673]. In this case, the smaller the quantization scale, the smaller the error (quantization error) that may occur due to quantization. Conversely, the larger the quantization scale, the larger the quantization error ¶ [0795]-[0796]).
Regarding claim 6, the combination of Iguchi and Li discloses, wherein the data bits are quantized at a log level of 2 (Iguchi: the scale and the quantization value (quantization parameter (QP) value) are expressed by Equation G3 below. quantization value(QP value)=log(scale) (Equation G3); quantization value(QP value)=default value(reference value)+quantization delta(difference information) (Equation G4) ¶ [0870]).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 3-5 and 7-20 are rejected under 35 U.S.C. 103 as being unpatentable over Iguchi in view of Li in view of CHOI; Yoo Jin et al. (US 20180107925 A1) [Choi].
Regarding claim 3, the combination of Iguchi and Li teaches all the limitation of claim 1.
However, neither one of the Iguchi or Li explicitly facilitate wherein the data set corresponds to a subset selected from some pieces of data, of which a similarity is high with a subset generated based on a Lloyd-Max quantization technique, of a randomly generated universal set.
Choi discloses, wherein the data set corresponds to a subset selected from some pieces of data, of which a similarity is high with a subset generated based on a Lloyd-Max quantization technique, of a randomly generated universal set (FIG. 4 illustrates an exemplary flowchart for performing k-means clustering using Lloyd's algorithm, according to one embodiment ¶ [0020], [0022], [0060], [0061], [ 0087]).
It would have been obvious to one ordinary skilled in the art at the time of the present invention to combine the teachings of the cited references because Choi’s system would have allowed Iguchi and Li to facilitate wherein the data set corresponds to a subset selected from some pieces of data, of which a similarity is high with a subset generated based on a Lloyd-Max quantization technique, of a randomly generated universal set. The motivation to combine is apparent in the Iguchi and Li’s reference, because there is a need to improve deploying deep neural networks on devices with limited storage, such as mobile/portable devices.
Regarding claim 4, the combination of Iguchi, Li, and Choi discloses, wherein the preset groups comprise either one or both of a channel and a layer of a neural network (Choi: network parameter quantization is performed by quantizing the network parameters of all layers of a deep neural network together at once [Abstract]. Also see ¶ [0093]-[0094]).
Regarding claim 5, the combination of Iguchi, Li, and Choi discloses, the scale parameter is derived through an iterative operation based on a following equation:
∀i,αqi=argmin|p -wi| ℒ=∑i(wi-αqi)2α*=Σiwi·qi/Σiqi2,
And α denotes the scale parameter, wi denotes an element of a subset, qj denotes a quantization point of the subset, L denotes the quantization error, and SQ={αqj}
(Choi: see ¶ [0057]-[0065], also see ¶ [0081]-[0085]).
Regarding claims 7, 14 and 15, the combination of Iguchi, Li, and Choi discloses, a processor-implemented method, the method comprising: selecting some pieces of data from a quantized universal set in a preset operation (Iguchi: Quantizer 7004 quantizes the coefficient value to generate a quantized value. Quantization controller 7005 controls a quantization parameter used for the quantization by quantizer 7004. For example, quantization controller 7005 may change the quantization parameter (or quantization step) according to the hierarchical structure for the encoding. In this way, an appropriate quantization parameter can be selected for each layer of the hierarchical structure, so that the amount of codes occurring in each layer can be controlled. Quantization controller 7005 also sets quantization parameters for a certain layer and the layers lower than the certain layer that include a frequency component that has a small effect on the subjective image quality at a maximum value, and sets quantization coefficients for the certain layer and the layers lower than the certain layer at 0, for example ¶ [1109]);
performing quantization with an approximate weight on the selected pieces of data (Choi: In the previous section, the local impact of Hessian-weighted quantization on the average loss function at w=ŵ was approximately quantified in Equation (7)(d) ¶ [0084], [0134]);
determining a quantization error for the quantized pieces of data (Iguchi: an error (quantization error) which may be generated by quantization reduces as the quantization scale is smaller. In the other case where the quantization scale is larger, the resulting quantization error is larger ¶ [0661]-[0662], [0673]. In this case, the smaller the quantization scale, the smaller the error (quantization error) that may occur due to quantization. Conversely, the larger the quantization scale, the larger the quantization error ¶ [0795]-[0796]); and
deriving a scale parameter value for minimizing the determined quantization error (Iguchi: For example, the three-dimensional data encoding device performs quantization by dividing the prediction residual by a quantization scale (also referred to as a quantization step). In this case, an error (quantization error) which may be generated by quantization reduces as the quantization scale is smaller. In the other case where the quantization scale is larger, the resulting quantization error is larger ¶ [0661]-[0662]).
representing neural network weights; generating quantized neural network weights (Li: analyze the distribution of one or more NN weights in weight tensors in NN layers ¶ [0004]-[0007]; )
Regarding claims 8, and 16, the combination of Iguchi, Li, and Choi discloses, wherein the selecting the pieces of data in the preset operation comprises selecting the pieces of data, of which a similarity is high with a subset generated based on a Lloyd-Max quantization technique, from the universal set (Choi: FIG. 4 illustrates an exemplary flowchart for performing kmeans clustering using Lloyd's algorithm, according to one embodiment ¶ [0020], [0022], [0060], [0061], [0087], A weight rearrangement method may rearrange weights for NN layers (for example, for CNN layers) ¶ [0163]).
Regarding claims 9, and 17, the combination of Iguchi, Li, and Choi discloses, wherein the deriving the scale parameter value for minimizing the determined quantization error comprises: determining an initial value of the scale parameter; updating the scale parameter based on a change of the quantization error (Iguchi: It is to be noted that the three-dimensional data encoding device may change the quantization scale to be used for each LoD. For example, the three-dimensional data encoding device reduces the quantization scale more for a higher layer, and increases the quantization scale more for a lower layer. The value of attribute information of a three-dimensional point belonging to a higher layer may be used as a predicted value of attribute information of a three-dimensional point belonging to a lower layer. Thus, it is possible to increase the coding efficiency by reducing the quantization scale for the higher layer to reduce the quantization error that can be generated in the higher layer and to increase the prediction accuracy of the predicted value ¶ [0662]. Also see ¶ [0749], [0759], [0796]); and
outputting the scale parameter when the change of the quantization error is less than or equal to a preset reference value (Iguchi: Inverse quantizer 1305 generates an inverse quantized coefficient of the prediction residual by performing inverse quantization on the quantized coefficient generated by quantizer 1304 using the quantization control parameter, and outputs the generated inverse quantized coefficient to inverse transformer 1306 ¶ [569]. Quantizer 1304 generates a quantized coefficient by performing quantization using a quantization control parameter on a frequency component of the prediction residual generated by transformer 1303. With this, the amount of information is further reduced. The generated quantized coefficient is outputted to entropy encoder 1313. Quantizer 1304 may control the quantization control parameter in units of worlds, units of spaces, or units of volumes ¶ [0567]. The threshold value (R_TH) is changed according to a quantization scale in quantization. With this, since the three-dimensional data encoding device can use the threshold value suitably according to the quantization scale, it is possible to increase the coding efficiency ¶ [0749]).
Regarding claims 10, and 18, the combination of Iguchi, Li, and Choi discloses, the scale parameter is derived through an iterative operation based on a following equation:
∀i,αqi=argmin|p -wi| ℒ=∑i(wi-αqi)2α*=Σiwi·qi/Σiqi2,
And α denotes the scale parameter, wi denotes an element of a subset, qj denotes a quantization point of the subset, L denotes the quantization error, and SQ={αqj}
(Choi: see ¶ [0057]-[0065], also see ¶ [0081]-[0085]).
Regarding claims 11, and 19, the combination of Iguchi, Li, and Choi discloses, wherein the scale parameter is set differently for each channel or each layer (Choi: network parameter quantization is performed by quantizing the network parameters of all layers of a deep neural network together at once [Abstract]. Also see ¶ [0093]-[0094]).
Regarding claims 12, and 20, the combination of Iguchi, Li, and Choi discloses, further comprising reperforming the quantization with the approximate weight on the selected pieces of data using the derived scale parameter value (Iguchi: It is to be noted that the three-dimensional data encoding device may change the quantization scale to be used for each LoD. For example, the three-dimensional data encoding device reduces the quantization scale more for a higher layer, and increases the quantization scale more for a lower layer. The value of attribute information of a three-dimensional point belonging to a higher layer may be used as a predicted value of attribute information of a three-dimensional point belonging to a lower layer. Thus, it is possible to increase the coding efficiency by reducing the quantization scale for the higher layer to reduce the quantization error that can be generated in the higher layer and to increase the prediction accuracy of the predicted value. It is to be noted that the three-dimensional data encoding device may add the quantization scale to be used for each LoD to, for example, a header. In this way, the three-dimensional data encoding device can decode the quantization scale correctly, thereby appropriately decoding the bitstream ¶ [0662]. Also see ¶ [0703], [0749], [0759], [0796]).
Regarding claim 13, the combination of Iguchi, Li, and Choi discloses, determining a quantized approximate weight based on the reperforming of the quantization (Choi: In the previous section, the local impact of Hessian-weighted quantization on the average loss function at w=ŵ was approximately quantified in Equation (7)(d) ¶ [0084], [0134]); and
quantizing a neural network using the quantized approximate weight (Choi: Apparatuses and methods of manufacturing same, systems, and methods for performing network parameter quantization in deep neural networks are described [Abstract]. Also see ¶ [0009]-[0014]).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MOHAMMAD S ROSTAMI whose telephone number is (571)270-1980. The examiner can normally be reached Mon-Fri From 9 a.m. to 5 p.m..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Boris Gorney can be reached at (571)270-5626. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
6/4/2026
/MOHAMMAD S ROSTAMI/Primary Examiner, Art Unit 2154
Prosecution Insights