DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 9/9/2026 has been entered.
Status of Claims
Claims 1-20 are pending of which claims 1, 8 and 15 are in independent form.
Claims 1-20 are rejected under 35 U.S.C. 103.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 2, and 6 are rejected under 35 U.S.C. 103 as being unpatentable over IGUCHI; Noritaka et al. (US 20210409769 A1) [Iguchi] in view of Li; Duanshun et al. (US 20220261616 A1) [Li] in view of COENEN; Ivo Leonardus (US 20210287074 A1) [Coenen].
Regarding claim 1, Iguchi discloses, an electronic device, the device comprising: a shifter circuit configured to perform a shift operation [based on a codebook supporting a plurality of quantization levels preset for data bits of a data set] (it is further possible that a first bit number is different from a second bit number, the first bit number being a number of bits shifted by each of the right bit shift and the left bit shift performed in the first decoding scheme, the second bit number being a number of bits shifted by each of the right bit shift and the left bit shift performed in the second decoding scheme, and in the first decoding scheme, the transforming of the first quantization parameter to the first scale value includes applying the first table to the first quantization parameter to determine the first scale value, and in the second decoding scheme, the transforming of the first quantization parameter to the first scale value includes applying the first table to the first quantization parameter to determine a third scale value, and performing bit shift on the third scale value by a number of bits corresponding to a difference between the first bit number and the second bit number ¶ [0202]-[0203]. Attribute information encoder 7921 includes left bit shifter 7941, transformer 7942, right bit shifter 7943, scale value calculator 7944, quantizer 7945, and entropy encoder 7946 ¶ [1295]. Also see ¶ [0200], [1295], [1299], [1301], [1304], [1312], [1377]. Examiner specifies that first table including a first scale has been interpreted as codebook); and
set quantization scales (Scale3/Scale4 generated by scale value calculator ¶ [1312], Quantizer 7945A/7945B performs quantization using scale value (Scale3/4) ¶ [1336]) of the data bits differently [for preset groups of a neural network] (Scale3 versus Scale4 ¶ [1312], different quantization operations using Scale3 and Scale4 ¶ [1336]. Quantization scale…in layer I ¶ [0693], [0796], Also see ¶ [0202]);
quantizes (Quantizer 7945A performs both the right bit shift performed by right bit shifter 7943 and the quantization process performed by quantizer 7945 shown in FIG. 165 by dividing the shifted coefficient value obtained by transformer 7942A by the scale value (Scale3) generated by scale value calculator 7952B. Similarly, quantizer 7945B performs both the right bit shift performed by right bit shifter 7943 and the quantization process performed by quantizer 7945 shown in FIG. 165 by dividing the shifted coefficient value obtained by transformer 7942B by the scale value (Scale4) generated by scale value calculator 7952B ¶ [1336]. Also see ¶ [1295], [1299], [1304], [1310], [1312], [1345]-[1348], [1365], [1367]-[1368], [1377]) and outputs the data bits (Quantizer 7945A performs both the right bit shift performed by right bit shifter 7943 and the quantization process performed by quantizer 7945 … by dividing the shifted coefficient value obtained by transformer 7942A by the scale value (Scale3) generated by scale value calculator 7952B ¶ [1336]) based on the set quantization scales (This reconstructed volume is outputted as decoded three-dimensional data ¶ [0600], [0741], [1201], That is, inverse quantizer 7953C performs both the inverse quantization process performed by inverse quantizer 7953A and the left bit shift performed by left bit shifter 7954A shown in FIG. 167 by multiplying the quantized coefficient obtained by entropy decoder 7951A by a scale value (Scale3) generated by scale value calculator 7952B. Similarly, inverse quantizer 7953D performs both the inverse quantization process performed by inverse quantizer 7953B and the left bit shift performed by left bit shifter 7954B shown in FIG. 167 by multiplying the quantized coefficient obtained by entropy decoder 7951B by a scale value (Scale4) generated by scale value calculator 7952B ¶ [1312], [1343]; different scales for higher and lower layers ¶ [0662], layer specific quantization scale ¶ [0693], [0796], Scale3 and Scale4 ¶ [1312], [1336]).
However, Iguchi does not explicitly facilitate based on a codebook supporting a plurality of quantization levels preset for data bits of a data set; for preset groups of a neural network; for the preset groups.
Li discloses, based on a codebook supporting a plurality of quantization levels preset for data bits of a data set (the cluster centers identified during clustering may be used to form the codebook… A quantized weight tensor may be represented with a codebook ¶ [0173], scalar quantization may cluster the n elements into k clusters ¶ [0171], Scalar quantization and vector quantization may be effective quantization in NN compression ¶ [0157]);
for preset groups of a neural network; for the preset groups (obtain an NN model including an NN layer associated with a weight matrix ¶ [0124], Clustering-based quantization, as disclosed herein, may address tensor arrangement of CNN layers and/or may reduce the impact of outliers on clustering (for example, during scalar quantization and/or vector quantization of NN weights) ¶ [0160], Scalar quantization may cluster the n elements into k clusters ¶ [0171], clustering-based quantization or inverse quantization for NN ¶ [0193]).
It would have been obvious to one ordinary skilled in the art at the time of the filing of the present invention to combine the teachings of the cited references because Li’s system would have allowed Iguchi to facilitate based on a codebook supporting a plurality of quantization levels preset for data bits of a data set; for preset groups of a neural network; for the preset groups. The motivation to combine is apparent in the Iguchi’s reference, because there is a need to improve reducing the storage and/or transmission bandwidth needed for models.
However neither Iguchi nor Li explicitly facilitates a decoder circuit configured to: generate a control signal that controls the shifter circuit; wherein the shifter circuit, based on the control signal generated by the decoder circuit.
Coenen discloses, a decoder circuit configured to: generate a control signal that controls the shifter circuit; wherein the shifter circuit, based on the control signal generated by the decoder circuit (The weight decoder 511 includes an index word decoder 544, a plurality of shifters 540, and a plurality of digital logic gates 542. The shifters 540 includes a first shifter 540-1, and a second shifter 540-2 through N shifter 540-N. The number of shifters 540 may be dependent on the amount of decoded words provided to the input-weight multipliers 508. In some examples, the shifters 540 are connected to the index word decoder 544 in parallel, which may make the decoding relatively fast. In some examples, each shifter 540 is configured to receive the sequence of non-pruned weight words 520 that follow the index weight word 518 and a control signal from the index word decoder 544 that determines the shift operation applied to the sequence of non-pruned weight words 520 ¶ [0067]-[0069]; The decompression logic circuit 810 includes a shifter 880 ¶ [0081], [0089]; FIG. 5 illustrates a decompression logic circuit for decoding the encoded weights according to an aspect ¶ [0017]; more specifically, Fig. 5 teaches, decoder 544 generating control signals for respective shifter 540; and shifting according to control signals).
It would have been obvious to one ordinary skilled in the art at the time of the filing of the present invention to combine the teachings of the cited references because Coenen’s system would have allowed Iguchi and Li to facilitate a decoder circuit configured to: generate a control signal that controls the shifter circuit; wherein the shifter circuit, based on the control signal generated by the decoder circuit. The motivation to combine is apparent in Iguchi and Li’s reference, because there is a need to improve processing power to perform to perform the NN signal processing.
Regarding claim 2, the combination of Iguchi, Li and Coenen discloses, wherein the quantization scales are determined based on a scale parameter for minimizing a quantization error when quantizing the data set with an approximate weight in a preset operation (Iguchi: an error (quantization error) which may be generated by quantization reduces as the quantization scale is smaller. In the other case where the quantization scale is larger, the resulting quantization error is larger ¶ [0661]-[0662], [0673]. In this case, the smaller the quantization scale, the smaller the error (quantization error) that may occur due to quantization. Conversely, the larger the quantization scale, the larger the quantization error ¶ [0795]-[0796]).
Regarding claim 6, the combination of Iguchi, Li and Coenen discloses, wherein the data bits are quantized at a log level of 2 (Iguchi: the scale and the quantization value (quantization parameter (QP) value) are expressed by Equation G3 below. quantization value(QP value)=log(scale) (Equation G3); quantization value(QP value)=default value(reference value)+quantization delta(difference information) (Equation G4) ¶ [0870]).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 3-5 and 7-20 are rejected under 35 U.S.C. 103 as being unpatentable over Iguchi in view of Li in view of Coenen in view of CHOI; Yoo Jin et al. (US 20180107925 A1) [Choi].
Regarding claim 3, the combination of Iguchi, Li and Coenen teaches all the limitation of claim 1.
However, neither one of the Iguchi, Li or Coenen explicitly facilitate wherein the data set corresponds to a subset selected from some pieces of data, of which a similarity is high with a subset generated based on a Lloyd-Max quantization technique, of a randomly generated universal set.
Choi discloses, wherein the data set corresponds to a subset selected from some pieces of data, of which a similarity is high with a subset generated based on a Lloyd-Max quantization technique, of a randomly generated universal set (FIG. 4 illustrates an exemplary flowchart for performing k-means clustering using Lloyd's algorithm, according to one embodiment ¶ [0020], [0022], [0060], [0061], [ 0087]).
It would have been obvious to one ordinary skilled in the art at the time of the present invention to combine the teachings of the cited references because Choi’s system would have allowed Iguchi, Li and Coenen to facilitate wherein the data set corresponds to a subset selected from some pieces of data, of which a similarity is high with a subset generated based on a Lloyd-Max quantization technique, of a randomly generated universal set. The motivation to combine is apparent in Iguchi, Li and Coenen’s reference, because there is a need to improve deploying deep neural networks on devices with limited storage, such as mobile/portable devices.
Regarding claim 4, the combination of Iguchi, Li, Coenen and Choi discloses, wherein the preset groups comprise either one or both of a channel and a layer of a neural network (Choi: network parameter quantization is performed by quantizing the network parameters of all layers of a deep neural network together at once [Abstract]. Also see ¶ [0093]-[0094]).
Regarding claim 5, the combination of Iguchi, Li, Coenen and Choi discloses, the scale parameter is derived through an iterative operation based on a following equation:
∀i,αqi=argmin|p -wi| ℒ=∑i(wi-αqi)2α*=Σiwi·qi/Σiqi2,
And α denotes the scale parameter, wi denotes an element of a subset, qj denotes a quantization point of the subset, L denotes the quantization error, and SQ={αqj}
(Choi: see ¶ [0057]-[0065], also see ¶ [0081]-[0085]).
Regarding claims 7, 14 and 15, the combination of Iguchi, Li, Coenen and Choi discloses, a processor-implemented method, the method comprising: selecting some pieces of data from a quantized universal set in a preset operation (Iguchi: Quantizer 7004 quantizes the coefficient value to generate a quantized value. Quantization controller 7005 controls a quantization parameter used for the quantization by quantizer 7004. For example, quantization controller 7005 may change the quantization parameter (or quantization step) according to the hierarchical structure for the encoding. In this way, an appropriate quantization parameter can be selected for each layer of the hierarchical structure, so that the amount of codes occurring in each layer can be controlled. Quantization controller 7005 also sets quantization parameters for a certain layer and the layers lower than the certain layer that include a frequency component that has a small effect on the subjective image quality at a maximum value, and sets quantization coefficients for the certain layer and the layers lower than the certain layer at 0, for example ¶ [1109]);
performing quantization with an approximate weight on the selected pieces of data (Choi: In the previous section, the local impact of Hessian-weighted quantization on the average loss function at w=ŵ was approximately quantified in Equation (7)(d) ¶ [0084], [0134]);
determining a quantization error for the quantized pieces of data (Iguchi: an error (quantization error) which may be generated by quantization reduces as the quantization scale is smaller. In the other case where the quantization scale is larger, the resulting quantization error is larger ¶ [0661]-[0662], [0673]. In this case, the smaller the quantization scale, the smaller the error (quantization error) that may occur due to quantization. Conversely, the larger the quantization scale, the larger the quantization error ¶ [0795]-[0796]); and
deriving a scale parameter value for minimizing the determined quantization error (Iguchi: For example, the three-dimensional data encoding device performs quantization by dividing the prediction residual by a quantization scale (also referred to as a quantization step). In this case, an error (quantization error) which may be generated by quantization reduces as the quantization scale is smaller. In the other case where the quantization scale is larger, the resulting quantization error is larger ¶ [0661]-[0662]).
representing neural network weights; generating quantized neural network weights (Li: analyze the distribution of one or more NN weights in weight tensors in NN layers ¶ [0004]-[0007]; )
Regarding claims 8, and 16, the combination of Iguchi, Li, Coenen and Choi discloses, wherein the selecting the pieces of data in the preset operation comprises selecting the pieces of data, of which a similarity is high with a subset generated based on a Lloyd-Max quantization technique, from the universal set (Choi: FIG. 4 illustrates an exemplary flowchart for performing kmeans clustering using Lloyd's algorithm, according to one embodiment ¶ [0020], [0022], [0060], [0061], [0087], A weight rearrangement method may rearrange weights for NN layers (for example, for CNN layers) ¶ [0163]).
Regarding claims 9, and 17, the combination of Iguchi, Li, Coenen and Choi discloses, wherein the deriving the scale parameter value for minimizing the determined quantization error comprises: determining an initial value of the scale parameter; updating the scale parameter based on a change of the quantization error (Iguchi: It is to be noted that the three-dimensional data encoding device may change the quantization scale to be used for each LoD. For example, the three-dimensional data encoding device reduces the quantization scale more for a higher layer, and increases the quantization scale more for a lower layer. The value of attribute information of a three-dimensional point belonging to a higher layer may be used as a predicted value of attribute information of a three-dimensional point belonging to a lower layer. Thus, it is possible to increase the coding efficiency by reducing the quantization scale for the higher layer to reduce the quantization error that can be generated in the higher layer and to increase the prediction accuracy of the predicted value ¶ [0662]. Also see ¶ [0749], [0759], [0796]); and
outputting the scale parameter when the change of the quantization error is less than or equal to a preset reference value (Iguchi: Inverse quantizer 1305 generates an inverse quantized coefficient of the prediction residual by performing inverse quantization on the quantized coefficient generated by quantizer 1304 using the quantization control parameter, and outputs the generated inverse quantized coefficient to inverse transformer 1306 ¶ [569]. Quantizer 1304 generates a quantized coefficient by performing quantization using a quantization control parameter on a frequency component of the prediction residual generated by transformer 1303. With this, the amount of information is further reduced. The generated quantized coefficient is outputted to entropy encoder 1313. Quantizer 1304 may control the quantization control parameter in units of worlds, units of spaces, or units of volumes ¶ [0567]. The threshold value (R_TH) is changed according to a quantization scale in quantization. With this, since the three-dimensional data encoding device can use the threshold value suitably according to the quantization scale, it is possible to increase the coding efficiency ¶ [0749]).
Regarding claims 10, and 18, the combination of Iguchi, Li, Coenen and Choi discloses, the scale parameter is derived through an iterative operation based on a following equation:
∀i,αqi=argmin|p -wi| ℒ=∑i(wi-αqi)2α*=Σiwi·qi/Σiqi2,
And α denotes the scale parameter, wi denotes an element of a subset, qj denotes a quantization point of the subset, L denotes the quantization error, and SQ={αqj}
(Choi: see ¶ [0057]-[0065], also see ¶ [0081]-[0085]).
Regarding claims 11, and 19, the combination of Iguchi, Li, Coenen and Choi discloses, wherein the scale parameter is set differently for each channel or each layer (Choi: network parameter quantization is performed by quantizing the network parameters of all layers of a deep neural network together at once [Abstract]. Also see ¶ [0093]-[0094]).
Regarding claims 12, and 20, the combination of Iguchi, Li, Coenen and Choi discloses, further comprising reperforming the quantization with the approximate weight on the selected pieces of data using the derived scale parameter value (Iguchi: It is to be noted that the three-dimensional data encoding device may change the quantization scale to be used for each LoD. For example, the three-dimensional data encoding device reduces the quantization scale more for a higher layer, and increases the quantization scale more for a lower layer. The value of attribute information of a three-dimensional point belonging to a higher layer may be used as a predicted value of attribute information of a three-dimensional point belonging to a lower layer. Thus, it is possible to increase the coding efficiency by reducing the quantization scale for the higher layer to reduce the quantization error that can be generated in the higher layer and to increase the prediction accuracy of the predicted value. It is to be noted that the three-dimensional data encoding device may add the quantization scale to be used for each LoD to, for example, a header. In this way, the three-dimensional data encoding device can decode the quantization scale correctly, thereby appropriately decoding the bitstream ¶ [0662]. Also see ¶ [0703], [0749], [0759], [0796]).
Regarding claim 13, the combination of Iguchi, Li, Coenen and Choi discloses, determining a quantized approximate weight based on the reperforming of the quantization (Choi: In the previous section, the local impact of Hessian-weighted quantization on the average loss function at w=ŵ was approximately quantified in Equation (7)(d) ¶ [0084], [0134]); and
quantizing a neural network using the quantized approximate weight (Choi: Apparatuses and methods of manufacturing same, systems, and methods for performing network parameter quantization in deep neural networks are described [Abstract]. Also see ¶ [0009]-[0014]).
Conclusion
The examiner requests, in response to this Office action, support be shown for language added to any original claims on amendment and any new claims. That is, indicate support for newly added claim language by specifically pointing to page(s) and line no(s) in the specification and/or drawing figure(s). This will assist the examiner in prosecuting the application.
When responding to this office action, Applicant is advised to clearly point out the patentable novelty which he or she thinks the claims present, in view of the state of the art disclosed by the references cited or the objections made. He or she must also show how the amendments avoid such references or objections See 37 CFR 1.111(c).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MOHAMMAD S ROSTAMI whose telephone number is (571)270-1980. The examiner can normally be reached Mon-Fri From 9 a.m. to 5 p.m..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Boris Gorney can be reached at (571)270-5626. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
9/17/2026
/MOHAMMAD S ROSTAMI/Primary Examiner, Art Unit 2154
Prosecution Insights