DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
The rejection related to 35 USC § 101 regarding to claim 1-20 is withdrawn.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5, 8-20 are rejected under 35 U.S.C. 103 as being unpatentable over Lu et al. (Lu) US 2021/0089922 in view of Asad et al. (Asad) US 2022/0044096 and Van Baalen et al. (Van Baalen) US 20230058159
In regard to claim 1, Lu disclose A method comprising: ([0004] method)
computing gradient function estimation values for two or more design options of three or more of a plurality of design decisions for at least one layer of a neural network of a processor design, ([0004]-[0007] [0023]-[00031] [0039]-[0045] [0054]-[0063] computer a gradient vector for the weights to approximates the error gradient such the error rate has reached a target level, in a NN design, the various weights, activation parameters combinations are multiple design options, different bit-width specified for layers of deep NN, with various memory constraint or other constraints or various combinations of these constrains on the multiple layers of NN, there are more than 3 design decisions. “determining a pruning ratio for a channel and a mixed-precision quantization based on the memory budget or the computation budget; quantizing at least one of a weight parameter of the deep neural network, an activation parameter of the deep neural network, or a combination thereof based on the mixed-precision quantization;”) based, at least in part, on weights applicable to nodes and/or edges in the neural network and coefficients associated with respective design options, ([0004]-[0007] [0023]-[00031] [0030]-[0045] [0051]-[0063] [0066]-[0084] and pruning the channel of the deep neural network based on the pruning ratio” determine a joint pruning and quantization function based on weights and activation parameters, etc.) each of the design decisions having a plurality of such design options; ([0030]-[0045] [0051]-[0063] quantize weights parameters and activation values for the deep NN are many design choices of the each NN design based on the target)
the three or more of the plurality of the design decisions comprising at least one of layer width, number of channels, weight bitwidth, activation bitwidth, operator type, network connectivity, network depth, weight sparsity level, or activation resolution; ([0004]-[0007] [0023]-[00034] [0039]-[0045] [005]-[0063] in a NN design, the various weights, activation parameters combinations are multiple design options, different bit-width specified for layers of deep NN, with various memory constraint or other constraints or various combinations of these constrains on the multiple layers of NN, network connectivity. Weight parameter and activation parameter based on quantization bit-width) and
applying the objective function subject to one or more processing constraints to select the design options for the three or more of the plurality of design decisions of the at least one layer of the neural network. ([0054]-[0063][0066]-[0084] for example, BOP is the constraint, apply the BOP count in the loss function for joint optimization of pruning and quantization parameters. (weights and activation parameters) for one layer of the NN and the BOP can be replaced by another resource constrain, memory cost, MAC constraint, etc.)
But Lu fail to explicitly disclose “determining an objective function based, at least in part, on a combination of the computed gradient function estimation values associated with the three of more of the plurality of design decisions, applying the objective function based on the combination of the computed gradient function estimation values subject to one or more processing constraints to identify to generate an optimized neural network;”
Asad disclose determining an objective function based, at least in part, on a combination of the computed gradient function estimation values associated with the three of more of the plurality of design decisions, ([0014]-[0025] [0044]-[0052] [0135]-[0142] [0168]-[0179] [0182]-[0195] by combining (a sum of squared errors with outlier weighting or mean variance of) the gradients with respect the network output associated with multiple design decisions, such as kernel height, kernel width, number of input and output channels, etc. using the number format selection algorithm) to determine a statistics)
applying the objective function based on the combination of the computed gradient function estimation values subject to one or more processing constraints to select the formats to generate an optimized neural network; ([0044]-[0047] [0135]-[0142][0147]-[0162] [0168]-[0179] [0182]-[0195] using the suitable statistics (a sum of squared errors with outlier weighting or mean variance) associated with multiple design decisions, to select formats and derive the optimized NN based on the computer resources required to run the algorithm Note: please further clarify how the combination of the computed gradient function estimation values is associated with the design decisions to help move forward the prosecution, such as the steps, etc. and please use functional language to define the claim limitations)
It would have been obvious to one having ordinary skill in the art before the effective filing data of the claimed invention was made to incorporate Asad’s method of NN design into Lu’s invention as they are related to the same field endeavor of NN training and generation. The motivation to combine these arts, as proposed above, at least because Asad’s of NN design method based on the gradient criteria would help to provide more NN design method into Lu’s system. Therefore it would have been obvious to one having ordinary skill in the art before the effective filing data of the claimed invention was made providing more NN design method based on gradient criteria that would help to improve efficiency of the neural network.
But Lu and Asad failed to explicitly disclose “to generate the optimized neural network, the selected design options representing design options for the three or more of the plurality of design decisions optimized for optimized neural network predictive performance subject to the one or more processing constraints.”
Van Baalen disclose to generate the optimized neural network, [0016] [0121] generate the NN)
the selected design options representing design options for the three or more of the plurality of design decisions optimized for optimized neural network predictive performance subject to the one or more processing constraints. ([0042]-[0047][0056]-[0075] [0137]optimized design options are selected for the optimized NN with different bit-width configurations and sparsity, etc. its predictive performance depends upon the application and design constraints given the input tensor)
It would have been obvious to one having ordinary skill in the art before the effective filing data of the claimed invention was made to incorporate Van Baalen’s method of NN quantization into Asad and Lu’s invention as they are related to the same field endeavor of NN training and generation. The motivation to combine these arts, as proposed above, at least because Van Baalen’s method of NN quantization based on mixed-precision quantization and structured pruning would help to provide more NN design method into Asad and Lu’s system. Therefore it would have been obvious to one having ordinary skill in the art before the effective filing data of the claimed invention was made providing more NN design method based on mixed-precision quantization and structured pruning that would help to improve accuracy or efficiency for the neural network.
In regard to claim 2, Lu and Asad, Van Baalen disclose The method of claim 1, the rejection is incorporated herein.
Lu disclose wherein the combination of computed gradient function estimation values comprises first computed gradient function estimation values associated with a design decision for bit width and second computed gradient function estimation values associated with a design decision for a random pruning approach, ([0004]-[0007] [0023]-[00031] [0030]-[0045] [0051]-[0063] the joint pruning and quantization function based on weights for bit width applied to the nodes of the NN with pruning the channel based on the pruning ratio, etc.) and wherein the objective function is to optimize the at least one layer with respect to design decisions for bit width and the random pruning approach concurrently. ([0023]-[0031] [0054]-[0063][0066]-[0076] both a pruning ration and quantization bit width are optimized to compress the DNN to satisfy a predefined memory and computation budget based on the equations)
In regard to claim 3, Lu and Asad, Van Baalen disclose The method of claim 1, the rejection is incorporated herein.
Lu disclose wherein the computed gradient function estimation values are associated with at least a design decision for bit width conditioned on a design decision for a random pruning approach. ([0023]-[0031] [0051]-[0063][0066]-[0076] joint pruning and quantization bit width are optimized to compress the DNN to satisfy a predefined memory and computation budget, so there are conditioned for each other or given a target pruning ratio, fine tunes weights.)
In regard to claim 4, Lu and Asad, Van Baalen disclose The method of claim 1, the rejection is incorporated herein.
Lu disclose wherein the computed gradient function estimation values are associated with at least a design decision for a random pruning approach conditioned on a design decision for bit width. ([0023]-[0031] [0051]-[0063][0066]-[0076] joint pruning and quantization bit width are optimized to compress the DNN to satisfy a predefined memory and computation budget, so there are conditioned for each other)
In regard to claim 5, Lu and Asad, Van Baalen disclose The method of claim 1, the rejection is incorporated herein.
Lu disclose wherein the objective function is determined for iterations of computations of the computed gradient function estimation values, [0039]-[0042] adjust the weights based on the error and repeated until the error rate reached a target value) and the method further comprises:
updating coefficients to compute the gradient function estimation values based, at least in part, on the objective function determined based, at least in part, on a first iteration of the gradient function estimation values, the updated coefficients to be applied in computation of the gradient function estimation values in a second, subsequent iteration of the gradient function estimation values. ([0036]-[0040] adjust the weights based on the error generated and calculation is repeated until the error rate reached a target value)
In regard to claim 8, Lu and Asad, Van Baalen disclose The method of claim 1, the rejection is incorporated herein.
Lu disclose wherein coefficients associated with design options of at least a first design decision of the three or more of the plurality of design decisions are categorically distributed. ([0051]-[0063] quantization approaches summarized into two categories and pruning approaches are summarized into two categories too)
In regard to claim 9, Lu and Asad, Van Baalen disclose The method of claim 1, the rejection is incorporated herein.
Lu disclose wherein the combination of computed gradient function estimation values associated with the design decisions comprises a sum of the computed gradient function estimation values. ([[0059]-0063][0077]- [0079] bit width with a summation equation of function values)
In regard to claim 10, Lu and Asad, Van Baalen disclose The method of claim 1, the rejection is incorporated herein.
Lu disclose wherein determining the objective function further comprises determining the objective function subject to the one or more processing constraints. ([0050]-[0063][0066]-[0076] determine a joint pruning and quantization function to optimize trade-off between a quantizing bit-width and a pruning ratio of the DNN to compress the DNN to satisfy a predefined memory and computation budget based on the equations)
In regard to claim 11, Lu and Asad, Van Baalen disclose The method of claim 10, the rejection is incorporated herein.
Lu disclose wherein at least one of the one or more processing constraints comprises an availability of memory. ([0050]-[0063][0066]-[0076] determine a joint pruning and quantization function to optimize trade-off between a quantizing bit-width and a pruning ratio of the DNN to compress the DNN to satisfy a predefined memory)
In regard to claim 12, Lu and Asad, Van Baalen disclose The method of claim 10, the rejection is incorporated herein.
Lu disclose wherein at least one of the one or more processing constraints is represented by a first numerical value and at least one attribute of the processor design is associated with a second numerical value, ([0021]-[0024] [0050]-[0063][0066]-[0076] the constraints are memory and computation budget constraints, etc. first value maybe bit-width b, the second value maybe (power-of-two representable value) based on the hardware configuration) and wherein the objective function is determined at, least in part, on an expected absolute value of a difference between the first and second numerical values. ([0021]-[0024] [0050]-[0063][0066]-[0076] first value maybe bit-width b, the second value maybe (power-of-two representable value) based on the hardware configuration, b needed to rounded to the nearest power-of-tow representable value based on the difference between the two values)
In regard to claims, claims 13-17 are computing device claims corresponding to the method claims 1-4, 10 above and, therefore, are rejected for the same reasons set forth in the rejections of claims 1-4, 10.
In regard to claims, claims 18-20 are article claims corresponding to the method claims 1-3 above and, therefore, are rejected for the same reasons set forth in the rejections of claims 1-3.
Claims 6-7 are rejected under 35 U.S.C. 103 as being unpatentable over Lu et al. (Lu) US 2021/0089922, Asad et al. (Asad) US 2022/0044096 and Van Baalen et al. (Van) US 20230058159 as applied to claim 1, further in view of Teramae US 2022/0188603
In regard to claim 6, Lu and Asad, Van Baalen disclose The method of claim 5, the rejection is incorporated herein.
Lu disclose wherein the updated coefficients are determined based, at least in part, according to a sampling of coefficients applied in computation of gradient function estimation values in a preceding iteration of gradient function estimation values. ([0036]-[0040] adjust the weights according to a small number of examples, sampled based on the error gradients generated and calculation is repeated until the error rate reached a target value)
Lu and Asad, Van Baalen fail to explicitly disclose “wherein the updated coefficients are determined based, according to a Markov chain monte carlo sampling of coefficients.”
Teramae disclose wherein the updated coefficients are determined based, according to a Markov chain monte carlo sampling of coefficients. ([0008]-[0017][0093][0152] the weights are determined based on MCMC sampling from a conditional probability distribution under a condition)
It would have been obvious to one having ordinary skill in the art before the effective filing data of the claimed invention was made to incorporate Teramae’s sampling method into Van Baalen, Asad and Lu’s invention as they are related to the same field endeavor of NN training and generation. The motivation to combine these arts, as proposed above, at least because Teramae’s Markov chain monte carlo sampling would help to provide more sampling method to use to Van Baalen, Asad and Lu’s system. Therefore it would have been obvious to one having ordinary skill in the art before the effective filing data of the claimed invention was made providing Markov chain monte carlo sampling that would help to facilitate train and learn of the neural network.
In regard to claim 7, Lu, and Asad, Van Baalen and Teramae disclose The method of claim 6, the rejection is incorporated herein.
Lu disclose wherein the updated coefficients are further determined based, at least in part, on a gradient operation applied to iterations of the objective function. ([0036]-[0040] adjust the weights based on the error gradients of higher layers generated and calculation is repeated until the error rate reached a target value to reduce the error)
Response to Arguments
Applicant's arguments filed on 1/6/2026 respect to claims 1-20 have been fully considered but they are not persuasive.
Conclusion
The prior art made of record and not relied upon is considered pertinent to Applicant's disclosure.
PATENT PUB. # PUB. DATE INVENTOR(S) TITLE
US 20200202199 A1 2020-06-25 LEE et al.
NEURAL NETWORK PROCESSING METHOD AND APPARATUS BASED ON NESTED BIT REPRESENTATION
LEE et al. disclose A neural network processing method and apparatus based on nested bit representation is provided. The processing method includes obtaining first weights for a first layer of a source model of a first layer of a neural network, determining a bit-width for the first layer of the neural network, obtaining second weights for the first layer of the neural network by extracting at least one bit corresponding to the determined bit-width from each of the first weights for the first layer of a source model corresponding to the first layer of the neural network, and processing input data of the first layer of the neural network by executing the first layer of the neural network based on the obtained second weights… see abstract.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to XUYANG XIA whose telephone number is (571)270-3045. The examiner can normally be reached Monday-Friday 8am-4pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Welch can be reached at 571-272-7212. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
XUYANG XIA
Primary Examiner
Art Unit 2143
/XUYANG XIA/Primary Examiner, Art Unit 2143