Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Claims 1-20 are presented for examination.
Claims 1, 12, 13, 14, 17, 18 and 20 were amended.
This is a Non-Final Action.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 06/04/2020 has been entered.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
101 abstract idea has been obviated due to current amendment to the claims.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 rejected under 35 U.S.C. 103 as being unpatentable over Ahmed et al. (“ReLeQ: A Reinforcement Learning Approach for Deep Quantization of Neural Networks” – IDS) in view of Ovtcharov et al. (WO2020/190542) further in view of Gong et al. (“Mixed Precision Neural Architecture Search for Energy Efficient Deep Learning” – (NPL – IEEE))
1. Ahmed teaches, A computer-implemented method for quantizing a neural network model while accounting for performance (Page 3; Col 1, first paragraph teaches – “RELEQ is automated method for efficient… different results”), the method comprising:
receiving, by a computing system comprising one or more computing devices, a reference neural network model (Fig 4, Page 2, Col 1, first paragraph teaches - “The RL agent starts from a full-precision previously trained model and learns the sensitivity of final classification accuracy with respect to the quantization of each layer”, Ahmed);
modifying, by the computing system, the reference neural network model to generate a candidate neural network model (Fig 4, Page 3, Col 1, Paragraph 2 - teaches steps sequentially through the layers and chooses a bitwidth from a predefined set…; Page 5, Col 1, last paragraph teaches - weights for this layer are quantized to the predicted bitwidth, Ahmed), wherein the candidate neural network model is generated by selecting, based on policy one or more values from a first searchable subspace and one or more values from a second searchable subspace (Table 1, Page 3, Col 1: Sec 2.3, 1st paragraph - teaches the agent… chooses a bitwidth from a predefined set… per layer… table 1 shows parameters used to embed the state space, including layer-specific parameters such as layer dimensions and quantization level (bitwidth); Page 5: Col 2: 1st paragraph – teaches “Policy in terms of neural network quantization is to learn to choose the optimal bitwidth for each layer in the networt, further Fig 5 – teaches the evolution of RELEQ agent’s bitwidth selection probabilities…Ahmed), wherein the first searchable subspace corresponds to a quantization scheme for quantizing one or more values of a layer of the candidate neural network model (Page 3, Col 1, Sec 2.3, 1st paragraph - teaches in order to consider the effects of previous layer’s quantization levels, the agent steps sequentially through the layers and chooses bitwidth… the set of bitwidth is (1. 2. 3. 4. 5. 6. 7, 8)… the quantization level of each layers, Ahmed);
executing, by the computer system, the layer of evaluate one or more performance metrics of the candidate neural network model (Page 5, Col 1, 2nd paragraph: “Weights for this particular layer are quantized to the predicted bitwidth…; Page 5: Col 2: 1st paragraph: “we perform a long retraining step using the quantized bidwidths predicted by the agent and then obtain the final accuracy for the quantized version of the network – thus disclosing executing/evaluating the quantized candidate model using accuracy and cost metrics, Ahmed); and
outputting, by the computing system, a new neural network model based at least in part on the one or more performance metrics (Page 5, Sec 3, Col 2, 1st paragraph - teaches after the learning process is complete and the agent has converged… perform a long retraining… then obtain the final accuracy for the quantized version of the network, Ahmed).
Ahmed further teaches
the second searchable subspace of the candidate neural network model. ( Table 1 - teaches a layer dimensions listed under layer-specific/static parameter, Ahmed).
However,
Ahmed does not explicitly teach,
the second searchable subspace corresponds to a size of a layer of the candidate neural network model; and
…wherein modifying the reference neural network model comprises at least one of: increasing a quantity of calculations for executing the layer while decreasing a precision of weights in the layer, or decreasing a quantity of calculations for executing the layer while increasing a precision of weights in the layer.
However, Ovtcharov teaches,
the second searchable subspace corresponds to a size of a layer of the candidate neural network model (Paragraph 105 - teaches determines network topology parameters including number of filters and number of neurons per layers).
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which said subject matter pertains to allow Ahmed’s invention to be combined with Ovtcharov’s invention because both prior arts are in the same field of machine learning model optimization with focus on quantization and architecture tuning.
Further Gong teaches,
…wherein modifying the reference neural network model comprises at least one of: increasing a quantity of calculations for executing the layer while decreasing a precision of weights in the layer, or decreasing a quantity of calculations for executing the layer while increasing a precision of weights in the layer (Abstract - “searches for the optimal combination of architectures and precisions (bit-widths) to directly optimize both the prediction accuracy and hardware energy consumption” and Page 1: Col 2: Paragraph 2 - teaches the optimal choices of bit-widths and architectures are correlated; Page 1: Col 2: Paragraph 3 - further teaches “co-optimizing both precision and NAS”; Page 3: col 2: Paragraphs 1-3 teaches “filter size… and expansion ratio” and precision choice “b1 , b2 E {2, 4, 6, 8}”; Page 4: Col 2: paragraph 4 – further teaches “bottleneck layers with small expand ratios (fewer parameters) tend to have larger bitwidth to preserve good accuracy”).
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which said subject matter pertains to modify the combination of Ahmed/Ovtcharov with Gong because Gong expressly teaches that neural architecture choices and bitwidth/precision choices are correlated and should be co-optimized to improve accuracy and hardware energy consumption. Gong’s architecture parameters affect calculation quantity, while its bitwidth parameters affect weight precision.
2. The combination of Ahmed, Ovtcharov and Gong teach, The computer-implemented method of claim 1, wherein modifying, by the computing system, the reference neural network model to generate the candidate neural network model comprises:
selecting, by the computing system, the one or more values from the first searchable subspace (Page 3, Col 1: Sec 2.3, 1st paragraph - teaches the agent chooses a bitwidth from a predefined set … per layer, Ahmed) and the one or more values from the second searchable subspace (Table 1 – teaches parameters including layer dimensions and quantization level (bitwidth), Ahmed) using a controller model (Page 3, Section 2.3, Col 1: 2nd paragraph - teaches ReLeQ trains a reinforcement learning agent… policy and value networks… select quantization levels…, Ahmed).
3. The combination of Ahmed, Ovtcharov and Gong teach, The computer-implemented method of claim 2, wherein outputting, by the computing system, the new neural network model comprises:
updating, by the computing system, the controller model based at least in part on the one or more performance metrics (Page 3, Section 2.3, Col 1: 2nd paragraph - teaches reward signal… proportional to its accuracy after quantization and its benefits in terms of computation and memory cost… Page 5: col 2: 1st paragraph - use proximal policy optimization… to update the policy and value networks of RELEQ agent, Ahmed); and
generating, by the computing system, the new neural network model using the updated controller model (Page 5, Col 2: 1st paragraph - teaches after.. the agent has converged to a quantization level for each layer… perform long retraining… obtain final accuracy for the quantized version of the network, Ahmed).
4. The combination of Ahmed, Ovtcharov and Gong teach, The computer-implemented method of claim 2, wherein the controller model comprises a reinforcement learning agent (Page 5: col 2: 1st paragraph – teaches RELEQ trains a reinforcement learning agent… employs… Proximal policy Optimization … consists of both policy and value networks).
5. The combination of Ahmed, Ovtcharov and Gong teach, The computer-implemented method of claim 1, wherein the quantization scheme is selected from binary and ternary schemes (P. 3, section 2.3, Col 1: 2nd paragraph; section 2.4 Col 2: 2nd paragraph - teaches bitwidth from a predefined set (1..8) … includes ternary (2-bit) and binary (1-bit) as possible quantization levels, Ahmed).
The combination of Ahmed and Ovtcharov do not explicitly teach,
… modified binary, exponent, and mantissa quantization schemes.
However, It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which said subject matter pertains to allow the Ahmed to extend the quantization schemes selectable in RELEQ to include other well-known schemes such as modified binary, exponent-based, and mantissa-based quantization. At the time of invention, quantization was mature area of deep learning optimization, and these schemes were recognized equivalent to binary and ternary for representing neural network weights and activations. The choice among these known schemes would have been a matter of design selection and optimization for hardware or accuracy requirements, yielding predictable results. Modified binary was understood to improve dynamic range over pure binary; exponent and mantissa quantization were recognized as common low-precision floating-point techniques. Substituting or adding such known schemes into RELEQ’s quantization search space would have been no more than the predictable use of prior art elements according to their established functions, and would have been well within the routine skill of an ordinary artisan, requiring no more than ordinary creativity.
6. The combination of Ahmed, Ovtcharov and Gong teach, The computer-implemented method of claim 1, wherein the second searchable subspace corresponds to at least one of a quantity of output units and a quantity of filters (Paragraphs 29 - controller determines network topology parameters including number of filters per convolutional layer and number of neurons per fully connected layer, Ovtcharov).
7. The combination of Ahmed, Ovtcharov and Gong teach, The computer-implemented method of claim 1, wherein the one or more performance metrics comprises an estimated energy consumption of the candidate neural network model directly computed using one or more look up tables or estimation functions (P. 3, Sec 2.4, Col 2, paragraph 2 – teaches memory cost layer l… scaled by... memory access energy… computed… as part of state of quantization… linear scaling with bits per layer, Ahmed).
8. The combination of Ahmed, Ovtcharov and Gong teach, The computer-implemented method of claim 1, wherein the one or more performance metrics comprises a real-world energy consumption associated with implementation of the candidate neural network model on a real- world device ( Fig 9, Sec 5.4, p. 8 – evaluated… on stripes custom accelerator… speedup and energy reduction benefits compared to 8-bits, Ahmed).
9. The combination of Ahmed, Ovtcharov and Gong teach, The computer-implemented method of claim 2, wherein outputting, by the computing system, the new neural network model comprises:
determining, by the computing system, a reward based at least in part on the one or more performance metrics (P. 4, Sec 2.3, Col 1: 2nd paragraph and Sec 2.6, Col 2 - teaches receive a reward signal that is proportional to accuracy… and benefits… computation and memory cost… shaped to prioritize accuracy, Ahmed); and
modifying, by the computing system, one or more parameters of the controller model based on the reward (P 4, Sec 2.7, Col 2 – PPO… updates the policy and value networks of the RELEQ agent, Ahmed).
10. The combination of Ahmed, Ovtcharov and Gong teach, The computer-implemented method of claim 2, wherein the controller model is configured to generate the candidate neural network model through performance of reinforcement learning (Page 4, Col 2, 2nd paragraph RELEQ agent, - PPO is an actor critic style algorithm so RELEQ agent consists of both policy and value networks, Ahmed) and wherein modifying, by the computing system, the reference neural network model to generate a new neural network model comprises: determining, by the computing system, whether to retain or discard the candidate neural network model based at least in part on the one or more performance metrics (Fig 6, page 7, paragraph 2 – “Each point on these chars is a unique combination of bitwidth that are assigned to the layers of the network. The boundary of the solutions denotes the Pareto frontier and is highlighted by a dashed line. The solution found by RELEQ is marked out using an arrow and lays on the desired section of the Pareto frontier where the accuracy loss can be recovered through fine-tuning, which demonstrates the quality of the obtained solution.”).
Ahmed does not explicitly teach, performing evolutionary mutations. However, both evolutionary mutation and reinforcement learning are well-known heuristic search and optimization techniques for exploring large design spaces and refining candidate solutions in machine learning model optimization. As recognized in the art, the two approaches are interchangeable methods for generating and refining candidate model configurations: evolutionary algorithms apply random or guided “mutations” to parameters and select high-performing variants, while RL applies policy-driven “actions” to adjust parameters and select high-reward variants.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which said subject matter pertains, wherein PHOSITA in 2018 familiar with both evolutionary algorithms and RL, would have recognized that RL could be substituted for evolutionary mutations to perform the same role – generating candidate models, evaluating them and discarding poor performers – with predictable results, namely an automated exploration of the architecture/quantization search space. This is a straightforward substitution of one known search technique with another, each performing the same abstract mathematical activity of estimating parameters of a neural network model based on feedback. (KSR Int’l Co. v. Teleflex inc.)
11. The combination of Ahmed, Ovtcharov and Gong teach, The computer-implemented method of claim 1, wherein the one or more performance metrics (Section 2.3 teaches “The underlying optimization problem is multi-objective (higher accuracy, lower compute and reduced memory); however preserving the accuracy is the primary concern”, Ahmed) comprises a scaling factor (Section 2.6 teaches “Reward Shaping formulation provides the asymmetry and puts more emphasis on maintaining the accuracy… The reward uses the same terms of State of Quantization and state of relative Accuracy” – the reward shaping function scales the optimization objective using factors derived from quantization and accuracy metrics, Ahmed) which negatively correlates to a difference in energy consumption (Section 2.4 – teaches “State of quantization is a metric to evaluate the benefit of quantization… computed using the compute cost and memory cost of each layer” and “Compute cost of layer l is the number of Multiple-Accumulate Operations (MAcc)… memory cost is the number of weights scaled by the ratio of memory access energy to computation energy” – thus explicitly teaching computing cost and memory cost corresponding to energy consumption differences between model with different quantization levels and further joining the metric to energy consumption characteristics of the model, Ahmed) between the candidate neural network model (Section 2.6 – teaches the reward formulation for ReLeQ aims to preserve accuracy and minimize bitwidth of the layers simultaneously – thus disclosing lower bitwidth reduces compute and memory cost (energy), but may reduce accuracy – creating the negative correlation described in the claim– thus teaching compute cost and memory cost corresponds to the energy consumption difference between models with different quantization levels., Ahmed) and the reference neural network model (Section 2.3 – teaches “The state of Relative accuracy is defined as the ratio of the current accuracy… to accuracy of the network when it runes with full precision” – thus disclosing full-precision network functions as the reference neural network model for performance comparison, Ahmed).
12. The combination of Ahmed, Ovtcharov and Gong teach, The computer-implemented method of claim 1, wherein the reference neural network model comprises a plurality of layers (Fig 4, Ahmed), and wherein the method further comprises:
evaluating, by the computing system, an energy cost associated with each of two or more of the plurality of layers (Page 3, Col 1, paragraph 2 - the agent, consequently, receives a reward signal that is proportional to its accuracy after quantization and its benefits in terms of computation and memory cost; page 3, col 2, paragraph 2 “state of quantization is a metric to evaluate the benefit of quantization for the network and it is calculated using the compute cost and memory cost of each layer”); modifying, by the computing system, each of the two or more of the plurality of layers in an order determined by a descending order of the energy costs associated with each of the two or more of the plurality of layers ( Page 11, - teaches, “techniques for selecting quantization levels” … HAQ utilizes accuracy in the reward formulation and then adjusts the RL solution through an approach that sequentially decreases the layers bitwidth to stay within a predefined resource budget while; agent sequentially steps though layers, per-layer energy cost metrics are computed and could be used for ordering, Ahmed)
13. The combination of Ahmed, Ovtcharov and Gong teach, The computer-implemented method of claim 12, wherein modifying, by the computing system, each of the two or more of the plurality of layers comprises:
selecting, by the computing system, a first quantization scheme for quantizing values within a first layer and a second quantization scheme for quantizing values within a second layer, wherein the first quantization scheme is different than the second quantization scheme, and wherein the first layer is associated with a first energy cost higher than a second energy cost associated with the second layer (Table 2, Page 7: col 1, last paragraph, Ahmed).
Claim 14 is similar to claim 1 hence rejected similarly.
Claim 3 is similar to claim 15 hence rejected similarly.
Claim 16 is similar to claim 7 hence rejected similarly.
Claim 17 is similar to claim 8 hence rejected similarly.
Claim 18 is similar to claim 9 hence rejected similarly.
Claim 19 is similar to the combination of claims 5 and 6 hence rejected similarly.
Claim 20 is similar to claim 1 hence rejected similarly.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AMRESH SINGH whose telephone number is (571)270-3560. The examiner can normally be reached Monday-Friday 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ann J. Lo can be reached at (571) 272-9767. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AMRESH SINGH/Primary Examiner, Art Unit 2159
Prosecution Insights