DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is responsive to the original application filed on 11/27/2023. Acknowledgment is made with respect to a claim of priority to Korean Application KR10-2022-0184863 filed on 12/26/2022 and PCT Application PCT/KKR2023/018220 filed on 11/14/2023.
Claim Objections
Claims 1-20 are objected to because of the following informalities:
Claims 1 and 12 recite the limitations “acquire a neural network model, test data for the neural network model, and information on required performance condition, quantize at least one layer among a plurality of layers comprised in the neural network model and acquire a first quantized neural network model” (emphasis added) which should read as “acquire a neural network model, test data for the neural network model, and information on a required performance condition, quantize at least one layer among a plurality of layers comprised in the neural network model [[and]] to acquire a first quantized neural network model” (emphasis added) for better grammatical clarity. Dependent claims 2-11 and 13-20 depend on objected claims 1 and 12, and are also objected to by virtue of this dependency. Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The analysis of the claims will follow the 2019 Revised Patent Subject Matter Eligibility Guidance, 84 Fed. Reg. 50 (“2019 PEG”).
When considering subject matter eligibility under 35 U.S.C. 101, it must be determined whether the claim is directed to one of the four statutory categories of invention, i.e., process, machine, manufacture, or composition of matter (Step 1). If the claim does fall within one of the statutory categories, the second step in the analysis is to determine whether the claim is directed to a judicial exception (Step 2A). The Step 2A analysis is broken into two prongs. In the first prong (Step 2A, Prong 1), it is determined whether or not the claims recite a judicial exception (e.g., mathematical concepts, mental processes, certain methods of organizing human activity). If it is determined in Step 2A, Prong 1 that the claims recite a judicial exception, the analysis proceeds to the second prong (Step 2A, Prong 2), where it is determined whether or not the claims integrate the judicial exception into a practical application. If it is determined at step 2A, Prong 2 that the claims do not integrate the judicial exception into a practical application, the analysis proceeds to determining whether the claim is a patent-eligible application of the exception (Step 2B). If an abstract idea is present in the claim, any element or combination of elements in the claim must be sufficient to ensure that the claim integrates the judicial exception into a practical application, or else amounts to significantly more than the abstract idea itself.
Claim 1
Step 1: The claim recites n electronic device; therefore, it is directed to the statutory category of a machine.
Step 2A Prong 1: The claim recites, inter alia:
quantize at least one layer among a plurality of layers comprised in the neural network model and acquire a first quantized neural network model: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of quantizing a layer in a neural network, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. For example, one can practically and mentally quantize layers of a neural network by adjusting weight values.
based on the result data and the profile information, based on the first quantized neural network model satisfying the required performance condition, add the first quantized neural network model to available quantized neural network model candidates: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of adding a neural network to a list of candidate neural networks, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. For example, one can practically and mentally determine to update a model based on the occurrence of another model being generated.
Step 2A Prong 2: The claim does not recite any additional limitations which integrate the abstract idea into a practical application. Specifically, the additional elements consist of “a communication interface; a memory; and at least one processor, wherein the at least one processor is configured to”, “acquire a neural network model, test data for the neural network model, and information on required performance condition”, “control the communication interface to transmit the first quantized neural network model and the test data to a target apparatus”, and “receive, from the target apparatus, result data acquired from the first quantized neural network model with the test data as an input, and profile information of the target apparatus related to the first quantized neural network model through the communication interface”.
The additional elements of “a communication interface; a memory; and at least one processor, wherein the at least one processor is configured to” amount to generic computer components used as a tool to perform an existing process. Thus, the additional elements amount to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
The additional elements “acquire a neural network model, test data for the neural network model, and information on required performance condition”, “control the communication interface to transmit the first quantized neural network model and the test data to a target apparatus”, and “receive, from the target apparatus, result data acquired from the first quantized neural network model with the test data as an input, and profile information of the target apparatus related to the first quantized neural network model through the communication interface” are insignificant extra-solution activities required for any uses of the abstract ideas (see MPEP § 2106.05(g)).
Thus, even when viewed individually and as an ordered combination, these additional elements do not integrate the abstract idea into a practical application and the claim is thus directed to the abstract idea.
Step 2B: Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea.
The additional elements of “a communication interface; a memory; and at least one processor, wherein the at least one processor is configured to” amount to generic computer components used as a tool to perform an existing process. Thus, the additional elements amount to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
The additional elements “acquire a neural network model, test data for the neural network model, and information on required performance condition”, “control the communication interface to transmit the first quantized neural network model and the test data to a target apparatus”, and “receive, from the target apparatus, result data acquired from the first quantized neural network model with the test data as an input, and profile information of the target apparatus related to the first quantized neural network model through the communication interface” are insignificant extra-solution activities required for any uses of the abstract ideas (see MPEP § 2106.05(g)), and are well-understood, routine, conventional activities (see MPEP § 2106.05(d)(II)(i); “Receiving or transmitting data over a network” and “Storing and retrieving information in memory”).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 2
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites, inter alia:
quantize the at least one layer among the plurality of layers comprised in the neural network model with first bit-precision and acquire the first quantized neural network model: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of quantizing a layer with a bit precision or changing weights in layers, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
based on a number of neural network models comprised in the quantized neural network model candidates being smaller than a predetermined number and the first quantized neural network model not satisfying the required performance condition, quantize the neural network model with second bit-precision and acquire a second quantized neural network model: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of determining when to update a model, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
when the second user uses the updated master model instead of the custom model, the second training data stored in the second storage is deleted: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of quantizing a layer with a bit precision or changing weights in layers, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
Step 2A Prong 2, Step 2B: The claim does not recite any additional elements that are sufficient to integrate the judicial exceptions into a practical application or amount to significantly more than the judicial exception. As such, the claim is ineligible.
Claim 3
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites, inter alia:
based on the result data acquired from the second quantized neural network model and the profile information of the target apparatus related to the second quantized neural network model, based on the second quantized neural network model satisfying the required performance condition, add the second quantized neural network model to the available quantized neural network model candidates: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of adding a neural network to a list of network candidates, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
Step 2A Prong 2, Step 2B: The additional elements “transmit the second quantized neural network model to the target apparatus, receive, from the target apparatus, result data acquired from the second quantized neural network model, and profile information of the target apparatus related to the second quantized neural network model” are insignificant extra-solution activities required for any uses of the abstract ideas (see MPEP § 2106.05(g)), and are well-understood, routine, conventional activities (see MPEP § 2106.05(d)(II)(i); “Storing and retrieving information in memory” and “Receiving or transmitting data over a network”). Taken alone or in combination, the additional elements of the claim do not provide an inventive concept, integrate the abstract ideas into a practical application, or provide significantly more than the abstract ideas of the claim and thus the claim is subject-matter ineligible.
Claim 4
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites, inter alia:
vary a layer to be quantized among the plurality of layers comprised in the neural network model: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of varying a layer to be quantized in a NN, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
based on a number of neural network models comprised in the available quantized neural network model candidates being smaller than a predetermined number, acquire scores indicating suitability for the required performance condition for each model excluding models comprised in the available quantized neural network model candidates among the plurality of first quantized neural network models: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mathematical concept of acquiring or determining scores indicating a performance condition, which is performed through mathematical processes as evidenced by equations 1-5 of the originally filed specification.
Step 2A Prong 2, Step 2B: The additional elements “and acquire a plurality of first quantized neural network models comprising the first quantized neural network model” are insignificant extra-solution activities required for any uses of the abstract ideas (see MPEP § 2106.05(g)), and are well-understood, routine, conventional activities (see MPEP § 2106.05(d)(II)(i); “Storing and retrieving information in memory” and “Receiving or transmitting data over a network”). Taken alone or in combination, the additional elements of the claim do not provide an inventive concept, integrate the abstract ideas into a practical application, or provide significantly more than the abstract ideas of the claim and thus the claim is subject-matter ineligible.
Claim 5
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites, inter alia:
identify the neural network models in the predetermined number in an order of having higher scores among the models excluding the models comprised in the available quantized neural network model candidates: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of identifying neural networks based on having particular scores, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
Step 2A Prong 2, Step 2B: The claim does not recite any additional elements that are sufficient to integrate the judicial exceptions into a practical application or amount to significantly more than the judicial exception. As such, the claim is ineligible.
Claim 6
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites, inter alia:
quantize the neural network model with second bit-precision based on a quantization method by which the neural network models in the predetermined number were quantized and acquire a second quantized neural network model: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of quantizing a model with a bit precision or changing weights in layers, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
Step 2A Prong 2, Step 2B: The claim does not recite any additional elements that are sufficient to integrate the judicial exceptions into a practical application or amount to significantly more than the judicial exception. As such, the claim is ineligible.
Claim 7
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2, Step 2B: The additional element of “wherein the predetermined number is determined by at least one of the number of layers comprised in the neural network model or performance of the electronic apparatus” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h). Taken alone or in combination, the additional elements of the claim do not provide an inventive concept, integrate the abstract ideas into a practical application, or provide significantly more than the abstract ideas of the claim and thus the claim is subject-matter ineligible.
Claim 8
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites, inter alia:
identify neural network models satisfying a limiting condition among the models excluding the models comprised in the available quantized neural network model candidates: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of identifying models satisfying a condition, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
acquire scores indicating suitability for the required performance condition for each of the neural network models satisfying the limiting condition: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mathematical concept of acquiring or determining scores indicating a performance condition, which is performed through mathematical processes as evidenced by equations 1-5 of the originally filed specification.
identify the neural network models in the predetermined number in an order of having higher scores among the neural network models satisfying the limiting condition: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of identifying models having particular scores satisfying a condition, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
Step 2A Prong 2, Step 2B: The claim does not recite any additional elements that are sufficient to integrate the judicial exceptions into a practical application or amount to significantly more than the judicial exception. As such, the claim is ineligible.
Claim 9
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites, inter alia:
based on a number of neural network models comprised in the available quantized neural network model candidates being greater than or equal to a predetermined number, acquire scores indicating suitability for the required performance condition for each of the neural network models comprised in the available quantized neural network model candidates: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mathematical concept of acquiring or determining scores indicating a performance condition, which is performed through mathematical processes as evidenced by equations 1-5 of the originally filed specification.
identify a neural network model having highest score among the neural network models comprised in the quantized neural network model candidates: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of identifying models having particular scores, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
Step 2A Prong 2, Step 2B: The additional element “control the communication interface to transmit identified neural network model to the target apparatus” is an insignificant extra-solution activity required for any uses of the abstract ideas (see MPEP § 2106.05(g)), and is a well-understood, routine, conventional activity (see MPEP § 2106.05(d)(II)(i); “Receiving or transmitting data over a network”). Taken alone or in combination, the additional elements of the claim do not provide an inventive concept, integrate the abstract ideas into a practical application, or provide significantly more than the abstract ideas of the claim and thus the claim is subject-matter ineligible.
Claim 10
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites, inter alia:
based on a quantization error regarding the first quantized neural network model and the profile information of the target apparatus related to the first quantized neural network model being stored in the memory, add the first quantized neural network model in the available quantized neural network model candidates by using the quantization error and the profile information stored in the memory: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of adding a model to a list of candidate models, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
Step 2A Prong 2, Step 2B: The claim does not recite any additional elements that are sufficient to integrate the judicial exceptions into a practical application or amount to significantly more than the judicial exception. As such, the claim is ineligible.
Claim 11
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites the abstract ideas of the preceding claims from which it depends.
Step 2A Prong 2, Step 2B: The additional element of “at least one of inference latency incurred while the target apparatus was performing inference of the first quantized neural network model, memory use amount of the target apparatus, or power consumption of the target apparatus” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h). Taken alone or in combination, the additional elements of the claim do not provide an inventive concept, integrate the abstract ideas into a practical application, or provide significantly more than the abstract ideas of the claim and thus the claim is subject-matter ineligible.
Claims 12-20
Claims 12-20 recite a method (step 1: a process) to perform the steps of claims 1-9, respectively, without any additional elements that integrate the abstract ideas into a practical application or provide significantly more than the abstract idea by itself, and are thus rejected for the same reasons set forth in the rejection of claims 1-9, respectively.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 9-14, and 20 are rejected under 35 USC § 103 as being obvious over Yang et al. (Yang et al., “NetAdapt: Platform-Aware Neural Network Adaptation for Mobile Applications”, Sep. 28, 2018, arXiv:1804.03230v2, pp. 1-16, hereinafter “Yang”) in view of Wang et al., (Wang et al., “HAQ:Hardware-Aware Automated Quantization with Mixed Precision”, Apr. 6, 2019, arXiv:1811.08886v3, pp. 1-10, hereinafter “Wang”) and Rezk et al. (Rezk et al., “MOHAQ: Multi-Objective Hardware-Aware Quantization of Recurrent Neural Networks”, Jan 20, 2022, arXiv:2108.01192v3, pp. 1-14, hereinafter “Rezk”).
Regarding claim 1, Yang discloses [a]n electronic apparatus comprising: a communication interface; a memory; and at least one processor, wherein the at least one processor is configured to: (§4; the section discloses the hardware deployment of the experiments of Yang which are inherently performed using an electronic apparatus with a communication interface, memory, and a processor)
acquire a neural network model, test data for the neural network model, and information on required performance condition, (Algorithm 1; the algorithm discloses a “Pretrained Network: Net0” which is acquired as an input neural network model as well as “Resource Budget: Bud” which is information on a required performance condition; and Page 9, §4.1; “We preserve ten thousand images from the training set, ten images per class, as the holdout set. The new training set without the holdout images is used to perform short-term fine-tuning, and the holdout set is used to pick the highest accuracy network out of the simplified networks at each iteration”, which discloses the received test data or holdout set for the NN model)
control the communication interface to transmit the [[first quantized]] neural network model and the test data to a target apparatus, (Figure 1 Caption; “At each iteration, NetAdapt generates many network proposals and measures the proposals on the target platform. The measurements are used to guide NetAdapt to generate the next set of network proposals at the next iteration”, which discloses transmitting a quantized NN or network proposal to a target apparatus or platform; and Algorithm 1, Line 5; and §4,1; “For experiments on mobile CPUs, the latency is measured on a single large core of Google Pixel 1 phone”)
receive, from the target apparatus, result data acquired from the [[first quantized]] neural network model with the test data as an input, and profile information of the target apparatus related to the first quantized neural network model through the communication interface, and (§4.1; “We preserve ten thousand images from the training set, ten images per class, as the holdout set. The new training set without the holdout images is used to perform short-term fine-tuning, and the holdout set is used to pick the highest accuracy network out of the simplified networks at each iteration”; and Algorithm 1; “TakeEmpiricalMeasurement(Neti)… PickHighestAccuracy(NetSimp:,ResSimp:)”; and §3.1; “The resource can be latency, energy, memory footprint, etc., or a combination of these metrics”, wherein the profile information is latency, energy, or memory footprint).
Yang fails to explicitly disclose but Wang discloses quantize at least one layer among a plurality of layers comprised in the neural network model and acquire a first quantized neural network model, … first quantized neural network model (§3.4; “We linearly quantize the weights and activations of each layer using the action ak given by our agent, as linearly quantized model only needs fixed point arithmetic unit which is more efficient to implement on the hardware. Specifically, for each weight value w in the kth layer, we first truncate it into the range of [−c,c], and we then quantize it linearly into ak bits”, which discloses mapping each weight value w in the k-th layer to a quantized value to result in a multi-layer model with quantized weights; and §3.4, Equation 4; and §3.2; “After our RL agent gives actions {ak} to all layers, we measure the amount of resources that will be used by the quantized model”).
Yang and Wang are analogous art because both are concerned with neural network optimization. Before the effective filing date of the claimed invention, it would have been obvious to one skilled in neural network computing to combine the layer-wise neural network quantization of Wang and the apparatus of Yang to yield to the predictable result of quantize at least one layer among a plurality of layers comprised in the neural network model and acquire a first quantized neural network model, control the communication interface to transmit the first quantized neural network model and the test data to a target apparatus, receive, from the target apparatus, result data acquired from the first quantized neural network model with the test data as an input, and profile information of the target apparatus related to the first quantized neural network model through the communication interface. The motivation for doing so would be to determine an optimal neural network quantization policy (Wang; Abstract).
Yang fails to explicitly disclose but Rezk discloses based on the result data and the profile information, based on the first quantized neural network model satisfying the required performance condition, add the first quantized neural network model to available quantized neural network model candidates (Figure 1 Caption; “The beacon-based search is the search method that requires retraining of the model using some candidate solutions variables. The output of the search is a Pareto set of optimal solutions”, wherein an optimal solution is the determined quantized neural network that is added to a Pareto Front of optimal solutions or available quantized NN candidates).
Yang, Wang, and Rezk are analogous art because all are concerned with neural network optimization. Before the effective filing date of the claimed invention, it would have been obvious to one skilled in neural network computing to combine the optimal neural network solutions of Rezk with the apparatus of Yang and the layer-wise quantization of Wang to yield to the predictable result of based on the result data and the profile information, based on the first quantized neural network model satisfying the required performance condition, add the first quantized neural network model to available quantized neural network model candidates. The motivation for doing so would be to provide for a Pareto set of optimal quantized neural network solutions (Rezk; Figure 1 Caption).
Regarding claim 12, it is a method claim corresponding to the steps of claim 1, and is rejected for the same reasons as claim 1.
Regarding claims 2 and 13, the rejection of claims 1 and 12 are incorporated and Yang fails to explicitly disclose but Wang discloses wherein the at least one processor is further configured to: quantize the at least one layer among the plurality of layers comprised in the neural network model with first bit-precision and acquire the first quantized neural network model, and (§3.2; and Equation 3; and Tables 3-5)
based on a number of neural network models comprised in the quantized neural network model candidates being smaller than a predetermined number and the first quantized neural network model not satisfying the required performance condition, quantize the neural network model with second bit-precision and acquire a second quantized neural network model (§3.2; and Equation 3; and Tables 3-5).
The motivation to combine Yang and Wang is the same as discussed above with respect to claim 1.
Regarding claims 3 and 14, the rejection of claims 1, 2, 12, and 13 are incorporated and Yang discloses transmit the second quantized neural network model to the target apparatus (Algorithm 1, Line 5; and §4,1; “For experiments on mobile CPUs, the latency is measured on a single large core of Google Pixel 1 phone”)
receive, from the target apparatus, result data acquired from the second quantized neural network model, and profile information of the target apparatus related to the second quantized neural network model (Algorithm 1, Line 5; and §4,1; “We preserve ten thousand images from the training set, ten images per class, as the holdout set. The new training set without the holdout images is used to perform short-term fine-tuning, and the holdout set is used to pick the highest accuracy network out of the simplified networks at each iteration”)
based on the result data acquired from the second quantized neural network model and the profile information of the target apparatus related to the second quantized neural network model, based on the second quantized neural network model satisfying the required performance condition, add the second quantized neural network model to the available quantized neural network model candidates (§3.1; “It outputs the final adapted network and can also generate a sequence of simplified networks… to provide the efficient frontier of accuracy and resource consumption trade-offs”; and Algorithm 1, Line 9).
Regarding claims 9 and 20, the rejection of claims 1 and 12 are incorporated and Yang further discloses control the communication interface to transmit identified neural network model to the target apparatus (Figure 1 Caption; “At each iteration, NetAdapt generates many network proposals and measures the proposals on the target platform. The measurements are used to guide NetAdapt to generate the next set of network proposals at the next iteration”, which discloses transmitting a quantized NN or network proposal to a target apparatus or platform; and Algorithm 1, Line 5; and §4,1; “For experiments on mobile CPUs, the latency is measured on a single large core of Google Pixel 1 phone”).
Yang fails to explicitly disclose but Wang discloses based on a number of neural network models comprised in the available quantized neural network model candidates being greater than or equal to a predetermined number, acquire scores indicating suitability for the required performance condition for each of the neural network models comprised in the available quantized neural network model candidates (§3.5; the reward function outputs the highest scoring quantization configuration from all the explored candidates; and §3.6)
identify a neural network model having highest score among the neural network models comprised in the quantized neural network model candidates, and (§3.5; and §3.6).
The motivation to combine Yang and Wang is the same as discussed above with respect to claim 1.
Regarding claim 10, the rejection of claim 1 is incorporated and Yang further discloses based on a quantization error regarding the first quantized neural network model and the profile information of the target apparatus related to the first quantized neural network model being stored in the memory (§3.4; “We solve this problem by building layer-wise look-up tables with pre-measured resource consumption of each layer. When executing the algorithm, we look up the table of each layer, and sum up the layer-wise measurements to estimate the network-wise resource consumption”).
Yang fails to explicitly disclose but Wang discloses the quantization error (§3.4; “In this paper, we choose the value of c by finding the optimal value x that minimizes the KL-divergence between the original weight distribution Wk and the quantized weight distribution quantize”).
The motivation to combine Yang and Wang is the same as discussed above with respect to claim 1.
Yang fails to explicitly disclose but Rezk discloses add the first quantized neural network model in the available quantized neural network model candidates by using [[the quantization error and]] the profile information stored in the memory (Figure 1 Caption; “The beacon-based search is the search method that requires retraining of the model using some candidate solutions variables. The output of the search is a Pareto set of optimal solutions”, wherein an optimal solution is the determined quantized neural network that is added to a Pareto Front of optimal solutions or available quantized NN candidates).
The motivation to combine Yang, Wang, and Rezk is the same as discussed above with respect to claim 1.
Regarding claim 11, the rejection of claim 1 is incorporated and Yang further discloses wherein the profile information comprises:
at least one of inference latency incurred while the target apparatus was performing inference of the first quantized neural network model, memory use amount of the target apparatus, or power consumption of the target apparatus (§3.1; “The resource can be latency, energy, memory footprint, etc., or a combination of these metrics”).
Claims 4-8 and 15-19 are rejected under 35 USC § 103 as being obvious over Yang in view of Wang and Rezk and further in view of Dong et al. (Dong et al., “HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precision”, Apr. 29, 2019, arXiv:1905.03696v1, pp. 1-12, hereinafter “Dong”).
Regarding claims 4 and 15, the rejection of claims 1 and 12 are incorporated and Yang discloses wherein the at least one processor is further configured to: vary a layer to be quantized among the plurality of layers comprised in the neural network model and acquire a plurality of first quantized neural network models comprising the first quantized neural network model, and (§3.2; “In each iteration, the previous three steps (highlighted in bold) are applied on each of the CONV or FC layers individually3. As a result, NetAdapt generates K(i.e., the number of CONV and FC layers) network proposals in one iteration, each of which has a single layer modified from the previous iteration. The network proposal with the highest accuracy is carried over to the next iteration (the Pick Highest Accuracy block)”; and Algorithm 1, Line 5).
Yang fails to explicitly disclose but Dong discloses based on a number of neural network models comprised in the available quantized neural network model candidates being smaller than a predetermined number, acquire scores indicating suitability for the required performance condition for each model excluding models comprised in the available quantized neural network model candidates among the plurality of first quantized neural network models (Algorithm 2, “for i = 1,2,...,b do // Compute Quantization Precision Si=λi/ni”; and §III.B, Equation 5; “Therefore, as a compromise, we weight the spectrum with block’s memory footprint and use the following metric for sorting the blocks: Si = λi/ni, (5) where λi is the top eigenvalue of Hi”).
Yang, Wang, Rezk, and Dong are analogous art because all are concerned with neural network optimization. Before the effective filing date of the claimed invention, it would have been obvious to one skilled in neural network computing to combine the suitability scores of Dong with the apparatus of Yang and the layer-wise quantization of Wang and Rezk to yield to the predictable result of based on a number of neural network models comprised in the available quantized neural network model candidates being smaller than a predetermined number, acquire scores indicating suitability for the required performance condition for each model excluding models comprised in the available quantized neural network model candidates among the plurality of first quantized neural network models. The motivation for doing so would be to use the Hessian spectrum of each block to sort the different blocks and perform less aggressive quantization to layers with large spectrum (Dong; §III.B).
Regarding claims 5 and 16, the rejection of claims 1, 4, 12, and 15 are incorporated and Yang fails to explicitly disclose but Dong discloses wherein the at least one processor is further configured to: identify the neural network models in the predetermined number in an order of having higher scores among the models excluding the models comprised in the available quantized neural network model candidates (Algorithm 2, “for i = 1,2,...,b do // Compute Quantization Precision Si=λi/ni”; and §III.B, Equation 5; “Therefore, as a compromise, we weight the spectrum with block’s memory footprint and use the following metric for sorting the blocks: Si = λi/ni, (5) where λi is the top eigenvalue of Hi. Based on this sorting, layers that have large number of parameters and have small eigenvalue would be quantized to lower bits, and vice versa.”).
The motivation to combine Yang, Wang, Rezk, and Dong is the same as discussed above with respect to claim 4.
Regarding claims 6 and 17, the rejection of claims 1, 4, 5, 12, 15, and 16 are incorporated and Yang fails to explicitly disclose but Dong discloses quantize the neural network model with second bit-precision based on a quantization method by which the neural network models in the predetermined number were quantized and acquire a second quantized neural network model (Algorithm 2, “for i = 1,2,...,b do // Compute Quantization Precision Si=λi/ni”; and §III.B, Equation 6; “We sort different blocks for fine-tuning based on the following metric: Ωi =λi Q(Wi)−Wi 2 2, (6) where i refers to ith block, λi is the Hessian eigenvalue, and Q(Wi)−Wi 2 is the L2 norm of quantization perturbation”).
The motivation to combine Yang, Wang, Rezk, and Dong is the same as discussed above with respect to claim 4.
Regarding claims 7 and 18, the rejection of claims 1, 4, 12, and 15 are incorporated and Yang discloses wherein the predetermined number is determined by at least one of the number of layers comprised in the neural network model or performance of the electronic apparatus (§3.2; “In each iteration, the previous three steps (highlighted in bold) are applied on each of the CONV or FC layers individually3. As a result, NetAdapt generates K(i.e., the number of CONV and FC layers) network proposals in one iteration, each of which has a single layer modified from the previous iteration.”).
Regarding claims 8 and 19, the rejection of claims 1, 4, 12, and 15 are incorporated and Yang discloses identify neural network models satisfying a limiting condition among the models excluding the models comprised in the available quantized neural network model candidates (Algorithm 1, Lines 5 and 6).
Yang fails to explicitly disclose but Dong discloses acquire scores indicating suitability for the required performance condition for each of the neural network models satisfying the limiting condition (Algorithm 2; and §III.B)
identify the neural network models in the predetermined number in an order of having higher scores among the neural network models satisfying the limiting condition (Algorithm 2; and §III.B).
The motivation to combine Yang, Wang, Rezk, and Dong is the same as discussed above with respect to claim 4.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Yao et al., “HAWQ-V3: Dyadic Neural Network Quantization”, Jun. 23, 2021, arXiv:2011.10680v3, pp. 1-17.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Brent Hoover whose telephone number is (303)297-4403. The examiner can normally be reached Monday - Friday 9-5 MST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abdullah Kawsar can be reached at 571-270-3169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BRENT JOHNSTON HOOVER/Primary Examiner, Art Unit 2127
Prosecution Insights