Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections – 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 8 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. As per claim 8, the claim limitation recites “recording medium”. However, the usage of the phrase “recording medium” is broad enough to include both “non-transitory” and “transitory” media. The specification further explicitly does not limit the utilization of a non-transitory recording medium (See specification, paragraph 0048-49). Also, extrinsic evidence suggests that recording medium covers a signal per se. Therefore, when the broadest reasonable interpretation of a claim covers a signal per se, the claim must be rejected under 35 U.S.C. 101 as covering non-statutory subject matter. See In re Nuijten, 500 F.3d 1346, 1356-57 (Fed. Cir. 2007) (transitory embodiments are not directed to statutory subject matter). Therefore, claim 8 is non-statutory. A suggestion is made to the Applicant to amend the claim to recite non-transitory recording medium.
Claims 1-8 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claims 1-7 are method claims. Claim 8 are machine/system/product claims. Therefore, claims 1-8 are directed to either a process, machine, manufacture or composition of matter.
With respect to claim 1:
Step 2A – Prong 1:
…
…
selecting one DNN model to perform the inference among a plurality of DNN models; (mental process – a person can manually select a DNN model with the assistance of a pen/paper.)
…
… and the selecting of the DNN model is performed by an agent through an action based on the state of energy stored in the storage. (mental process – a person can manually select a DNN model based on the state of the energy stored in the storage.)
Step 2A – Prong 2: This judicial exception is not integrated into a practical application.
An inference method using a DNN model in an energy harvesting system, the method comprising: performing energy harvesting and storing energy in a storage; (Adding insignificant extra-solution activity to the judicial exception - see MPEP 2106.05(g)). (mere instructions to apply the exception using a generic computer component – storage applies exception.)
receiving a request for inference using data collected during the period of energy harvesting; (Adding insignificant extra-solution activity to the judicial exception - see MPEP 2106.05(g)).
…
performing the inference through the selected DNN model; (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: High level recitation of training the DNN to make an inference.);
and performing energy harvesting again when the inference is completed and storing energy in the storage, wherein the plurality of DNN models are DNN models with different inference accuracy and energy consumption required for inference, … (Adding insignificant extra-solution activity to the judicial exception - see MPEP 2106.05(g)).
Step 2B: The claim does not include additional elements considered individually and in combination that are sufficient to amount to significantly more than the judicial exception.
An inference method using a DNN model in an energy harvesting system, the method comprising: performing energy harvesting and storing energy in a storage; (MPEP 2106.05(d)(II) indicate that merely “Storing and retrieving information in memory” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim – the energy is merely harvested and stored in storage). Thereby, a conclusion that the claimed distribute step is well-understood, routine, conventional activity is supported under Berkheimer.)
receiving a request for inference using data collected during the period of energy harvesting; (MPEP 2106.05(d)(II) indicate that merely “Receiving or transmitting data over a network, e.g., using the Internet to gather data” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim – the request is merely received). Thereby, a conclusion that the claimed distribute step is well-understood, routine, conventional activity is supported under Berkheimer.)
…
performing the inference through the selected DNN model; (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: High level recitation of training the DNN to make an inference.);
and performing energy harvesting again when the inference is completed and storing energy in the storage, wherein the plurality of DNN models are DNN models with different inference accuracy and energy consumption required for inference, … (MPEP 2106.05(d)(II) indicate that merely “Receiving or transmitting data over a network, e.g., using the Internet to gather data” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim – the energy is merely received by the storage). Thereby, a conclusion that the claimed distribute step is well-understood, routine, conventional activity is supported under Berkheimer.)
With respect to claim 2:
Step 2A – Prong 1:
The method of claim 1, wherein the performing of the inference through the selected DNN model includes: refraining from the inference if it is determined that the amount of energy required to complete the inference through the selected DNN model is greater than the amount of energy stored in the storage; (mental process – a person can manually refrain from the inference if it is determined that the amount of energy required to complete the inference through the selected DNN model is greater than the amount of energy stored in the storage with the assistance of a pen/paper.)
and performing energy harvesting again and storing energy in the storage if the inference is not performed. (mental process – a person can manually make the decision to perform energy harvesting again if they recognize that the inference is not performed with the assistance of a pen/paper.)
With respect to claim 3:
Step 2A – Prong 1:
The method of claim 2, wherein the state of energy stored in the storage includes features corresponding to: (a) information on the amount of energy stored in the storage, (mental process – a person can recognize that the state of energy stored includes features corresponding to information on the amount of energy stored.)
(b) information on the amount of energy stored during the interval up to a previous time step, (mental process – a person can recognize that the state of energy stored includes features corresponding to information on the amount of energy stored during the interval up to a previous time step.)
(c) information on the average amount of energy stored during a period comprising last 10 time steps, (mental process – a person can recognize that the state of energy stored includes features corresponding to information on the amount of energy stored during a period comprising last 10 time steps.)
and (d) information obtained by subtracting the average value of the quantity of energy stored during a period comprising three oldest time steps from the average value of the quantity of energy stored during a period comprising three most recent time steps within a period comprising last 10 time steps, wherein the agent performs the action based on the features (a) to (d). (mental process – a person can recognize that the state of energy stored includes features corresponding to information obtained by subtracting the average value of the quantity of energy stored during a period comprising three oldest time steps from the average value of the quantity of energy stored during a period comprising three most recent time steps.)
With respect to claim 4:
Step 2A – Prong 1:
The method of claim 2, wherein the policy is updated so that an accumulated sum of rewards received by the agent has the maximum value, wherein the agent receives: a ratio by which a first DNN model selected at a current time step among the plurality of DNN models is improved compared to a second DNN model with the highest error rate among the plurality of DNN models as a positive value, (mental process – a person can recognize that the policy is updated so that an accumulated sum of rewards received by the agent has the maximum value, wherein the agent receives a ratio by which a first DNN model selected at a current time step among the plurality of DNN models is improved compared to a second DNN model with the highest error rate among the plurality of DNN models as a positive value.)
and a preconfigured hyperparameter value as a negative reward when the performing of the inference is not performed. (mental process – a person can recognize that the policy is updated so that an accumulated sum of rewards received by the agent has the maximum value, wherein the agent receives a preconfigured hyperparameter value as a negative reward when the performing of the inference is not performed.)
With respect to claim 5:
Step 2A – Prong 1:
The method of claim 4, wherein the positive reward is calculated using an equation below: IRER(m_t dot m_min) = abs(ER(m_t) – ER(m_min) / ER(m_min)) wherein Error Rate(m_t) represents the error rate of a DNN model selected at time step t, and Error Rate(m_min) represents the error rate of a DNN model showing the highest error rate among the plurality of DNN models. (Mathematical concept – The claim recites the concept of a positive reward and recites the equation to calculate the positive reward.)
With respect to claim 6:
Step 2A – Prong 1:
The method of claim 5, wherein an accumulated sum of rewards received by the agent is calculated by an equation below: sum_{t}^{N} Reward(m_t) = sum^{N_1} IRER(m_t) + sum^{N_2} p = N – (1-p)N2 – 1/e sum_{i}^{N_1} Error(m_i) wherein N represents the number of inference requests within one episode, N1 represents the number of inference successes, N2 represents the number of inference failures, p represents the preconfigured hyperparameter value, and e represents the error rate of a DNN model with the highest error rate among the plurality of DNN models. (Mathematical concept – The claim recites the concept of the sum of rewards expressed both as a function of the positive reward and the hyperparameter values and also as a function of the inference failures and the error rate.)
With respect to claim 7:
Step 2A – Prong 1:
The method of claim 4, wherein evaluation of the policy which has been updated uses a metric shown in an equation below: Metric = failrate + successrate * Error(m), Error(m) = sum_{t}^{N_1}Error(m) / N_1 wherein failrate represents the inference failure rate, successrate represents the inference success rate, N1 represents the number of inference successes, and Error(m) represents the error rate of a selected DNN model. (Mathematical concept – The claim recites the concept of updating the policy using a metric that is the function of the fail rate, success rate, and the error rate.)
With respect to claim 8:
Step 2A – Prong 1:
… for executing a method of claim 1 in a computer. (mental process from claim 1)
Step 2A – Prong 2: This judicial exception is not integrated into a practical application.
A recording medium readable by a digital processing device, in which a program of commands executed by the digital processing device to provide inference using a DNN model in an energy harvesting system is implemented, recording a program … (mere instructions to apply the exception using a generic computer component – recording medium, digital processing device applies exception.)
Claim Rejections – 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, 4, 7, 8 are rejected under 35 U.S.C. 103 as being unpatentable over Wu et al. (“Intermittent Inference with Nonuniformly Compressed Multi-Exit Neural Network for Energy Harvesting Powered Devices”) hereinafter known as Wu in view of Hoffman et al. (US11009836B2) hereinafter known as Hoffman.
Regarding independent claim 1, Wu teaches:
An inference method using a DNN model in an energy harvesting system, the method comprising: performing energy harvesting and storing energy in a storage; (Wu [Page 1, Col. 1, Paragraph 1]: “energy harvesting (EH) technology and the recent emergence of intermittent computing, which stores harvested energy in energy storage and supports an episode of program execution during each power cycle” Wu teaches an energy harvesting technology that stores energy in storage throughout power cycles.)
…
…
…
and performing energy harvesting again when the inference is completed and storing energy in the storage, wherein the plurality of DNN models are DNN models with different inference accuracy and energy consumption required for inference, and the selecting of the DNN model is performed by an agent through an action based on the state of energy stored in the storage. (Wu [Page 2, Col. 1, Paragraph 5]: “when the power is not sufficient to finish the entire forward-pass, the system is forced to pause during the inference process and wait until enough energy is harvested” Wu teaches that energy harvesting is done during and after the inference. Wu [Page 4, Col. 1, Paragraph 5]: “R_{acc} aims to maximize the average accuracy of all events under the given power trace and event distribution” More specifically, Wu teaches that the reward function is maximized by minimizing the error rate, and this is used in the selection of the model, as represented by the exit.)
Wu does not explicitly teach:
receiving a request for inference using data collected during the period of energy harvesting;
selecting one DNN model to perform the inference among a plurality of DNN models;
performing the inference through the selected DNN model;
However, Hoffman teaches:
receiving a request for inference using data collected during the period of energy harvesting; (Hoffman [Col. 16, Lines 61-65]: “In step 214 of process 210, the computational performance and power consumption of the computing device 120 when executing the current application is measured for the nth computational configuration of the subset initialized in step 212” Hoffman teaches that the power consumption is measured when executing the application. This shows that the device has to have stored the power in an intermediary step. Hoffman [Col. 7, Lines 38-47]: “The server according to any of (1)-(10), wherein the processing circuitry is further configured to compare the received performance data of the device to the stored other performance data using … an artificial neural network … to determine the performance model of the device.” Hoffman teaches that while the power consumption is occurring, that there is a comparison of the performance of the data using a neural network to determine the model of the device.)
selecting one DNN model to perform the inference among a plurality of DNN models; (Hoffman [Fig. 2A]: Hoffman teaches that for all computational configurations, an estimate of the computational performance is made and that the optimal configuration/model is to be selected.)
performing the inference through the selected DNN model; (Hoffman [Col. 7, Lines 38-47]: “The server according to any of (1)-(10), wherein the processing circuitry is further configured to compare the received performance data of the device to the stored other performance data using … an artificial neural network … to determine the performance model of the device.” Hoffman teaches that while the power consumption is occurring, that there is a comparison of the performance of the data using a neural network to determine the model of the device.)
Wu and Hoffman are in the same field of endeavor as the present invention, as the references are directed to energy harvesting and selecting a neural network model based on energy/power performance, respectfully. It would have been obvious, before the effective filing date of the claimed invention, to a person of ordinary skill in the art, to combine performing energy harvesting to run interferences by deep neural networks (DNN) as taught in Wu with selecting a DNN based on energy profile as taught in Hoffman. Hoffman provides this additional functionality. As such, it would have been obvious to one of ordinary skill in the art to modify the teachings of Wu to include teachings of Hoffman because the combination would allow for the system to select the correct model that does not exceed the power limit. This has the potential benefit of choosing the most powerful and accurate inference giving model while staying until a power/energy consumption limit.
Regarding dependent claim 2, Wu teaches:
The method of claim 1, wherein the performing of the inference through the selected DNN model includes: refraining from the inference if it is determined that the amount of energy required to complete the inference through the selected DNN model is greater than the amount of energy stored in the storage; (Wu [Page 2, Col. 1, Paragraph 5]: “when the power is not sufficient to finish the entire forward-pass, the system is forced to pause during the inference process and wait until enough energy is harvested” Wu teaches that if there is not enough harvested energy to finish the forward-pass, then it refrains from inference until there is enough energy harvested.)
and performing energy harvesting again and storing energy in the storage if the inference is not performed. (Wu [Page 2, Col. 1, Paragraph 5]: “when the power is not sufficient to finish the entire forward-pass, the system is forced to pause during the inference process and wait until enough energy is harvested” Wu teaches that if there is not enough harvested energy to finish the forward-pass, then it refrains from inference until there is enough energy harvested. Wu [Page 5, Fig. 5]: Wu depicts a chart showing the number of interesting events per energy harvesting millijoule. This shows that the harvesting is done even when the inference is not performed.)
The reasons to combine are substantially similar to those of claim 1.
Regarding dependent claim 4, Wu teaches:
The method of claim 2, wherein the policy is updated so that an accumulated sum of rewards received by the agent has the maximum value, wherein the agent receives: a ratio by which a first DNN model selected at a current time step among the plurality of DNN models is improved compared to a second DNN model with the highest error rate among the plurality of DNN models as a positive value, (Wu [Page 4, Col. 2, Paragraph 5]: “When an event happens, the agent takes two steps, one for selecting the action and the other for updating the Q-table. The action for the exit is selected by finding the highest Q-value in current state” Wu teaches a Q-table that has the Q-values, with greatest to least. The ratio between the first DNN model is selected to the second one with the highest error rate is the fraction between the top and bottom Q-values in the Q-table.)
and a preconfigured hyperparameter value as a negative reward when the performing of the inference is not performed. (Wu [Page 4, Col. 2, Paragraph 7]: “To further improve the average accuracy, a second decision is made at the chosen exit for event j.” Wu teaches that another decision, which may be formatted as a negative reward, is taken when the inference is not made (also known as exit).)
The reasons to combine are substantially similar to those of claim 1.
Regarding dependent claim 5, Wu and Hoffman teach:
The method of claim 4, wherein the positive reward is calculated using an equation below: IRER(m_t dot m_min) = abs(ER(m_t) – ER(m_min) / ER(m_min)) wherein Error Rate(m_t) represents the error rate of a DNN model selected at time step t, and Error Rate(m_min) represents the error rate of a DNN model showing the highest error rate among the plurality of DNN models. (Wu [Page 4, Col. 2, Paragraph 5]: “The reward R is the accuracy of the selected exit r” Wu teaches that the reward of the agent is calculated with R. Wu [Page 4, Col. 1, Paragraph 5]: “R_{acc} aims to maximize the average accuracy of all events under the given power trace and event distribution” More specifically, Wu teaches that the reward function is maximized by minimizing the error rate.)
The reasons to combine are substantially similar to those of claim 1.
Regarding dependent claim 6, Wu and Hoffman teach:
The method of claim 5, wherein an accumulated sum of rewards received by the agent is calculated by an equation below: sum_{t}^{N} Reward(m_t) = sum^{N_1} IRER(m_t) + sum^{N_2} p = N – (1-p)N2 – 1/e sum_{i}^{N_1} Error(m_i), wherein N represents the number of inference requests within one episode, N1 represents the number of inference successes, N2 represents the number of inference failures, p represents the preconfigured hyperparameter value, and e represents the error rate of a DNN model with the highest error rate among the plurality of DNN models. (Wu [Page 4, Col. 2, Paragraph 3]: “During exploration, each agent aims to maximize the overall reward of one episode.” Wu teaches that the reward is maximized, which is consistent with the summation of the rewards functions. Wu [Page 4, Equations 11 and 12]: Wu teaches that the rewards for the agents is defined in terms of if one model is better than the other (which is defined in the positive reward function) and a scalar when the exit is taken).)
The reasons to combine are substantially similar to those of claim 1.
Regarding dependent claim 7, Wu teaches:
The method of claim 4, wherein evaluation of the policy which has been updated uses a metric shown in an equation below: Metric = failrate + successrate * Error(m), Error(m) = sum_{t}^{N_1}Error(m) / N_1, wherein failrate represents the inference failure rate, successrate represents the inference success rate, N1 represents the number of inference successes, and Error(m) represents the error rate of a selected DNN model. (Wu [Page 2, Col. 1, Paragraph 4]: “Experimental results show that the proposed techniques improve the number of correctly processed events per energy unit by 3.6x over [2], a state-of-the-art (SOTA) intermittent inference framework” Wu teaches that the evaluation of the policy uses a metric that is based on the fail rate, success rate, and the error because Wu shows experimental results that show a 3.6x improvement efficiency. This shows that the failure rate and error must have maintained while the success rate more than 3x increased.)
The reasons to combine are substantially similar to those of claim 1.
Regarding dependent claim 8, Hoffman teaches:
A recording medium readable by a digital processing device, in which a program of commands executed by the digital processing device to provide inference using a DNN model in an energy harvesting system is implemented, recording a program for executing a method of claim 1 in a computer. (Hoffman [Col. 72, Lines 39-42]: “The process data and instructions may also be stored on a storage medium disk 1004 such as a hard drive (HDD) or portable storage medium or may be stored remotely” Hoffman teaches that the data and instructions may be stored in a storage medium.)
The reasons to combine are substantially similar to those of claim 1.
Claim 3 is rejected under 35 U.S.C. 35 U.S.C. 103 as being unpatentable over Wu in view of Hoffman in view of Park et al. (KR20190023952A) hereinafter known as Park.
Regarding dependent claim 3, Wu and Hoffman teach:
The method of claim 2, wherein the state of energy stored in the storage includes features corresponding to: (a) information on the amount of energy stored in the storage, (Wu [Page 2, Col. 2, Paragraph 1]: “In this example, when Event 1 occurs, the stored energy is sufficient to support the inference to Exit 3, which is selected as the exit.” Wu teaches that a consideration is made to see if there is insufficient energy to support the inference. This shows that the information on the amount of stored energy is saved.)
However, Wu and Hoffman do not explicitly teach:
(b) information on the amount of energy stored during the interval up to a previous time step, (Park [Page 6, Paragraph 7]: “The learning circuit 252 can analyze the sensor data SD continuously provided over time. The learning circuit 252 can generate pattern data by analyzing the sensor data SD” Park teaches that the energy stored is continuously monitored, which may be at a previous time step.)
(c) information on the average amount of energy stored during a period comprising last 10 time steps, (Park [Page 6, Paragraph 7]: “The learning circuit 252 can analyze the sensor data SD continuously provided over time. The learning circuit 252 can generate pattern data by analyzing the sensor data SD” Park teaches that the energy stored is continuously monitored, which may be at the last 10 time steps.)
and (d) information obtained by subtracting the average value of the quantity of energy stored during a period comprising three oldest time steps from the average value of the quantity of energy stored during a period comprising three most recent time steps within a period comprising last 10 time steps, wherein the agent performs the action based on the features (a) to (d). (Park [Page 6, Paragraph 7]: “The learning circuit 252 can analyze the sensor data SD continuously provided over time. The learning circuit 252 can generate pattern data by analyzing the sensor data SD” Park teaches that the energy stored is continuously monitored, which may be at the last 10 time steps.)
Park is in the same field as the present invention, since it is directed to storing energy throughout power cycles. It would have been obvious, before the effective filing date of the claimed invention, to a person of ordinary skill in the art, to combine the selection of a DNN based on the energy harvest as taught in Wu as modified by Hoffman with tracking the information of various historical time step metrics as taught in Park. Park provides this additional functionality. As such, it would have been obvious to one of ordinary skill in the art to modify the teachings of Wu as modified by Hoffman to include teachings of Park because the combination would allow for the selection protocol to consider metrics in the energy harvesting those dates back more than just to the last iteration. This has the potential benefit of incorporating data, such as the last 10 time steps, which enable the most correct DNN to be chosen for an extensive historical time back.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KYU HYUNG HAN whose telephone number is (703) 756-5529. The examiner can normally be reached on MF 9-5.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alexey Shmatov can be reached on (571) 270-3428. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Kyu Hyung Han/
Examiner
Art Unit 2123
/ALEXEY SHMATOV/Supervisory Patent Examiner, Art Unit 2123
Prosecution Insights