DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Drawings
The drawings filed on 01/17/2024 are accepted.
Specification
The specification filed on 01/17/2024 is accepted.
Information Disclosure Statement
The examiner has considered the information disclosure statements (IDS) submitted on 01/17/2024, 06/21/2024 and 02/25/2026.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
Claim 1 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1:
The claim recites a system which falls within at least one of the four statutory categories of patent eligible subject matter.
Step 2:
Step 2A (prong 1):
The limitation of “select a current state from a plurality of preset states”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, “select” in the context of this claim encompasses the user selecting a specific condition of an entity from a list of conditions.
The limitation of “select an action, of the plurality of preset actions”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, “select” in the context of this claim encompasses the user selecting an item from a list of items.
The limitation of “determine a reward based on whether a process of the plurality of DNNS by the allocated plurality of processors satisfies preset constraints”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, “determine” in the context of this claim encompasses the user observing an outcome for performing an action.
The limitation of “update the at least one preset quality of the action selected in the current state based on the reward”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, “update” in the context of this claim encompasses the user may select another action based on the observed outcome.
Step 2A (prong 2):
This judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of “a plurality of processors”, “a memory” and “a plurality of deep neural networks”. The additional elements are recited at a high-level of generality (i.e., as a generic device performing the generic computer functions) such that they amount no more than mere instructions to apply the exception using the generic computer components (MPEP 2106.05(f)). Accordingly, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The additional elements of “the current state corresponding to a state of the system and to a plurality of preset actions having at least one preset quality” and “select an action, of the plurality of preset actions, having a maximum quality in the current state and respectively allocate a plurality of deep neural networks (DNNs) to the plurality of processors based on the selected action” amount to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not integrate into a practical application (see MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of “a plurality of processors”, “a memory” and “a plurality of deep neural networks” to perform the “selecting, determining and updating” steps amount to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept.
The additional elements of “the current state corresponding to a state of the system and to a plurality of preset actions having at least one preset quality” and “select an action, of the plurality of preset actions, having a maximum quality in the current state and respectively allocate a plurality of deep neural networks (DNNs) to the plurality of processors based on the selected action” amount to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the judicial exception (see MPEP 2106.05(h)).
Claim 2 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1:
The claim recites a system which falls within at least one of the four statutory categories of patent eligible subject matter.
Step 2:
Step 2A (prong 1):
The limitation of “select the current state based on at least one of a utilization or a temperature of each of the plurality of processors”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, “select” in the context of this claim encompasses the user selecting a specific condition of an entity based on some factors such as a temperature of the entity.
Step 2A (prong 2):
This judicial exception is not integrated into a practical application. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Indeed, the claim does not recite any additional element beside the limitation that can be performed in a human mind. The claim is not patent eligible.
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the claim does not recite any additional element beside the limitation that can be performed in a human mind. The claim is not patent eligible.
Claim 3 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1:
The claim recites a system which falls within at least one of the four statutory categories of patent eligible subject matter.
Step 2:
Step 2A (prong 1):
The limitation of “select the current state corresponding to the utilization of the memory”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, “select” in the context of this claim encompasses the user selecting a specific condition of an entity based on some factors such as an activity level of an entity.
Step 2A (prong 2):
This judicial exception is not integrated into a practical application. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Indeed, the claim does not recite any additional element beside the limitation that can be performed in a human mind. The claim is not patent eligible.
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the claim does not recite any additional element beside the limitation that can be performed in a human mind. The claim is not patent eligible.
Claim 4 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1:
The claim recites a system which falls within at least one of the four statutory categories of patent eligible subject matter.
Step 2:
Step 2A (prong 2):
This judicial exception is not integrated into a practical application. The claim recites an additional element of “the plurality of preset states represent a finite number of states covering a range of the utilization of the memory and at least one of the utilization or the temperature of each of the plurality of processors”. This limitation amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitation that amounts to merely indicating a field of use or technological environment in which to apply a judicial exception does not integrate into a practical application (see MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above with respect to integration of the abstract idea into a practical application, the additional element of “the plurality of preset states represent a finite number of states covering a range of the utilization of the memory and at least one of the utilization or the temperature of each of the plurality of processors” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitation that amounts to merely indicating a field of use or technological environment in which to apply a judicial exception does not amount to significantly more than the judicial exception (see MPEP 2106.05(h)).
Claim 5 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1:
The claim recites a system which falls within at least one of the four statutory categories of patent eligible subject matter.
Step 2:
Step 2A (prong 2):
This judicial exception is not integrated into a practical application. The claim recites an additional element of “select a preset state, from the plurality of preset states, as the current state, based onto at least one of a number of the DNNs corresponding to the preset state, or the number of the DNNs comprising a number of operations greater than a preset number from the DNNs”. This limitation amounts to insignificant extra-solution activities of data gathering, which does not amount to significantly more than the abstract idea (MPEP 2106.05(g)). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above with respect to integration of the abstract idea into a practical application, the additional element of “select a preset state, from the plurality of preset states, as the current state, based onto at least one of a number of the DNNs corresponding to the preset state, or the number of the DNNs comprising a number of operations greater than a preset number from the DNNs” is recited at a high level of generality and amounts to extra-solution activity of data gathering and/or transmitting (MPEP 2106.05(g)). The courts have found limitations directed to receiving and transmitting information electronically, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II), “receiving or transmitting data over a network”).
Claim 6 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1:
The claim recites a system which falls within at least one of the four statutory categories of patent eligible subject matter.
Step 2:
Step 2A (prong 2):
This judicial exception is not integrated into a practical application. The claim recites an additional element of “the operations comprise at least one of a multiplication operation, an accumulation operation, or a multiplication-accumulation (MAC) operation”. This limitation amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitation that amounts to merely indicating a field of use or technological environment in which to apply a judicial exception does not integrate into a practical application (see MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above with respect to integration of the abstract idea into a practical application, the additional element of “the operations comprise at least one of a multiplication operation, an accumulation operation, or a multiplication-accumulation (MAC) operation” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitation that amounts to merely indicating a field of use or technological environment in which to apply a judicial exception does not amount to significantly more than the judicial exception (see MPEP 2106.05(h)).
Claim 7 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1:
The claim recites a system which falls within at least one of the four statutory categories of patent eligible subject matter.
Step 2:
Step 2A (prong 2):
This judicial exception is not integrated into a practical application. The claim recites the additional element of “the plurality of processors comprise at least one of a central processing unit (CPU), a graphics processing unit (GPU), a neural processing unit (NPU), or a digital signal processor (DSP)”. This additional element is recited at a high-level of generality (i.e., as a generic device performing the generic computer functions) such that it amounts to no more than mere instructions to apply the exception using the generic computer components (MPEP 2106.05(f)). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above with respect to integration of the abstract idea into a practical application, the additional element of “the plurality of processors comprise at least one of a central processing unit (CPU), a graphics processing unit (GPU), a neural processing unit (NPU), or a digital signal processor (DSP)” amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept.
Claim 8 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1:
The claim recites a system which falls within at least one of the four statutory categories of patent eligible subject matter.
Step 2:
Step 2A (prong 2):
This judicial exception is not integrated into a practical application. The claim recites an additional element of “the plurality of preset actions comprise respectively allocating the plurality of DNNs to the plurality of processors in a preset combination”. This limitation amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitation that amounts to merely indicating a field of use or technological environment in which to apply a judicial exception does not integrate into a practical application (see MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above with respect to integration of the abstract idea into a practical application, the additional element of “the plurality of preset actions comprise respectively allocating the plurality of DNNs to the plurality of processors in a preset combination” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitation that amounts to merely indicating a field of use or technological environment in which to apply a judicial exception does not amount to significantly more than the judicial exception (see MPEP 2106.05(h)).
Claim 9 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1:
The claim recites a system which falls within at least one of the four statutory categories of patent eligible subject matter.
Step 2:
Step 2A (prong 1):
The limitation of “setting at least one value of a voltage or a frequency of each of the plurality of processors to preset values”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, “setting” in the context of this claim encompasses the user assigning a certain value for an entity.
Step 2A (prong 2):
This judicial exception is not integrated into a practical application. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Indeed, the claim does not recite any additional element beside the limitation that can be performed in a human mind. The claim is not patent eligible.
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the claim does not recite any additional element beside the limitation that can be performed in a human mind. The claim is not patent eligible.
Claim 10 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1:
The claim recites a system which falls within at least one of the four statutory categories of patent eligible subject matter.
Step 2:
Step 2A (prong 1):
The limitation of “set at least one value of a voltage or a frequency of each of the plurality of processors, for performing a process, according to the action selected in the current state”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, “set” in the context of this claim encompasses the user assigning a certain value for an entity based on some factors.
Step 2A (prong 2):
This judicial exception is not integrated into a practical application. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Indeed, the claim does not recite any additional element beside the limitation that can be performed in a human mind. The claim is not patent eligible.
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the claim does not recite any additional element beside the limitation that can be performed in a human mind. The claim is not patent eligible.
Claim 11 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1:
The claim recites a system which falls within at least one of the four statutory categories of patent eligible subject matter.
Step 2:
Step 2A (prong 1):
The limitation of “determine the reward based on whether the at least one of the time or the accuracy of the process satisfies the preset constraints”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, “determine” in the context of this claim encompasses the user determining an outcome associated with a certain feature when performing an action.
Step 2A (prong 2):
This judicial exception is not integrated into a practical application. The claim recites an additional element of “obtain at least one of time or accuracy of a process according to the action selected in the current state”. This limitation amounts to insignificant extra-solution activities of data gathering, which does not amount to significantly more than the abstract idea (MPEP 2106.05(g)). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above with respect to integration of the abstract idea into a practical application, the additional element of “obtain at least one of time or accuracy of a process according to the action selected in the current state” is recited at a high level of generality and amounts to extra-solution activity of data gathering and/or transmitting (MPEP 2106.05(g)). The courts have found limitations directed to receiving and transmitting information electronically, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II), “receiving or transmitting data over a network”).
Claim 12 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1:
The claim recites a system which falls within at least one of the four statutory categories of patent eligible subject matter.
Step 2:
Step 2A (prong 1):
The limitation of “determine the reward based on whether the temperature of the plurality of processors satisfies the preset constraints”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, “determine” in the context of this claim encompasses the user determining an outcome associated with a certain feature when performing an action.
Step 2A (prong 2):
This judicial exception is not integrated into a practical application. The claim recites an additional element of “obtain a temperature of the plurality of processors during a runtime of a process according to the action selected in the current state”. This limitation amounts to insignificant extra-solution activities of data gathering, which does not amount to significantly more than the abstract idea (MPEP 2106.05(g)). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above with respect to integration of the abstract idea into a practical application, the additional element of “obtain a temperature of the plurality of processors during a runtime of a process according to the action selected in the current state” is recited at a high level of generality and amounts to extra-solution activity of data gathering and/or transmitting (MPEP 2106.05(g)). The courts have found limitations directed to receiving and transmitting information electronically, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II), “receiving or transmitting data over a network”).
Claim 13 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1:
The claim recites a system which falls within at least one of the four statutory categories of patent eligible subject matter.
Step 2:
Step 2A (prong 1):
The limitation of “calculate the reward based on at least one of a time and accuracy of a process, according to the action selected in the current state, or a temperature and an energy consumption of the plurality of processors during a runtime of the process according to the action selected in the current state”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, “calculate” in the context of this claim encompasses the user determining an outcome associated with a certain feature when performing an action.
Step 2A (prong 2):
This judicial exception is not integrated into a practical application. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Indeed, the claim does not recite any additional element beside the limitation that can be performed in a human mind. The claim is not patent eligible.
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the claim does not recite any additional element beside the limitation that can be performed in a human mind. The claim is not patent eligible.
Claim 14 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1:
The claim recites a system which falls within at least one of the four statutory categories of patent eligible subject matter.
Step 2:
Step 2A (prong 2):
This judicial exception is not integrated into a practical application. The claim recites the additional element of “the reinforcement learning is based on Q-learning, and the at least one processor is further configured to update the quality of the action selected in the current state based on the Q-learning”. This additional element is recited at a high-level of generality (i.e., as a generic device performing the generic computer functions) such that it amounts to no more than mere instructions to apply the exception using the generic computer components (MPEP 2106.05(f)). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above with respect to integration of the abstract idea into a practical application, the additional element of “the reinforcement learning is based on Q-learning, and the at least one processor is further configured to update the quality of the action selected in the current state based on the Q-learning” amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept.
Claim 15 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1:
The claim recites a system which falls within at least one of the four statutory categories of patent eligible subject matter.
Step 2:
Step 2A (prong 1):
The limitation of “select a particular state from a plurality of preset states”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, “select” in the context of this claim encompasses the user selecting a specific condition of an entity from a list of conditions.
The limitation of “select a particular action, of the plurality of preset actions”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, “select” in the context of this claim encompasses the user selecting an item from a list of items.
The limitation of “respectively allocate a plurality of deep neural networks to the plurality of processors based on the selection of the particular action”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, “allocate” in the context of this claim encompasses the user placing the items into one or more entities based on some factors.
Step 2A (prong 2):
This judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of “a deep neural network”, “a memory” and “a plurality of processing units”. The additional elements are recited at a high-level of generality (i.e., as a generic device performing the generic computer functions) such that they amount no more than mere instructions to apply the exception using the generic computer components (MPEP 2106.05(f)). Accordingly, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The additional elements of “the particular state corresponding to a state of the system and to a plurality of preset actions having at least one preset quality” and “select a particular action, of the plurality of preset actions, having a maximum quality in the particular state” amount to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not integrate into a practical application (see MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of “a deep neural network”, “a memory” and “a plurality of processing units” to perform the “selecting and allocating” steps amount to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept.
The additional elements of “the particular state corresponding to a state of the system and to a plurality of preset actions having at least one preset quality” and “select a particular action, of the plurality of preset actions, having a maximum quality in the particular state” amount to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the judicial exception (see MPEP 2106.05(h)).
Claim 16 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1:
The claim recites a system which falls within at least one of the four statutory categories of patent eligible subject matter.
Step 2:
Step 2A (prong 2):
This judicial exception is not integrated into a practical application. The claim recites the additional element of “the reinforcement learning is based on Q-learning configured to update at least one quality to maximize a reward based on the particular action”. This additional element is recited at a high-level of generality (i.e., as a generic device performing the generic computer functions) such that it amounts to no more than mere instructions to apply the exception using the generic computer components (MPEP 2106.05(f)). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The additional elements of “wherein the reward has at least one of a larger value as a process time of a process of the plurality of DNNs, by the allocated plurality of processors and according to the particular action, decreases, a larger value as accuracy of the process increases, a larger value as temperature of the plurality of processors decreases during a runtime of the process, or a larger value as energy consumption of the plurality of processors decreases” amount to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not integrate into a practical application (see MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above with respect to integration of the abstract idea into a practical application, the additional element of “the reinforcement learning is based on Q-learning configured to update at least one quality to maximize a reward based on the particular action” amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept.
The additional elements of “wherein the reward has at least one of a larger value as a process time of a process of the plurality of DNNs, by the allocated plurality of processors and according to the particular action, decreases, a larger value as accuracy of the process increases, a larger value as temperature of the plurality of processors decreases during a runtime of the process, or a larger value as energy consumption of the plurality of processors decreases” amount to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the judicial exception (see MPEP 2106.05(h)).
Claim 17 is substantially similar to claim 4 and thus rejected for similar reasons as claim 4.
Claim 18 is substantially similar to claim 10 and thus rejected for similar reasons as claim 10.
Claim 19 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1:
The claim recites a method which falls within at least one of the four statutory categories of patent eligible subject matter.
Step 2:
Step 2A (prong 1):
The limitation of “selecting a particular state from a plurality of preset states”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, “selecting” in the context of this claim encompasses the user selecting a specific condition of an entity from a list of conditions.
The limitation of “selecting a particular action, of the plurality of preset actions”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, “selecting” in the context of this claim encompasses the user selecting an item from a list of items.
The limitation of “respectively allocating a plurality of deep neural networks to the plurality of processors based on the selection of the particular action”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, “allocating” in the context of this claim encompasses the user placing the items into one or more entities based on some factors.
Step 2A (prong 2):
This judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of “a plurality deep neural network” and “processing units”. The additional elements are recited at a high-level of generality (i.e., as a generic device performing the generic computer functions) such that they amount no more than mere instructions to apply the exception using the generic computer components (MPEP 2106.05(f)). Accordingly, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The additional elements of “the particular state corresponding to a state of a system and to a plurality of preset actions having at least one preset quality” and “selecting a particular action, from the plurality of preset actions, having a maximum quality in the particular state” amount to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not integrate into a practical application (see MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of “a plurality deep neural network” and “processing units” to perform the “selecting and allocating” steps amount to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept.
The additional elements of “the particular state corresponding to a state of a system and to a plurality of preset actions having at least one preset quality” and “selecting a particular action, from the plurality of preset actions, having a maximum quality in the particular state” amount to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the judicial exception (see MPEP 2106.05(h)).
Claim 20 is substantially similar to claim 10 and thus rejected for similar reasons as claim 10.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-11 and 13-20 are rejected under 35 U.S.C. 103 as being unpatentable over Kim et al. (AutoScale: Optimizing Energy Efficiency of End-to-End Edge Inference under Stochastic Variance – Applicant provided NPL) in view of Xun et al. (Optimising Resource Management for Embedded Machine Learning).
As per claim 1, Kim teaches a system configured to allocate deep neural networks to a plurality of processors based on reinforcement learning [page 5, section 3.3, last paragraph, “our proposed AutoScale which self-learns the optimal execution target under the presence of runtime variances based on reinforcement learning”], the system comprising:
the plurality of processors, wherein at least one processor of the plurality of processors, by performing the one or more instructions, is configured to [page 1, Introduction, Col. 2, 2nd paragraph, “To address these performance and energy efficiency challenges, modern mobile devices employ more and more accelerators and/or co-processors, such as Graphic Processing Units (GPU), Digital Signal Processors”]:
select a current state from a plurality of preset states, the current state corresponding to a state of the system and to a plurality of preset actions having at least one preset quality [page 5, section 4, 1st paragraph, “Figure 8 provides the design overview of AutoScale in the context of the mobile and edge-cloud DNN inference execution. For each inference execution, AutoScale observes the current execution state, including NN characteristics as well as runtime variances”; Fig. 8 shows a lookup table (Q-table) comprising a plurality of state-action combinations],
select an action, of the plurality of preset actions, having a maximum quality in the current state [page 5, section 4, 1st paragraph to page 6, Col. 1, 1st paragraph, “Figure 8 provides the design overview of AutoScale in the context of the mobile and edge-cloud DNN inference execution. For each inference execution, AutoScale observes the current execution state, including NN characteristics as well as runtime variances. For the observed state, AutoScale selects an action (i.e., execution target), which is expected to maximize energy efficiency satisfying QoS and inference quality target, based on a lookup table (i.e., Q-table) … executes DNN inference on the target defined by the selected action, while observing its result (i.e., energy, latency, and inference accuracy)”; page 8, section 4.2, last paragraph, “the Q-table is used to select A which maximizes Q(S,A) for the observed S”] and respectively allocate … to the plurality of processors based on the selected action [Fig. 8, page 7, Col. 1, 1st paragraph, “Actions in reinforcement learning represent the choosable control knobs of the system. In the context of the edge-cloud inference execution, we define the actions as the available execution targets. For the edge inference execution, available processors in mobile SoCs, such as CPUs, GPUs, DSPs, and NPUS, are defined as the actions. On the other hand, for the cloud execution, server-class processors, such as CPUs, GPUs, and TPUs, are defined as the actions”; Fig. 8 and the above reciting discloses that for inference execution, the actions are defined as execution targets (hardware or software platform used to perform inference), Fig. 8 shows the available processors used to execute the inference such as (CPU, GPU and DSP],
determine a reward [page 6, Col. 1, 1st paragraph, “Based on the observed result, AutoScale calculates the reward, which indicates how much the selected action improves energy efficiency and satisfies QoS and accuracy targets”], and
update the at least one preset quality of the action selected in the current state based on the reward [page 6, Col. 1, 1st and 2nd paragraphs, “updates Q-table with the calculated reward … AutoScale leverages Reinforcement Learning (RL) as an adaptive prediction mechanism. Generally, an RL agent learns a policy to select the best action for a given state, based on accumulated rewards”].
Kim does not explicitly teach
a memory configured to store one or more instructions;
allocate a plurality of deep neural networks (DNNs) to the plurality of processors;
determine a reward based on whether a process of the plurality of DNNS by the allocated plurality of processors satisfies preset constraints;
Xun teaches
a memory configured to store one or more instructions [Fig. 2, memory];
allocate a plurality of deep neural networks (DNNs) to the plurality of processors [page 1, Fig. 1, “1. DNNs can be deployed on a variety of hardware platforms (CPU, GPU and DSP) with different computing resources. At design time, the DNN is compressed … and then mapped onto different computing resources to meet the performance requirements of the application”; Since Kim on pages 5 and 8, teaches for the inference execution, certain processors which are defined as actions are selected and used to execute the inference such as (CPU, GPU and DSP) to improve energy efficiency and accuracy, while Xun teaches to optimize the performance requirements of the application, deep neural networks are allocated to multiple hardware platforms (CPU, GPU and DSP) with different computing resources, thus, the combination of Kim and Xun teaches the claim limitation of selecting an action which is defined as execution targets (hardware platforms used to perform inference) and allocation a plurality of deep neural networks to the plurality of processors (hardware platforms)];
determine a reward based on whether a process of the plurality of DNNS by the allocated plurality of processors satisfies preset constraints [page 1, Fig. 1, “1. DNNs can be deployed on a variety of hardware platforms (CPU, GPU and DSP) with different computing resources. At design time, the DNN is compressed … and then mapped onto different computing resources to meet the performance requirements of the application”; page 2, Col. 2, 3rd paragraph, “the same DNN might be used uncompressed on one platform with an NPU, while a compressed model (with offering lower accuracy but requiring fewer computations) is deployed on a different platform containing only CPU and GPU cores in order to meet the same execution time and energy consumption requirements”; It can be seen that allocating the model to other platform and satisfying the time and energy constraint indicating a reward];
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have modified the deep learning execution scaling engine of Kim to include allocating a plurality of deep neural networks (DNNs) to the plurality of processors, and determining a reward of Xun. Doing so would help meeting the performance requirements of the application (Xun, page 1, Fig. 1).
As per claim 2, Kim and Xun teach the system of claim 1.
Kim further teaches
select the current state based on at least one of a utilization or a temperature of each of the plurality of processors [page 6, section 4.1, 1st – 4th paragraphs, “In RL, there are three core components: State, Action, and Reward. In this section, we define the core components to formulate the optimization space for AutoScale … Table 1 summarizes the states … we use SCo_CPU and SCo-MEM which represent the CPU utilization and memory usage of co-running applications, respectively”].
As per claim 3, Kim and Xun teach the system of claim 2.
Kim further teaches
select the current state corresponding to the utilization of the memory [page 6, section 4.1, 1st – 4th paragraphs, “In RL, there are three core components: State, Action, and Reward. In this section, we define the core components to formulate the optimization space for AutoScale … Table 1 summarizes the states … we use SCo_CPU and SCo-MEM which represent the CPU utilization and memory usage of co-running applications, respectively”].
As per claim 4, Kim and Xun teach the system of claim 3.
Kim further teaches
the plurality of preset states represent a finite number of states covering a range of the utilization of the memory and at least one of the utilization or the temperature of each of the plurality of processors [Table 1 shows the finite number of states, utilization of CPU, memory usage, range of the utilization and their representation is discretised].
As per claim 5, Kim and Xun teach the system of claim 1.
Kim further teaches
select a preset state, from the plurality of preset states, as the current state, based onto at least one of a number of the DNNs corresponding to the preset state, or the number of the DNNs comprising a number of operations greater than a preset number from the DNNs [page 6, section 4.1, State, 2nd paragraph, “we identify states with layer types that are deeply correlated to the energy efficiency and performance of inference execution … We find CONV, FC, and RC layers are the most correlated to the energy efficiency and performance, due to their compute- and/or memory-intensive natures. Thus, we identify SCONV, SFC, and SRC which represent the number of CONV, FC, and RC layers in NNs, respectively”].
As per claim 6, Kim and Xun teach the system of claim 5.
Kim further teaches
the operations comprise at least one of a multiplication operation, an accumulation operation, or a multiplication-accumulation (MAC) operation [page 6, section 4.1, State, 2nd paragraph, “We also identify SMAC, the number of MAC operations to consider heaviness of NNs”].
As per claim 7, Kim and Xun teach the system of claim 1.
Kim further teaches
the plurality of processors comprise at least one of a central processing unit (CPU), a graphics processing unit (GPU), a neural processing unit (NPU), or a digital signal processor (DSP) [page 1, Introduction, Col. 2, 2nd paragraph, “To address these performance and energy efficiency challenges, modern mobile devices employ more and more accelerators and/or co-processors, such as Graphic Processing Units (GPU), Digital Signal Processors”; Fig. 8 shows the available processors used to execute the inference such as (CPU, GPU and DSP].
As per claim 8, Kim and Xun teach the system of claim 1.
Xun teaches
allocate a plurality of DNNs to the plurality of processors [page 1, Fig. 1, “1. DNNs can be deployed on a variety of hardware platforms (CPU, GPU and DSP) with different computing resources. At design time, the DNN is compressed … and then mapped onto different computing resources to meet the performance requirements of the application”; Since Kim on pages 5 and 8, teaches for the inference execution, certain processors which are defined as actions are selected and used to execute the inference such as (CPU, GPU and DSP) to improve energy efficiency and accuracy, while Xun teaches to optimize the performance requirements of the application, deep neural networks are allocated to multiple hardware platforms (CPU, GPU and DSP) with different computing resources, thus, the combination of Kim and Xun teaches selecting an action which is defined as execution targets (hardware platforms used to perform inference) for allocating a plurality of deep neural networks to the plurality of processors (hardware platforms)];
Xun further teaches
allocating the plurality of DNNs to the plurality of processors in a preset combination [page 3, Col. 2, 2nd paragraph, “To achieve consistent performance across different platforms, the DNN is compressed more on platforms with less computing capabilities … For example, the full DNN model can be deployed on an NPU, yet the smaller compressed models can be deployed on GPUs and CPUs to meet the same time budget with less accuracy”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have modified the deep learning execution scaling engine of Kim to include allocating the plurality of DNNs to the plurality of processors in a preset combination of Xun. Doing so would help achieving consistent performance across different platforms (Xun, page 3, Col. 2, 2nd paragraph).
As per claim 9, Kim and Xun teach the system of claim 8.
Kim further teaches
the plurality of preset actions further comprise setting at least one value of a voltage or a frequency of each of the plurality of processors to preset values [page 7, Col. 1, 1st paragraph, “The set of actions can be augmented to consider other control knobs, such as Dynamic Voltage and Frequency Scaling (DVFS); page 9, section 5.3, 1st paragraph, “Since the energy efficiency of mobile CPU/GPU can be further optimized via DVFS, we identify each Voltage/Frequency (V/F) step of mobile CPU/GPU as the augmented action; the number of available V/F steps is presented in Table 2”; It can be seen that DVFS is a power management technique for setting voltage and frequency of a processor in real-time to optimize energy efficiency].
As per claim 10, Kim and Xun teach the system of claim 1.
Kim further teaches
set at least one value of a voltage or a frequency of each of the plurality of processors, for performing a process, according to the action selected in the current state [page 7, Col. 1, 1st paragraph, “The set of actions can be augmented to consider other control knobs, such as Dynamic Voltage and Frequency Scaling (DVFS); page 9, section 5.3, 1st paragraph, “Since the energy efficiency of mobile CPU/GPU can be further optimized via DVFS, we identify each Voltage/Frequency (V/F) step of mobile CPU/GPU as the augmented action; the number of available V/F steps is presented in Table 2”; It can be seen that DVFS is a power management technique for setting voltage and frequency of a processor in real-time to optimize energy efficiency].
As per claim 11, Kim and Xun teach the system of claim 1.
Kim further teaches
obtain at least one of time or accuracy of a process according to the action selected in the current state, and determine the reward based on whether the at least one of the time or the accuracy of the process satisfies the preset constraints [Fig. 8, page 5, section 4, 1st paragraph to page 6, Col. 1, 1st paragraph, “For each inference execution, AutoScale observes the current execution state, including NN characteristics as well as runtime variances. For the observed state, AutoScale selects an action (i.e., execution target), which is expected to maximize energy efficiency satisfying QoS and inference quality target, based on a lookup table (i.e., Q-table) … executes DNN inference on the target defined by the selected action, while observing its result (i.e., energy, latency, and inference accuracy) … Based on the observed result, AutoScale calculates the reward, which indicates how much the selected action improves energy efficiency and satisfies QoS and accuracy targets”];
As per claim 13, Kim and Xun teach the system of claim 1.
Kim further teaches
calculate the reward based on at least one of a time and accuracy of a process, according to the action selected in the current state, or a temperature and an energy consumption of the plurality of processors during a runtime of the process according to the action selected in the current state [Fig. 8, page 5, section 4, 1st paragraph to page 6, Col. 1, 1st paragraph, “For each inference execution, AutoScale observes the current execution state, including NN characteristics as well as runtime variances. For the observed state, AutoScale selects an action (i.e., execution target), which is expected to maximize energy efficiency satisfying QoS and inference quality target, based on a lookup table (i.e., Q-table) … executes DNN inference on the target defined by the selected action, while observing its result (i.e., energy, latency (time), and inference accuracy) … Based on the observed result, AutoScale calculates the reward, which indicates how much the selected action improves energy efficiency and satisfies QoS and accuracy targets”];
As per claim 14, Kim and Xun teach the system of claim 1.
Kim further teaches
the reinforcement learning is based on Q-learning, and the at least one processor is further configured to update the quality of the action selected in the current state based on the Q-learning [page 6, Col. 1, 1st and 2nd paragraphs, “updates Q-table with the calculated reward … AutoScale leverages Reinforcement Learning (RL) as an adaptive prediction mechanism. Generally, an RL agent learns a policy to select the best action for a given state, based on accumulated rewards”; page 8, section 4.2, “In Q-learning, the value function, denoted as Q(S,A), takes State S and Action A as parameters. Q(S,A) is a form of a look-up table, called Q-table. Algorithm 1 shows the detailed algorithm for training the Q-table for on-device DNN inference. At runtime, for each DNN inference, the algorithm observes S by checking the NN characteristics and runtime variances … the algorithm chooses A with the largest Q(S,A). After choosing A, the algorithm runs the inference on a target defined by A. During the inference, the algorithm measures Rlatency and estimates Renergy, as explained in Section 4.1. In addition, it obtains Raccuracy from the stored inference accuracy of the given NN on the selected execution target. Based on these values, the algorithm calculates reward R as in (5) of Section 4.1. After calculating the R value, the algorithm observes new state S' and chooses A' for the given S' with the largest Q(S',A'). The algorithm updates the Q(S,A) based on the equation in Algorithm 1”].
As per claim 15, Kim teaches a system configured to allocate a deep neural network (DNN) based on reinforcement learning [page 5, section 3.3, last paragraph, “our proposed AutoScale which self-learns the optimal execution target under the presence of runtime variances based on reinforcement learning”], the system comprising:
a plurality of processing units, wherein at least one processor, of the plurality of processing units, by executing the one or more instructions, is configured to [page 1, Introduction, Col. 2, 2nd paragraph, “To address these performance and energy efficiency challenges, modern mobile devices employ more and more accelerators and/or co-processors, such as Graphic Processing Units (GPU), Digital Signal Processors”]:
select a particular state from a plurality of preset states, the particular state corresponding to a state of the system and to a plurality of preset actions having at least one preset quality [page 5, section 4, 1st paragraph, “Figure 8 provides the design overview of AutoScale in the context of the mobile and edge-cloud DNN inference execution. For each inference execution, AutoScale observes the current execution state, including NN characteristics as well as runtime variances”; Fig. 8 shows a lookup table (Q-table) comprising a plurality of state-action combinations],
select a particular action, of the plurality of preset actions, having a maximum quality in the particular state [page 5, section 4, 1st paragraph to page 6, Col. 1, 1st paragraph, “Figure 8 provides the design overview of AutoScale in the context of the mobile and edge-cloud DNN inference execution. For each inference execution, AutoScale observes the current execution state, including NN characteristics as well as runtime variances. For the observed state, AutoScale selects an action (i.e., execution target), which is expected to maximize energy efficiency satisfying QoS and inference quality target, based on a lookup table (i.e., Q-table) … executes DNN inference on the target defined by the selected action, while observing its result (i.e., energy, latency, and inference accuracy)”; page 8, section 4.2, last paragraph, “the Q-table is used to select A which maximizes Q(S,A) for the observed S”], and
respectively allocate … to the plurality of processors based on the selection of the particular action [Fig. 8, page 7, Col. 1, 1st paragraph, “Actions in reinforcement learning represent the choosable control knobs of the system. In the context of the edge-cloud inference execution, we define the actions as the available execution targets. For the edge inference execution, available processors in mobile SoCs, such as CPUs, GPUs, DSPs, and NPUS, are defined as the actions. On the other hand, for the cloud execution, server-class processors, such as CPUs, GPUs, and TPUs, are defined as the actions”; Fig. 8 and the above reciting discloses that for inference execution, the actions are defined as execution targets (hardware or software platform used to perform inference), Fig. 8 shows the available processors used to execute the inference such as (CPU, GPU and DSP],
Kim does not explicitly teach
a memory configured to store one or more instructions; and
allocate a plurality of deep neural networks to the plurality of processors based on the selection of the particular action.
Xun teaches
a memory configured to store one or more instructions [Fig. 2, memory];
allocate a plurality of deep neural networks to the plurality of processors based on the selection of the particular action [page 1, Fig. 1, “1. DNNs can be deployed on a variety of hardware platforms (CPU, GPU and DSP) with different computing resources. At design time, the DNN is compressed … and then mapped onto different computing resources to meet the performance requirements of the application”; Since Kim on pages 5 and 8, teaches for the inference execution, certain processors which are defined as actions are selected and used to execute the inference such as (CPU, GPU and DSP) to improve energy efficiency and accuracy, while Xun teaches to optimize the performance requirements of the application, deep neural networks are allocated to multiple hardware platforms (CPU, GPU and DSP) with different computing resources, thus, the combination of Kim and Xun teaches the claim limitation of selecting an action which is defined as execution targets (hardware platforms used to perform inference) and allocation a plurality of deep neural networks to the plurality of selected processors (hardware platforms)];
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have modified the deep learning execution scaling engine of Kim to include allocating a plurality of deep neural networks (DNNs) to the plurality of processors of Xun. Doing so would help meeting the performance requirements of the application (Xun, page 1, Fig. 1).
As per claim 16, Kim and Xun teach the system of claim 15.
Kim further teaches
the reinforcement learning is based on Q-learning configured to update at least one quality to maximize a reward based on the particular action [page 8, Algorithm 1, training Q-Learning Model, “Choose action A which maximizes Q(S,A)”], and
wherein the reward has at least one of
a larger value as a process time of a process of the plurality of DNNs, by the allocated plurality of processors and according to the particular action, decreases,
a larger value as accuracy of the process increases,
a larger value as temperature of the plurality of processors decreases during a runtime of the process, or
a larger value as energy consumption of the plurality of processors decreases.
[pages 7-8, section 4.2, “In Q-learning, the value function, denoted as Q(S,A), takes State S and Action A as parameters. Q(S,A) is a form of a look-up table, called Q-table. Algorithm 1 shows the detailed algorithm for training the Q-table for on-device DNN inference. At runtime, for each DNN inference, the algorithm observes S by checking the NN characteristics and runtime variances … the algorithm chooses A with the largest Q(S,A). After choosing A, the algorithm runs the inference on a target defined by A. During the inference, the algorithm measures Rlatency and estimates Renergy, as explained in Section 4.1. In addition, it obtains Raccuracy from the stored inference accuracy of the given NN on the selected execution target. Based on these values, the algorithm calculates reward R as in (5) of Section 4.1 … R = -Renergy + αRlarency + βRaccuracy; It can be seen that the reward R is larger as the accuracy of the process increase].
Claim 17 is substantially similar to claim 4 and thus rejected for similar reasons as claim 4.
Claim 18 is substantially similar to claim 10 and thus rejected for similar reasons as claim 10.
Claim 19 is substantially similar to claim 15 and thus rejected for similar reasons as claim 15.
Claim 20 is substantially similar to claim 10 and thus rejected for similar reasons as claim 10.
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Kim et al. in view of Xun et al. and further in view of Devulapalli et al. (US Pub. 2019/0042979).
As per claim 12, Kim and Xun teach the system of claim 1.
Kim and Xun do not explicitly teach
obtain a temperature of the plurality of processors during a runtime of a process according to the action selected in the current state, and determine the reward based on whether the temperature of the plurality of processors satisfies the preset constraints.
Devulapalli teaches
obtain a temperature of the plurality of processors during a runtime of a process according to the action selected in the current state, and determine the reward based on whether the temperature of the plurality of processors satisfies the preset constraints [paragraph 0016, “the reinforcement information may include one or more of reward information and penalty information. In some embodiments, the logic 22 may be configured to learn the thermal behavior of the system based on adjustments to increase the reward information and decrease the penalty information. For example, increased reward information may correspond to one or more of increased processor frequencies and reduced active cooling, and increased penalty information may correspond to processor temperatures above a threshold temperature”; paragraph 0040, “control the CPU frequency to keep the temperature below a specified limit (e.g., 70 degrees Celsius) with little or no effect on performance. The agent may receive rewards for increasing frequency (e.g., the higher the frequency, the higher the reward) and the agent may be penalized if the CPU temperature exceeded the specified limit. The passive cooling RL agent may initially explore different actions and try all the possible frequency settings. After a number of reinforced learning steps, the passive cooling RL agent may learn to select an action that maximizes the CPU frequency while maintaining the CPU temperature below the specified limit (e.g., or a set critical point)”; It can be seen that the system monitoring the thermal behavior of the processor, and the reward is based on the processor temperatures, higher reward is obtained when he CPU temperature below the specified limit].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have modified the deep learning execution scaling engine of Kim to include obtain a temperature of the plurality of processors during a runtime of a process according to the action selected in the current state, and determine the reward based on whether the temperature of the plurality of processors satisfies the preset constraints of Devulapalli. Doing so would help selecting an action that maximizes the CPU frequency while maintaining the CPU temperature below the specified limit (Devulapalli, 0040).
Prior Art
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure.
Jia et al. (US Pub. 2022/0237045) describes a method for allocating a set of operations to multiple computing units in a computing system.
Huber et al. (US Patent 12,013,673) describes building control method using reinforcement learning.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TRI T NGUYEN whose telephone number is 571-272-0103. The examiner can normally be reached M-F, 8 AM-5 PM, (CT).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, OMAR FERNANDEZ can be reached at 571-272-2589. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TRI T NGUYEN/Examiner, Art Unit 2128
/OMAR F FERNANDEZ RIVAS/Supervisory Patent Examiner, Art Unit 2128
Prosecution Insights