Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This action is responsive to the Application filed on 09/10/2024. Claims 1-20 are pending in this case. Claims 1, 15 and 20 are independent claim. No claims have been amended and/or canceled.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim(s) 1-20 are rejected under 35 U.S.C. § 101 because the claimed invention is directed to an abstract idea without significantly more.
When considering subject matter eligibility under 35 U.S.C. 101, it must be determined whether the claim is directed to one of the four statutory categories of invention, i.e., process, machine, manufacture, or composition of matter (Step 1). If the claim does fall within one of the statutory categories, the second step in the analysis is to determine whether the claim is directed to a judicial exception (Step 2A). The Step 2A analysis is broken into two prongs. In the first prong (Step 2A, Prong 1), it is determined whether or not the claims recite a judicial exception (e.g., mathematical concepts, mental processes, certain methods of organizing human activity). If it is determined in Step 2A, Prong 1 that the claims recite a judicial exception, the analysis proceeds to the second prong (Step 2A, Prong 2), where it is determined whether or not the claims integrate the judicial exception into a practical application. If it is determined at step 2A, Prong 2 that the claims do not integrate the judicial exception into a practical application, the analysis proceeds to determining whether the claim is a patent-eligible application of the exception (Step 2B). If an abstract idea is present in the claim, any element or combination of elements in the claim must be sufficient to ensure that the claim integrates the judicial exception into a practical application, or else amounts to significantly more than the abstract idea itself. Applicant is advised to consult the 2019 PEG for more details of the analysis.
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Claims 1-14 are drawn to a method, claims 15-19 are drawn to a system and claim 20 is drawn to a non-transitory computer-readable medium, therefore each of these claim groups falls under one of four categories of statutory subject matter (machine/products/apparatus, process/method, manufactures and compositions of matter; Step 1). Nonetheless, the claims are directed to a judicially recognized exception of an abstract idea without significant more (Step 2A, see below). Independent claims 1, 13 and 19 are nonverbatim but similar in claim construction, hence share the same rationale that the claimed inventions are directed to non-statutory subject matter as follows:
Regarding claim 1:
Claim 1 recites: A method, comprising: identifying a training objective to be executable by a hardware computing device;
identifying a hardware cost corresponding to a set of features of the hardware computing device;
applying, by one or more processors, the hardware cost to a neural network during training to achieve the training objective;
generating, via the training according to the applied hardware cost, a sparsity pattern in a set of layers of the neural network; and
generating a hardware implementation of the training objective in the hardware computing device according to the sparsity pattern
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Claim 1 is directed to an abstract idea, specifically, a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Independent claim 1 recites in part:
identifying a training objective to be executable by a hardware computing device
The limitation above is broadly and reasonably interpreted as a mental concept that can practically be performed in the human mind or with pen and paper, including observation evaluations, judgments, and opinions. For example, a person could review information and identify what objective the neural network should accomplish, identifying that the objective is classification, prediction, or execution of a particular function. See MPEP § 2106.04(a)(2)(III).
identifying a hardware cost corresponding to a set of features of the hardware computing device
The limitation above is broadly and reasonably interpreted as a mental concept that can practically be performed in the human mind or with pen and paper, including observation evaluations, judgments, and opinions. For example, looking at hardware characteristics and identifying an associated latency, power, logic-resource, or timing cost—that is an evaluation of information and falls under a mental process. See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
Independent claim 1 recites in part:
A method, comprising: applying, by one or more processors, the hardware cost to a neural network during training to achieve the training objective, as drafted, amount to the judicial exception is not integrated into a practical application. In particular, the claims only recites generic computing components. Such generic computing components are recited at a high-level of generality (i.e., as a generic processor performing data gathering and mathematical calculations) such that they amount to no more than mere instructions to apply the exception using generic computer components.
generating, via the training according to the applied hardware cost, a sparsity pattern in a set of layers of the neural network is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception and reciting only the idea of a solution or outcome, i.e., the claim fails to recite details of how a solution to a problem is accomplished because it is unclear how the “NN” is used nor the specification makes it clear how these actions are performed. Thus, these additional elements are recited in a manner that represent no more than mere instructions to apply the judicial exceptions on a computer. See MPEP § 2106.05(f) and § 2106.04(d).
generating a hardware implementation of the training objective in the hardware computing device according to the sparsity pattern is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h).
Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea when considered as an ordered combination and as a whole.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
First, the additional elements directed to generally linking the use of a judicial exception to a particular technological environment or field of use are deemed insufficient to transform the judicial exception to a patentable invention because the claimed limitations generally link the judicial exception to the technology environment, see MPEP 2106.05(h). However, they are included below for the sake of completeness.
Second, the additional elements mere application of the abstract idea or mere instructions to implement an abstract idea on a computer are deemed insufficient to transform the judicial exception to a patentable invention because the limitations generally apply the use of a generic computer and/or process with the judicial exception. See MPEP 2106.05(f). However, they are included below for the sake of completeness.
Independent claim 1 recites in part:
A method, comprising: applying, by one or more processors, the hardware cost to a neural network during training to achieve the training objective, as drafted, amount to the judicial exception is not integrated into a practical application. In particular, the claims only recites generic computing components. Such generic computing components are recited at a high-level of generality (i.e., as a generic processor performing data gathering and mathematical calculations) such that they amount to no more than mere instructions to apply the exception using generic computer components.
generating, via the training according to the applied hardware cost, a sparsity pattern in a set of layers of the neural network is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception and reciting only the idea of a solution or outcome, i.e., the claim fails to recite details of how a solution to a problem is accomplished because it is unclear how the “NN” is used nor the specification makes it clear how these actions are performed. Thus, these additional elements are recited in a manner that represent no more than mere instructions to apply the judicial exceptions on a computer. See MPEP § 2106.05(f) and § 2106.04(d).
generating a hardware implementation of the training objective in the hardware computing device according to the sparsity pattern is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h).
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. The claims are not eligible subject matter.
Therefore, in examining elements as recited by the limitations individually and as an ordered combination, as a whole the independent claim limitations do not recite what have the courts have identified as “significantly more”.
Regarding claim 15
Claim 15 recites: A system, comprising: memory configured to store at least one of a training objective, a hardware cost, or a hardware implementation of the training objective; and
one or more processors operatively coupled to the memory, the one or more processors being configured to:
identify the training objective to be executable by a hardware computing device;
identify the hardware cost corresponding to a set of features of the hardware computing device;
apply the hardware cost to a neural network during training to achieve the training objective; generate via the training according to the applied hardware cost, a sparsity pattern in a set of layers of the neural network; and
generate the hardware implementation of the training objective in the hardware computing device according to the sparsity pattern
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Claim 15 is directed to an abstract idea, specifically, a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
identifying a training objective to be executable by a hardware computing device
The limitation above is broadly and reasonably interpreted as a mental concept that can practically be performed in the human mind or with pen and paper, including observation evaluations, judgments, and opinions. For example, a person could review information and identify what objective the neural network should accomplish, identifying that the objective is classification, prediction, or execution of a particular function. See MPEP § 2106.04(a)(2)(III).
identifying a hardware cost corresponding to a set of features of the hardware computing device
The limitation above is broadly and reasonably interpreted as a mental concept that can practically be performed in the human mind or with pen and paper, including observation evaluations, judgments, and opinions. For example, looking at hardware characteristics and identifying an associated latency, power, logic-resource, or timing cost—that is an evaluation of information and falls under a mental process. See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
Independent claim 15 recites in part:
A system, comprising: memory configured to store at least one of a training objective, a hardware cost, or a hardware implementation of the training objective is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g).
one or more processors operatively coupled to the memory, the one or more processors being configured to: is reciting generic computing components. Such generic computing components are recited at a high-level of generality (i.e., as a generic processor performing data gathering and mathematical calculations) such that they amount to no more than mere instructions to apply the exception using generic computer components. Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
apply the hardware cost to a neural network during training to achieve the training objective as drafted, amount to the judicial exception is not integrated into a practical application. In particular, the claims only recites generic computing components. Such generic computing components are recited at a high-level of generality (i.e., as a generic processor performing data gathering and mathematical calculations) such that they amount to no more than mere instructions to apply the exception using generic computer components.
generate via the training according to the applied hardware cost, a sparsity pattern in a set of layers of the neural network is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception and reciting only the idea of a solution or outcome, i.e., the claim fails to recite details of how a solution to a problem is accomplished because it is unclear how the “NN” is used nor the specification makes it clear how these actions are performed. Thus, these additional elements are recited in a manner that represent no more than mere instructions to apply the judicial exceptions on a computer. See MPEP § 2106.05(f) and § 2106.04(d).
generate the hardware implementation of the training objective in the hardware computing device according to the sparsity pattern is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h).
Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea when considered as an ordered combination and as a whole.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
First, the additional elements directed to generally linking the use of a judicial exception to a particular technological environment or field of use are deemed insufficient to transform the judicial exception to a patentable invention because the claimed limitations generally link the judicial exception to the technology environment, see MPEP 2106.05(h). However, they are included below for the sake of completeness.
Second, the additional elements mere application of the abstract idea or mere instructions to implement an abstract idea on a computer are deemed insufficient to transform the judicial exception to a patentable invention because the limitations generally apply the use of a generic computer and/or process with the judicial exception. See MPEP 2106.05(f). However, they are included below for the sake of completeness.
Independent claim 15 recites in part:
A system, comprising: memory configured to store at least one of a training objective, a hardware cost, or a hardware implementation of the training objective is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g).
one or more processors operatively coupled to the memory, the one or more processors being configured to: is reciting generic computing components. Such generic computing components are recited at a high-level of generality (i.e., as a generic processor performing data gathering and mathematical calculations) such that they amount to no more than mere instructions to apply the exception using generic computer components. Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
apply the hardware cost to a neural network during training to achieve the training objective as drafted, amount to the judicial exception is not integrated into a practical application. In particular, the claims only recites generic computing components. Such generic computing components are recited at a high-level of generality (i.e., as a generic processor performing data gathering and mathematical calculations) such that they amount to no more than mere instructions to apply the exception using generic computer components.
generate via the training according to the applied hardware cost, a sparsity pattern in a set of layers of the neural network is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception and reciting only the idea of a solution or outcome, i.e., the claim fails to recite details of how a solution to a problem is accomplished because it is unclear how the “NN” is used nor the specification makes it clear how these actions are performed. Thus, these additional elements are recited in a manner that represent no more than mere instructions to apply the judicial exceptions on a computer. See MPEP § 2106.05(f) and § 2106.04(d).
generate the hardware implementation of the training objective in the hardware computing device according to the sparsity pattern is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h).
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. The claims are not eligible subject matter.
Therefore, in examining elements as recited by the limitations individually and as an ordered combination, as a whole the independent claim limitations do not recite what have the courts have identified as “significantly more”.
Regarding claim 20
Claim 20 recites: A non-transitory computer-readable medium having instructions stored thereon, the instructions, when executed by one or more processors of a computing system, performing a method comprising:
identifying a training objective to be executable by a hardware computing device;
identifying a hardware cost corresponding to a set of features of the hardware computing device;
applying the hardware cost to a neural network during training to achieve the training objective;
generating, via the training according to the applied hardware cost, a sparsity pattern in a set of layers of the neural network; and
generating a hardware implementation of the training objective in the hardware computing device according to the sparsity pattern
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Claim 20 is directed to an abstract idea, specifically, a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
identifying a training objective to be executable by a hardware computing device
The limitation above is broadly and reasonably interpreted as a mental concept that can practically be performed in the human mind or with pen and paper, including observation evaluations, judgments, and opinions. For example, a person could review information and identify what objective the neural network should accomplish, identifying that the objective is classification, prediction, or execution of a particular function. See MPEP § 2106.04(a)(2)(III).
identifying a hardware cost corresponding to a set of features of the hardware computing device
The limitation above is broadly and reasonably interpreted as a mental concept that can practically be performed in the human mind or with pen and paper, including observation evaluations, judgments, and opinions. For example, looking at hardware characteristics and identifying an associated latency, power, logic-resource, or timing cost—that is an evaluation of information and falls under a mental process. See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
Independent claim 20 recites in part:
A non-transitory computer-readable medium having instructions stored thereon, the instructions, when executed by one or more processors of a computing system, performing a method comprising, reciting generic computing components. Such generic computing components are recited at a high-level of generality (i.e., as a generic processor performing data gathering and mathematical calculations) such that they amount to no more than mere instructions to apply the exception using generic computer components. Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
apply the hardware cost to a neural network during training to achieve the training objective as drafted, amount to the judicial exception is not integrated into a practical application. In particular, the claims only recites generic computing components. Such generic computing components are recited at a high-level of generality (i.e., as a generic processor performing data gathering and mathematical calculations) such that they amount to no more than mere instructions to apply the exception using generic computer components.
generate via the training according to the applied hardware cost, a sparsity pattern in a set of layers of the neural network is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception and reciting only the idea of a solution or outcome, i.e., the claim fails to recite details of how a solution to a problem is accomplished because it is unclear how the “NN” is used nor the specification makes it clear how these actions are performed. Thus, these additional elements are recited in a manner that represent no more than mere instructions to apply the judicial exceptions on a computer. See MPEP § 2106.05(f) and § 2106.04(d).
generate the hardware implementation of the training objective in the hardware computing device according to the sparsity pattern is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h).
Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea when considered as an ordered combination and as a whole.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
First, the additional elements directed to generally linking the use of a judicial exception to a particular technological environment or field of use are deemed insufficient to transform the judicial exception to a patentable invention because the claimed limitations generally link the judicial exception to the technology environment, see MPEP 2106.05(h). However, they are included below for the sake of completeness.
Second, the additional elements mere application of the abstract idea or mere instructions to implement an abstract idea on a computer are deemed insufficient to transform the judicial exception to a patentable invention because the limitations generally apply the use of a generic computer and/or process with the judicial exception. See MPEP 2106.05(f). However, they are included below for the sake of completeness.
Independent claim 20 recites in part:
A non-transitory computer-readable medium having instructions stored thereon, the instructions, when executed by one or more processors of a computing system, performing a method comprising, reciting generic computing components. Such generic computing components are recited at a high-level of generality (i.e., as a generic processor performing data gathering and mathematical calculations) such that they amount to no more than mere instructions to apply the exception using generic computer components. Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
apply the hardware cost to a neural network during training to achieve the training objective as drafted, amount to the judicial exception is not integrated into a practical application. In particular, the claims only recites generic computing components. Such generic computing components are recited at a high-level of generality (i.e., as a generic processor performing data gathering and mathematical calculations) such that they amount to no more than mere instructions to apply the exception using generic computer components.
generate via the training according to the applied hardware cost, a sparsity pattern in a set of layers of the neural network is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception and reciting only the idea of a solution or outcome, i.e., the claim fails to recite details of how a solution to a problem is accomplished because it is unclear how the “NN” is used nor the specification makes it clear how these actions are performed. Thus, these additional elements are recited in a manner that represent no more than mere instructions to apply the judicial exceptions on a computer. See MPEP § 2106.05(f) and § 2106.04(d).
generate the hardware implementation of the training objective in the hardware computing device according to the sparsity pattern is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h).
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. The claims are not eligible subject matter.
Therefore, in examining elements as recited by the limitations individually and as an ordered combination, as a whole the independent claim limitations do not recite what have the courts have identified as “significantly more”.
Furthermore, regarding dependent claims 2-14 which are dependent on claim 1 and claims 16-19 which are dependent on claim 15, the claims are directed to a judicial exception without significantly more as highlighted below in the claim limitations by evaluating the claim limitations under Step 2A and 2B:
Claims 2 and 16, which are dependent on claims 1 and 15 respectively, and include an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g).
Claim 3 is dependent on claim 2, and include additional elements recited in a manner that represent no more than mere instructions to apply the judicial exceptions on a computer. See MPEP § 2106.05(f) and § 2106.04(d).
Claim 4 is dependent on claim 2, and include additional elements recited in a manner that represent no more than mere instructions to apply the judicial exceptions on a computer. See MPEP § 2106.05(f) and § 2106.04(d).
Claim 5 is dependent on claim 5, and include an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g).
Claim 6 and 17 is dependent on claims 1 and 15, and include the abstract idea of a mathematical calculation, as directed to “a claim that recites a mathematical calculation, when the claim is given its broadest reasonable interpretation in light of the specification, will be considered as falling within the "mathematical concepts" grouping. A mathematical calculation is a mathematical operation (such as multiplication) or an act of calculating using mathematical methods to determine a variable or number”. See MPEP § 2106.04(a)(2)(I)(C).
Claim 7 is dependent on claim 6, and include the abstract idea of a mathematical calculation, as directed to “a claim that recites a mathematical calculation, when the claim is given its broadest reasonable interpretation in light of the specification, will be considered as falling within the "mathematical concepts" grouping. A mathematical calculation is a mathematical operation (such as multiplication) or an act of calculating using mathematical methods to determine a variable or number”. See MPEP § 2106.04(a)(2)(I)(C).
Claims 8 and 18 are dependent on claims 1 and 15, and include the abstract idea of a mathematical calculation, as directed to “a claim that recites a mathematical calculation, when the claim is given its broadest reasonable interpretation in light of the specification, will be considered as falling within the "mathematical concepts" grouping. A mathematical calculation is a mathematical operation (such as multiplication) or an act of calculating using mathematical methods to determine a variable or number”. See MPEP § 2106.04(a)(2)(I)(C).
Claims 9 and 19 are dependent on claims 1 and 15, and include additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(h).
Claim 10 is dependent on claim 9, and include additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(h).
Claim 11 is dependent on claim 10, and include an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g).
Claim 12 is dependent on claim 1, and include additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(h).
Claim 13 is dependent on claim 1, and include additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(h).
Claim 14 is dependent on claim 1, and include additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(h).
The dependent claims as analyzed above, do not recite limitations that integrated the judicial exception into a practical application. In addition, the claim limitations do not include additional elements that are sufficient to amount to significantly more than the judicial exception (Step 2B). Therefore, the claims do not recite any limitations, when considered individually or as a whole, that recite what the courts have identified as “significantly more”, see MPEP 2106.05; and therefore, as a whole the claims are not patent eligible.
As shown above, the dependent claims do not provide any additional elements that when considered individually or as an ordered combination, amount to significantly more than the abstract idea identified. Therefore, as a whole the dependent claims do not recite what the courts have identified as “significantly more” than the recited judicial exception.
Therefore, claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception and does not recite, when claim elements are examined individually and as a whole, elements that the courts have identified as “significantly more” than the recited judicial exception.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-3, 12-13, 15-16 and 20 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Xu et al. (US Pub No.: 20200401891 A1), hereinafter referred to as Xu.
With respect to claim 1, Xu discloses:
A method, comprising: identifying a training objective to be executable by a hardware computing device (In paragraph [0072], Xu discloses determining a hardware configuration of a target hardware platform on which the machine learning model is to be executed on a target hardware platform, including CPU, GPU, DSP, VPU, or other acceleration resource.)
Identifying a hardware cost corresponding to a set of features of the hardware computing device (In paragraph [0076], Xu discloses a hardware cost corresponding to features of a hardware computing device by determining a number of execution cycles associated with processing neural-network layers on a target hardware platform. The number of cycles is determined based on the configuration of the target hardware platform and represents the computational hardware resources required to execute the neural-network layers. Xu discloses the number of cycles in the training/reward process and adjusts layer-specific sparsity ratios to achieve a target reduction in execution cycles while maintaining model accuracy. Accordingly, the number of execution cycles reasonably corresponds to the claimed hardware cost under the broadest reasonable interpretation.)
Applying, by one or more processors, the hardware cost to a neural network during training to achieve the training objective (In paragraph [0079], Xu discloses how many cycles are required to process neural-network layers on the target hardware platform. It then uses that hardware-dependent cycle information inside the neural-network training/optimization process. Furthermore, Xu discloses incorporating the number of cycles into a reward used during training, wherein the reward is based on model accuracy and the total number of cycles required to execute the layers having assigned sparsity ratios. )
Generating, via the training according to the applied hardware cost, a sparsity pattern in a set of layers of the neural network (In paragraph [0137], Xu discloses generating different sparsity configurations for multiple neural-network layers during hardware-aware training. Furthermore, Xu discloses a model training controller configured to perform reinforcement learning to determine respective sparsity configurations for a plurality of layers of a neural network based on a hardware configuration of a target hardware platform.)
Generating a hardware implementation of the training objective in the hardware computing device according to the sparsity pattern (In paragraph [0165], Xu discloses generating hardware implementation of the training objective in the hardware computing device according to the sparsity pattern by deploying the trained machine-learning model having the assigned layer-specific sparsity configuration to a target hardware platform. Furthermore, Xu disclosed the deployment controller deploys the final sparse model to one or more target hardware platforms, such as a CPU or acceleration resource, after determining that the model having the assigned sparsity configurations satisfies applicable accuracy and clock-cycle thresholds. Thus, the trained neural-network model having the hardware sparsity pattern is provided to and executed by the target hardware platform, thereby providing a hardware implementation according to the generated sparsity pattern. )
Regarding claim 2, Xu disclose elements of claim 1. In addition, Xu disclose:
The method of claim 1, wherein the sparsity pattern is generated in one or more layers of the set of layers of the neural network based on adjustment of weights or biases in the one or more layers (In paragraph [0080], Xu discloses that the sparsity pattern is generated in one or more layers of the set of layers of the neural network based on adjustment of weights in the one or more layers, by teaching a model training controller configured to perform weight pruning on respective layers of a neural network.)
Regarding claim 3, Xu disclose elements of claim 2. In addition, Xu disclose:
The method of claim 2, wherein the adjustment includes pruning one or more of the weights in the one or more layers (In paragraph [0080], Xu discloses that the sparsity pattern is generated in one or more layers of the set of layers of the neural network based on adjustment of weights in the one or more layers.)
Regarding claim 12, Xu disclose elements of claim 1. In addition, Xu disclose:
The method of claim 1, wherein the hardware computing device is a field-programmable gate array (FPGA) device (In paragraph [0047], Xu discloses that the general-purpose processing resource 112 may be a different type of hardware (e.g., a field-programmable gate array (FPGA)).)
Regarding claim 13, Xu disclose elements of claim 1. In addition, Xu disclose:
The method of claim 1, wherein the hardware computing device is an application-specific integrated circuit (ASIC) device (In paragraph [0047], Xu disclose the general-purpose processing resource 112 may be a different type of hardware such as a digital signal processor (DSP), an application specific integrated circuit (ASIC))
With respect to claim 15, Xu disclose:
A system, comprising: memory configured to store at least one of a training objective, a hardware cost, or a hardware implementation of the training objective (In paragraph [0050], Xu disclose a datastore 120 that stores machine-learning models and hardware configurations. Also, Xu disclose deployment controller 260 stores the trained/final sparse machine-learning model in datastore 120. That model is the result of hardware aware training and is subsequently deployable to the target hardware platform.)
One or more processors operatively coupled to the memory, the one or more processors being configured to: identify the training objective to be executable by a hardware computing device (In paragraph [0072], Xu discloses determining a hardware configuration of a target hardware platform on which the machine learning model is to be executed on a target hardware platform, including CPU, GPU, DSP, VPU, or other acceleration resource.)
Identify the hardware cost corresponding to a set of features of the hardware computing device (In paragraph [0076], Xu discloses a hardware cost corresponding to features of a hardware computing device by determining a number of execution cycles associated with processing neural-network layers on a target hardware platform. The number of cycles is determined based on the configuration of the target hardware platform and represents the computational hardware resources required to execute the neural-network layers. Xu discloses the number of cycles in the training/reward process and adjusts layer-specific sparsity ratios to achieve a target reduction in execution cycles while maintaining model accuracy. Accordingly, the number of execution cycles reasonably corresponds to the claimed hardware cost under the broadest reasonable interpretation.)
Apply the hardware cost to a neural network during training to achieve the training objective (In paragraph [0079], Xu discloses how many cycles are required to process neural-network layers on the target hardware platform. It then uses that hardware-dependent cycle information inside the neural-network training/optimization process. Furthermore, Xu discloses incorporating the number of cycles into a reward used during training, wherein the reward is based on model accuracy and the total number of cycles required to execute the layers having assigned sparsity ratios. )
Generate via the training according to the applied hardware cost, a sparsity pattern in a set of layers of the neural network (In paragraph [0137], Xu discloses generating different sparsity configurations for multiple neural-network layers during hardware-aware training. Furthermore, Xu discloses a model training controller configured to perform reinforcement learning to determine respective sparsity configurations for a plurality of layers of a neural network based on a hardware configuration of a target hardware platform.)
Generate the hardware implementation of the training objective in the hardware computing device according to the sparsity pattern (In paragraph [0165], Xu discloses generating hardware implementation of the training objective in the hardware computing device according to the sparsity pattern by deploying the trained machine-learning model having the assigned layer-specific sparsity configuration to a target hardware platform. Furthermore, Xu disclosed the deployment controller deploys the final sparse model to one or more target hardware platforms, such as a CPU or acceleration resource, after determining that the model having the assigned sparsity configurations satisfies applicable accuracy and clock-cycle thresholds. Thus, the trained neural-network model having the hardware sparsity pattern is provided to and executed by the target hardware platform, thereby providing a hardware implementation according to the generated sparsity pattern. )
Regarding claim 16, Xu disclose elements of claim 1. In addition, Xu disclose:
The system of claim 15, wherein the sparsity pattern is generated in one or more layers of the set of layers of the neural network based on adjustment of weights or biases in the one or more layers (In paragraph [0080], Xu discloses that the sparsity pattern is generated in one or more layers of the set of layers of the neural network based on adjustment of weights in the one or more layers, by teaching a model training controller configured to perform weight pruning on respective layers of a neural network.)
With respect to claim 20, Xu disclose:
A non-transitory computer-readable medium having instructions stored thereon, the instructions, when executed by one or more processors of a computing system, performing a method comprising: identifying a training objective to be executable by a hardware computing device (In paragraph [0072], Xu discloses determining a hardware configuration of a target hardware platform on which the machine learning model is to be executed on a target hardware platform, including CPU, GPU, DSP, VPU, or other acceleration resource.)
Identifying a hardware cost corresponding to a set of features of the hardware computing device (In paragraph [0076], Xu discloses a hardware cost corresponding to features of a hardware computing device by determining a number of execution cycles associated with processing neural-network layers on a target hardware platform. The number of cycles is determined based on the configuration of the target hardware platform and represents the computational hardware resources required to execute the neural-network layers. Xu discloses the number of cycles in the training/reward process and adjusts layer-specific sparsity ratios to achieve a target reduction in execution cycles while maintaining model accuracy. Accordingly, the number of execution cycles reasonably corresponds to the claimed hardware cost under the broadest reasonable interpretation.)
Applying the hardware cost to a neural network during training to achieve the training objective (In paragraph [0079], Xu discloses how many cycles are required to process neural-network layers on the target hardware platform. It then uses that hardware-dependent cycle information inside the neural-network training/optimization process. Furthermore, Xu discloses incorporating the number of cycles into a reward used during training, wherein the reward is based on model accuracy and the total number of cycles required to execute the layers having assigned sparsity ratios. )
Generating, via the training according to the applied hardware cost, a sparsity pattern in a set of layers of the neural network (In paragraph [0137], Xu discloses generating different sparsity configurations for multiple neural-network layers during hardware-aware training. Furthermore, Xu discloses a model training controller configured to perform reinforcement learning to determine respective sparsity configurations for a plurality of layers of a neural network based on a hardware configuration of a target hardware platform.)
Generating a hardware implementation of the training objective in the hardware computing device according to the sparsity pattern (In paragraph [0165], Xu discloses generating hardware implementation of the training objective in the hardware computing device according to the sparsity pattern by deploying the trained machine-learning model having the assigned layer-specific sparsity configuration to a target hardware platform. Furthermore, Xu disclosed the deployment controller deploys the final sparse model to one or more target hardware platforms, such as a CPU or acceleration resource, after determining that the model having the assigned sparsity configurations satisfies applicable accuracy and clock-cycle thresholds. Thus, the trained neural-network model having the hardware sparsity pattern is provided to and executed by the target hardware platform, thereby providing a hardware implementation according to the generated sparsity pattern. )
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 4-5 are rejected under 35 U.S.C. 103 as being unpatentable over Xu , in view of HASSANTABAR et al. (US Pub No.: 20220036150 A1), hereinafter referred to as HASSANTABAR.
Regarding claim 4, Xu discloses elements of claim 2. Xu does not explicitly disclose:
The method of claim 2, wherein the adjustment further includes pruning routes in the one or more layers
However, HASSANTABAR discloses the limitation (In paragraph [0066], HASSANTABAR discloses pruning connections. )
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention having the teachings of Xu to include HASSANTABAR, with pruning connections until a desired neural network architecture has been generated as taught by HASSANTABAR. The motivation for doing so would have been to improve the performance of machine learning models (See [0035] of HASSANTABAR.)
Regarding claim 5, Xu in view of HASSANTABAR discloses elements of claim 4. In addition, HASSANTABAR disclose:
The method of claim 4, further comprising varying a pruning threshold for pruning the routes (In paragraph [0066], HASSANTABAR discloses connections associated with a neuron being pruned, then the neuron is removed from the network. As shown in FIG. 7, one method for pruning connections is to remove connections of a small magnitude. Small magnitude is based on a predefined threshold. The rationale behind it is that since small weights have a relatively small influence on the network, ANN performance can be restored through retraining after pruning.)
Claims 6 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Xu , in view of Chilkunda et al. (US Patent No. 12,670,393 B1),hereinafter referred to as Chilkunda.
Regarding claim 6, Xu disclose elements of claim 1. Xu does not explicitly disclose:
The method of claim 1, wherein the sparsity pattern is generated by training a loss function that accounts for a prediction loss and the applied hardware cost
However, Chilkunda disclose the limitation (In Cols. 9-10, lines 19–11, Chilkunda discloses a stochastic super-network to determine the optimal sparsity level for each neural-network layer based on a cost/loss function.)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention having the teachings of Xu to include Chilkunda, with a sparsity of each of the candidate blocks in a respective layer is different as taught by Chilkunda. The motivation for doing so would have been to improve performance on edge devices without significant loss of accuracy (Col. 1, line 40, of Chilkunda).
Regarding claim 17, Xu disclose elements of claim 15. Xu does not explicitly disclose:
The system of claim 15, wherein the sparsity pattern is generated by training a loss function that accounts for a prediction loss and the applied hardware cost
However, Chilkunda disclose the limitation (In Cols. 9-10, lines 19–11, Chilkunda discloses a stochastic super-network to determine the optimal sparsity level for each neural-network layer based on a cost/loss function.)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention having the teachings of Xu to include Chilkunda, with a sparsity of each of the candidate blocks in a respective layer is different as taught by Chilkunda. The motivation for doing so would have been to improve performance on edge devices without significant loss of accuracy (Col. 1, line 40, of Chilkunda).
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Xu , in view of Chilkunda and further in view of Vahdat et al. (US Pub No.: 20210073612 A1), hereinafter referred to as Vahdat.
Regarding claim 7, Xu in view of Chilkunda disclose elements of claim 6. Xu in view of Chilkunda does not explicitly disclose:
The method of claim 6, wherein training the loss function is performed by applying a gradient descent approach to minimize the loss function
However, Vahdat disclose the limitation (In paragraph [0071], Vahdat discloses training the loss function by applying a gradient descent approach to minimize the loss function, by teaching determination of a neural network architecture using a gradient estimate of a loss function and performing gradient descent update during parameter updates.)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention having the teachings of Xu in view of Chilkunda to include Vahdat, with using non-differentiable loss functions as taught by Vahdat. The motivation for doing so would have been to minimize a loss function measuring performance of architecture specified by z using a performance measure such as classification loss (See [0067] of Vahdat.)
Claims 8, 14 and 18 is rejected under 35 U.S.C. 103 as being unpatentable over Xu , in view of Lee et al. (US Pub No.: 20230118505 A1), hereinafter referred to as Lee.
Regarding claim 8, Xu in disclose elements of claim 1. Xu does not explicitly disclose:
The method of claim 1, wherein the sparsity pattern is generated by training a loss function to account for the hardware cost
However, Lee disclose the limitation (In paragraph [0062], Lee discloses that performing an operation on the non-linear function, the neural network operation apparatus 10 may reduce hardware cost and latency by using an LUT and quantization.)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention having the teachings of Xu to include Lee, with a hardware cost and a latency by using an LUT and quantization as taught by Lee. The motivation for doing so would have been to increase in a number of LUT indexes may represent an increase in hardware cost (See [0135] of Lee).
Regarding claim 14, Xu in disclose elements of claim 1. Xu does not explicitly disclose:
The method of claim 1, wherein the training objective to be executable by the hardware computing device is a non-linear function
However, Lee disclose the limitation (In paragraph [0062], Lee discloses performing an operation on the non-linear function.)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention having the teachings of Xu to include Lee, with a hardware cost and a latency by using an LUT and quantization as taught by Lee. The motivation for doing so would have been to increase in a number of LUT indexes may represent an increase in hardware cost (See [0135] of Lee).
Regarding claim 18, Xu in disclose elements of claim 15. Xu does not explicitly disclose:
The system of claim 15, wherein the sparsity pattern is generated by training a loss function to account for the hardware cost
However, Lee disclose the limitation (In paragraph [0062], Lee discloses that performing an operation on the non-linear function, the neural network operation apparatus 10 may reduce hardware cost and latency by using an LUT and quantization.)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention having the teachings of Xu to include Lee, with a hardware cost and a latency by using an LUT and quantization as taught by Lee. The motivation for doing so would have been to increase in a number of LUT indexes may represent an increase in hardware cost (See [0135] of Lee).
Claims 9-10 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Xu , in view of Tseng et al. (US Pub No.: 20020152060 A1), hereinafter referred to as Tseng.
Regarding claim 9, Xu disclose elements of claim 1. Xu does not explicitly disclose:
The method of claim 1, wherein the hardware cost includes at least one of a logic cost or a set of spatiotemporal costs
However, Tseng disclose the limitation (In paragraph [0268], Tseng disclosed that G represents gate usage/availability of the FPGA hardware. The reference determines a cost based on how many hardware gates are required versus how many gates are available in the FPGA.)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention having the teachings of Xu to include Tseng, with communication system that transfers signals across FPGA chip boundaries only when these signals change values as taught by Tseng. The motivation for doing so would have been to increase the number of direct connections between FPGA logic devices (See [0594] of Tseng.)
Regarding claim 10, Xu in view of Tseng disclose elements of claim 9. In addition, Tseng disclose:
The method of claim 9, wherein the set of spatiotemporal costs includes at least one of a placement cost or a routing cost (In paragraph [0267], Tseng discloses the placement cost, which is calculated through a cost function f(P, G, D) for two or more circuits.)
Regarding claim 19, Xu disclose elements of claim 15. Xu does not explicitly disclose:
The system of claim 15, wherein the hardware cost includes at least one of a logic cost or a set of spatiotemporal costs
However, Tseng disclose the limitation (In paragraph [0268], Tseng disclosed that G represents gate usage/availability of the FPGA hardware. The reference determines a cost based on how many hardware gates are required versus how many gates are available in the FPGA.)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention having the teachings of Xu to include Tseng, with communication system that transfers signals across FPGA chip boundaries only when these signals change values as taught by Tseng. The motivation for doing so would have been to increase the number of direct connections between FPGA logic devices (See [0594] of Tseng.)
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Xu , in view of Tseng and further in view of Milton et al. (US Patent No. 7,555,741 B1), hereinafter referred to as Milton.
Regarding claim 11, Xu in view of Tseng disclose elements of claim 10. Xu in view of Tseng does not explicitly disclose:
The method of claim 10, wherein the at least one of the placement cost or the routing cost includes one or more factors including area, timing, or power
However, Milton disclose the limitation (In col. 20, lines 52-57, Milton discloses the overall clock routing cost for a given user clock grows with the actual power consumed by the routing for that user clock.)
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention having the teachings of Xu in view of Tseng, with producing logic designs for programmable logic devices that exhibit reduced power consumption as taught by Milton. The motivation for doing so would have been to improve performance optimization (e.g. increasing routability or improving timing) and the goals of power reduction (See (Col.24, lines 4-5) of Milton.)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to EVEL HONORE whose telephone number is (703)756-1179. The examiner can normally be reached Monday-Friday 8 a.m. -5:30 p.m.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Mariela D Reyes can be reached at (571) 270-1006. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
EVEL HONORE
Examiner
Art Unit 2142
/Mariela Reyes/Supervisory Patent Examiner, Art Unit 2142
