DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This office action is in response to the application filed on 05/08/2023.
Claims 1-15 are currently pending in this application.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Step 1 analysis for all claims:
In the instant case, claims 1-10 are directed to software per se. Claims 11-15 are directed to a method for compressing a neural network (process). Thus, each of the claims falls within one of the four statutory categories (i.e., process, machine, manufacture, or composition of matter).
Claim 1:
Step 2A, Prong 1 analysis:
The claim(s) recite(s) in part: “Selects an algorithm for compressing a neural network” As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, this limitation encompasses selecting an algorithm, which could be done mentally by a person having ordinary skill of the art.
“Divides the neural network into subgraphs;” As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, this limitation encompasses partitioning a neural network into subgraphs. This process can be done mentally by a person having ordinary skill in the art, who would be able divide a neural network in a plurality of ways.
“And an optimizing section which outputs a compression configuration in which one compression technique selected from a plurality thereof is associated with each of the subgraphs.” As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, this limitation encompasses outputting a compression technique that has been selected from a group of compression techniques, which has been associated with a subgraph.
Step 2A, Prong 2 analysis:
“information processing device” which is recited at a high level of generality and amount to adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). As explained by the Supreme Court; in order to make a claim directed to a judicial exception patent-eligible, the additional element or combination of elements must do '"more than simply state the judicial exception] while adding the words 'apply it"'. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, limitations that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not amount to significantly more than the exception itself and do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.05(g).
The judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of: “a subgraph dividing section” This limitation is recited at a high level of generality and amount to adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). As explained by the Supreme Court; in order to make a claim directed to a judicia I exception patent-eligible, the additional element or combination of elements must do '"more than simply state the judicial exception] while adding the words 'apply it"'. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, limitations that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not amount to significantly more than the exception itself and do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.05(f).
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Step 2B analysis:
“An information processing device” This limitation is recited at a high level of generality and amount to adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). As explained by the Supreme Court; in order to make a claim directed to a judicia I exception patent-eligible, the additional element or combination of elements must do '"more than simply state the judicial exception] while adding the words 'apply it"'. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, limitations that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not amount to significantly more than the exception itself and do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.05(g).
“a subgraph dividing section “This limitation is recited at a high level of generality and amount to adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). As explained by the Supreme Court; in order to make a claim directed to a judicia I exception patent-eligible, the additional element or combination of elements must do '"more than simply state the judicial exception] while adding the words 'apply it"'. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, limitations that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not amount to significantly more than the exception itself and do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.05(g).
Accordingly, at Step 2B the additional elements individually or in combination do not amount to significantly more than the judicial exception.
Claim 2:
Step 2A, Prong 1 analysis:
The claim(s) recite(s) in part:
“…wherein the subgraph dividing section divides the neural network having a hierarchical structure in units of layers.” As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, this limitation encompasses dividing the neural network into a hierarchical structure. This can be done mentally by a person having ordinary skill of the art utilizing a computing device.
Step 2A, Prong 2 analysis:
The judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
“Subgraph dividing section” This limitation is recited at a high level of generality and amount to adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). As explained by the Supreme Court; in order to make a claim directed to a judicia I exception patent-eligible, the additional element or combination of elements must do '"more than simply state the judicial exception] while adding the words 'apply it"'. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, limitations that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not amount to significantly more than the exception itself and do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.05(g).
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Step 2B analysis:
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional elements of: “a subgraph dividing section “This limitation is recited at a high level of generality and amount to adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). As explained by the Supreme Court; in order to make a claim directed to a judicia I exception patent-eligible, the additional element or combination of elements must do '"more than simply state the judicial exception] while adding the words 'apply it"'. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, limitations that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not amount to significantly more than the exception itself and do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.05(g).
Accordingly, at Step 2B the additional elements individually or in combination do not amount to significantly more than the judicial exception.
Claim 3:
Step 2A, Prong 1 analysis:
The claim(s) recite(s) in part:
“Outputs tentatively compressed subgraphs by compressing the subgraphs respectively associated with one compressing technique selected from a plurality thereof”. As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, this limitation encompasses outputting a compressed subgraph that has been associated with the chosen compression technique that was selected from a group of compressing techniques. This would be seen as a mental process since it would be obvious that a person having ordinary skill of the art would be able to output the compressed subgraph, See Mayo, 566 U.S. at 79, 101 USPQ2d at 1968. It would also be mentally possible to select a compression technique from a plurality thereof.
“the optimizing section outputs a compression configuration in which the perturbation of the neural network satisfies a predetermined value.” As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, this limitation encompasses outputting a compressed subgraph that has been associated with the chosen compression technique that was selected from a group of compressing techniques, that has a perturbation that satisfies a value. This would be seen as a mental process since it would be obvious that a person having ordinary skill of the art would be able to output a compression configuration in which the perturbation of a neural network would satisfy a value, See Mayo, 566 U.S. at 79, 101 USPQ2d at 1968.
“Based on a difference between a forward feeding value of the neural network and a forward feeding value obtained by combining a plurality of the tentatively compressed subgraphs in series”. As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, this limitation encompasses outputting a compressed subgraph that has been associated with the chosen compression technique that was selected from a group of compressing techniques. This would be seen as a mental process since it would be obvious that a person having ordinary skill of the art would be able to determine the difference between two values, based on the broadest reasonable interpretation of the limitation.
Step 2A, Prong 2 analysis:
The judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
“: a tentative compressing section” which is recited at a high level of generality and amount to adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). As explained by the Supreme Court; in order to make a claim directed to a judicial exception patent-eligible, the additional element or combination of elements must do '"more than simply state the judicial exception] while adding the words 'apply it"'. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, limitations that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not amount to significantly more than the exception itself and do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. See MPEP 2106.05(g).
“…and a first perturbation calculating section” which is recited at a high level of generality and amount to adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). As explained by the Supreme Court; in order to make a claim directed to a judicia I exception patent-eligible, the additional element or combination of elements must do '"more than simply state the judicial exception] while adding the words 'apply it"'. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, limitations that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not amount to significantly more than the exception itself and do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Step 2B analysis:
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional elements of:
“a tentative compressing section” This limitation is recited at a high level of generality and amount to adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). As explained by the Supreme Court; in order to make a claim directed to a judicia I exception patent-eligible, the additional element or combination of elements must do '"more than simply state the judicial exception] while adding the words 'apply it"'. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, limitations that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not amount to significantly more than the exception itself and do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
“And a first perturbation calculating section” This limitation is recited at a high level of generality and amount to adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). As explained by the Supreme Court; in order to make a claim directed to a judicia I exception patent-eligible, the additional element or combination of elements must do '"more than simply state the judicial exception] while adding the words 'apply it"'. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, limitations that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not amount to significantly more than the exception itself and do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Accordingly, at Step 2B the additional elements individually or in combination do not amount to significantly more than the judicial exception.
Claim 4:
Step 2A, Prong 1 analysis:
The claim(s) recite(s) in part:
“wherein the tentative compressing section refers to a compression table in Which a type of a device on which the neural network is to be implemented And priority of compression techniques to be applied are associated with each other And selects one compression method from a plurality thereof”. As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, this limitation encompasses implementing a compression technique on a priority basis and associating it with the device, as well as selecting a compression method from a plurality thereof. A person having ordinary skill of the art would be able to implement said compression technique and associate it with a device as described, as well as select a compression technique from a plurality thereof on a computing device.
Step 2A, Prong 2 analysis:
The judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
“The tentative compressing section” This limitation is recited at a high level of generality and amount to adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). As explained by the Supreme Court; in order to make a claim directed to a judicia I exception patent-eligible, the additional element or combination of elements must do '"more than simply state the judicial exception] while adding the words 'apply it"'. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, limitations that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not amount to significantly more than the exception itself and do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Step 2B analysis:
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional elements of:
“The tentative compressing section” This limitation is recited at a high level of generality and amount to adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). As explained by the Supreme Court; in order to make a claim directed to a judicia I exception patent-eligible, the additional element or combination of elements must do '"more than simply state the judicial exception] while adding the words 'apply it"'. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, limitations that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not amount to significantly more than the exception itself and do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Accordingly, at Step 2B the additional elements individually or in combination do not amount to significantly more than the judicial exception.
Claim 5:
Step 2A, Prong 1 analysis:
The claim(s) recite(s) in part:
“Outputs a perturbation of each of the subgraphs based on a difference between a forward feeding value of each of the subgraphs.” As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, this limitation encompasses outputting a perturbation of each subgraph based on the difference between values. This can be done mentally, by comparing the two values and outputting the perturbation. See Mayo, 566 U.S. at 79, 101 USPQ2d at 1968, and outputting each subgraphs perturbation accordingly.
“The subgraph dividing section selects a subgraph to be further divided based on perturbation of the subgraphs”. As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, this limitation encompasses selecting a subgraph based on the perturbation of said subgraph. This can be done mentally by comparing the perturbation of each subgraph.
Step 2A, Prong 2 analysis:
The judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
“Second perturbation calculating section” This limitation is recited at a high level of generality and amount to adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). As explained by the Supreme Court; in order to make a claim directed to a judicia I exception patent-eligible, the additional element or combination of elements must do '"more than simply state the judicial exception] while adding the words 'apply it"'. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, limitations that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not amount to significantly more than the exception itself and do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Step 2B analysis:
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional elements of:
“Second perturbation calculating section” This limitation is recited at a high level of generality and amount to adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). As explained by the Supreme Court; in order to make a claim directed to a judicia I exception patent-eligible, the additional element or combination of elements must do '"more than simply state the judicial exception] while adding the words 'apply it"'. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, limitations that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not amount to significantly more than the exception itself and do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Accordingly, at Step 2B the additional elements individually or in combination do not amount to significantly more than the judicial exception.
Claim 6:
Step 2A, Prong 1 analysis:
The claim(s) recite(s) in part:
“the subgraph dividing section selects a subgraph with a presently largest perturbation as a subgraph to be further divided.” As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, this limitation encompasses determining a subgraph with the largest perturbation that will then be further divided. This can be done mentally, by comparing the perturbations of the different subgraphs to determine the largest perturbation. See Mayo, 566 U.S. at 79, 101 USPQ2d at 1968, and dividing that subgraph.
Step 2A, Prong 2 analysis:
The judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
“Second perturbation calculating section” This limitation is recited at a high level of generality and amount to adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). As explained by the Supreme Court; in order to make a claim directed to a judicia I exception patent-eligible, the additional element or combination of elements must do '"more than simply state the judicial exception] while adding the words 'apply it"'. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, limitations that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not amount to significantly more than the exception itself and do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Step 2B analysis:
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional elements of:
“Second perturbation calculating section” This limitation is recited at a high level of generality and amount to adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). As explained by the Supreme Court; in order to make a claim directed to a judicia I exception patent-eligible, the additional element or combination of elements must do '"more than simply state the judicial exception] while adding the words 'apply it"'. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, limitations that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not amount to significantly more than the exception itself and do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Accordingly, at Step 2B the additional elements individually or in combination do not amount to significantly more than the judicial exception.
Claim 7:
Step 2A, Prong 1 analysis:
The claim(s) recite(s) in part:
“The information processing device according to claim 1, further comprising a compressing section, wherein the compressing section includes a computation reducing section and a subgraph combining section, the computation reducing section receives the subgraphs and the compression configuration as inputs, applies a compression technique described in the compression configuration to each of the subgraphs, and outputs compressed subgraphs, and” amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, (the compression section) merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f))
“the subgraph combining section receives an output of the computation reducing section as an input, combines the compressed subgraphs, and outputs a compressed neural network.” amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, (the compression section) merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f)).
Step 2A, Prong 2 analysis:
The judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
“Further comprising a compressing section” This limitation is recited at a high level of generality and amount to adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). As explained by the Supreme Court; in order to make a claim directed to a judicial exception patent-eligible, the additional element or combination of elements must do '"more than simply state the judicial exception] while adding the words 'apply it"'. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, limitations that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not amount to significantly more than the exception itself and do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
“a computation reducing section” This limitation is recited at a high level of generality and amount to adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). As explained by the Supreme Court; in order to make a claim directed to a judicia I exception patent-eligible, the additional element or combination of elements must do '"more than simply state the judicial exception] while adding the words 'apply it"'. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, limitations that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not amount to significantly more than the exception itself and do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
“And a subgraph combining section” This limitation is recited at a high level of generality and amount to adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). As explained by the Supreme Court; in order to make a claim directed to a judicia I exception patent-eligible, the additional element or combination of elements must do '"more than simply state the judicial exception] while adding the words 'apply it"'. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, limitations that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not amount to significantly more than the exception itself and do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Step 2B analysis:
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional elements of:
“Further comprising a compressing section” This limitation is recited at a high level of generality and amount to adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). As explained by the Supreme Court; in order to make a claim directed to a judicia I exception patent-eligible, the additional element or combination of elements must do '"more than simply state the judicial exception] while adding the words 'apply it"'. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, limitations that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not amount to significantly more than the exception itself and do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
“a computation reducing section” This limitation is recited at a high level of generality and amount to adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). As explained by the Supreme Court; in order to make a claim directed to a judicia I exception patent-eligible, the additional element or combination of elements must do '"more than simply state the judicial exception] while adding the words 'apply it"'. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, limitations that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not amount to significantly more than the exception itself and do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
“And a subgraph combining section” This limitation is recited at a high level of generality and amount to adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). As explained by the Supreme Court; in order to make a claim directed to a judicia I exception patent-eligible, the additional element or combination of elements must do '"more than simply state the judicial exception] while adding the words 'apply it"'. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, limitations that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not amount to significantly more than the exception itself and do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Accordingly, at Step 2B the additional elements individually or in combination do not amount to significantly more than the judicial exception.
Claim 8:
Step 2A, Prong 1 analysis:
The claim(s) recite(s) in part:
“Receives the compressed neural network and a training data set As discussed above, the additional elements of receiving a compressed neural network which is recited at a high level of generality and amounts to extra-solution activity of receiving data i.e. pre-solution activity of gathering data for use in the claimed process. The courts have found limitations directed to obtaining information electronically, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II), “receiving or transmitting data over a network”, "electronic record keeping," and "storing and retrieving information in memory").
“And trains the compressed neural network”. As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, this limitation encompasses training a compressed neural network, which can be done mentally by a person having ordinary skill of the art, utilizing a computing device.
Step 2A, Prong 2 analysis:
The judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
“comprising a retraining section” This limitation is recited at a high level of generality and amount to adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). As explained by the Supreme Court; in order to make a claim directed to a judicial exception patent-eligible, the additional element or combination of elements must do '"more than simply state the judicial exception] while adding the words 'apply it"'. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, limitations that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not amount to significantly more than the exception itself and do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Step 2B analysis:
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional elements of:
“comprising a retraining section” This limitation is recited at a high level of generality and amount to adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). As explained by the Supreme Court; in order to make a claim directed to a judicia I exception patent-eligible, the additional element or combination of elements must do '"more than simply state the judicial exception] while adding the words 'apply it"'. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, limitations that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not amount to significantly more than the exception itself and do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Accordingly, at Step 2B the additional elements individually or in combination do not amount to significantly more than the judicial exception.
Claim 9:
Step 2A, Prong 2 analysis:
The claim(s) recite(s) in part:
“the perturbation of the subgraphs is held in memory. “As discussed above, the additional elements of storing a perturbation in memory which is recited at a high level of generality and amounts to extra-solution activity of receiving data i.e. pre-solution activity of gathering data for use in the claimed process. The courts have found limitations directed to obtaining information electronically, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II),"electronic record keeping," and "storing and retrieving information in memory. Storing data in memory is an abstract idea per MPEP 2106.04.III.B, see CyberSource 654 F.3d at 1372-73, 99 USPQ2d at 1695.
The judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
“perturbation log” This limitation is recited at a high level of generality and amount to adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). As explained by the Supreme Court; in order to make a claim directed to a judicia I exception patent-eligible, the additional element or combination of elements must do '"more than simply state the judicial exception] while adding the words 'apply it"'. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, limitations that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not amount to significantly more than the exception itself and do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Step 2B analysis:
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional elements of:
“a perturbation log” This limitation is recited at a high level of generality and amount to adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). As explained by the Supreme Court; in order to make a claim directed to a judicia I exception patent-eligible, the additional element or combination of elements must do '"more than simply state the judicial exception] while adding the words 'apply it"'. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, limitations that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not amount to significantly more than the exception itself and do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Accordingly, at Step 2B the additional elements individually or in combination do not amount to significantly more than the judicial exception.
Claim 11:
Claim 11 recites similar limitations as claim 1, therefore it is rejected on the same basis.
Claim 12:
Claim 12 recites similar claims to claim 5, therefore it is rejected on the same basis.
Claim 13:
Claim 13 recites similar claims to claim 5, therefore it is rejected on the same basis.
Claim 14:
Step 2A, Prong 1 analysis:
The claim(s) recite(s) in part:
“And the neural network is stored at a root of the m-branch tree”. As discussed above, the additional elements of gathering the neural network which is recited at a high level of generality and amounts to extra-solution activity of receiving data i.e. pre-solution activity of gathering data for use in the claimed process. The courts have found limitations directed to obtaining information electronically, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II), “receiving or transmitting data over a network”, "electronic record keeping," and "storing and retrieving information in memory"). The claim recites the storing and gathering of data, as drafted under MPEP, such process is seen as an abstract idea. See MPEP 2106.05(1) (A. iii) and MPEP § 2106.05(g).
“The neural network stored at the root of the m-branch tree is divided into m”. As discussed above, the additional elements of gathering the neural network which is recited at a high level of generality and amounts to extra-solution activity of receiving data i.e. pre-solution activity of gathering data for use in the claimed process. The courts have found limitations directed to obtaining information electronically, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II), “receiving or transmitting data over a network”, "electronic record keeping," and "storing and retrieving information in memory"). The claim recites the storing and gathering of data, as drafted under MPEP, such process is seen as an abstract idea. See MPEP 2106.05(1) (A. iii) and MPEP § 2106.05(g).
“m subgraphs are stored at a node of depth 1 that is a child of the root”. As discussed above, the additional elements of gathering and storing subgraphs at the child root which is recited at a high level of generality and amounts to extra-solution activity of receiving data i.e. pre-solution activity of gathering data for use in the claimed process. The courts have found limitations directed to obtaining information electronically, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II), “receiving or transmitting data over a network”, "electronic record keeping," and "storing and retrieving information in memory"). The claim recites the storing and gathering of data, as drafted under MPEP, such process is seen as an abstract idea. See MPEP 2106.05(1) (A. iii) and MPEP § 2106.05(g).
“and the subgraphs stored at leaves of the m-branch tree are output”. As discussed above, the additional elements of gathering and storing the subgraphs which is recited at a high level of generality and amounts to extra-solution activity of receiving data i.e. pre-solution activity of gathering data for use in the claimed process. The courts have found limitations directed to obtaining information electronically, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II), “receiving or transmitting data over a network”, "electronic record keeping," and "storing and retrieving information in memory"). The claim recites the storing and gathering of data, as drafted under MPEP, such process is seen as an abstract idea. See MPEP 2106.05(1) (A. iii) and MPEP § 2106.05(g).
Step 2A, Prong 2 analysis:
The judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of: “m-branch tree” This limitation is recited at a high level of generality and amount to adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). As explained by the Supreme Court; in order to make a claim directed to a judicial exception patent-eligible, the additional element or combination of elements must do '"more than simply state the judicial exception] while adding the words 'apply it"'. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, limitations that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not amount to significantly more than the exception itself and do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
“Stored at a node” This limitation is recited at a high level of generality and amount to adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. As discussed above, the additional elements of storing data which is recited at a high level of generality and amounts to extra-solution activity of receiving data i.e. pre-solution activity of gathering data for use in the claimed process. The courts have found limitations directed to obtaining information electronically, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II), “receiving or transmitting data over a network”, "electronic record keeping," and "storing and retrieving information in memory").
“root” This limitation is recited at a high level of generality and amount to adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). As explained by the Supreme Court; in order to make a claim directed to a judicia I exception patent-eligible, the additional element or combination of elements must do '"more than simply state the judicial exception] while adding the words 'apply it"'. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, limitations that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not amount to significantly more than the exception itself and do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Step 2B analysis:
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional elements of:
“m-branch tree” This limitation is recited at a high level of generality and amount to adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). As explained by the Supreme Court; in order to make a claim directed to a judicia I exception patent-eligible, the additional element or combination of elements must do '"more than simply state the judicial exception] while adding the words 'apply it"'. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, limitations that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not amount to significantly more than the exception itself and do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
“stored at a node” This limitation is recited at a high level of generality and amount to adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). As explained by the Supreme Court; in order to make a claim directed to a judicia I exception patent-eligible, the additional element or combination of elements must do '"more than simply state the judicial exception] while adding the words 'apply it"'. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, limitations that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not amount to significantly more than the exception itself and do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
“root” This limitation is recited at a high level of generality and amount to adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. See MPEP 2106.05(f). As explained by the Supreme Court; in order to make a claim directed to a judicia I exception patent-eligible, the additional element or combination of elements must do '"more than simply state the judicial exception] while adding the words 'apply it"'. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, limitations that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not amount to significantly more than the exception itself and do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Accordingly, at Step 2B the additional elements individually or in combination do not amount to significantly more than the judicial exception.
Claim 15:
Claim 15 recites similar claims to claim 14, therefore it is rejected on the same basis.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1 and 11 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Zejda et Al. (US Patent #11610102, referred to as Zejda hereafter).
Claim 1
Zejda anticipates an information processing device that selects an algorithm for compressing a neural network, the information processing device comprising… (Zejda) [col.2, lines 16-28] "the neural network may, in various embodiments, be partitioned into different subgraphs, which may then be assigned to different processing units" "For example, operations may include convolution, pooling, normalization, and/or activation operations, in some embodiments."
As anticipated, Zejda teaches the use of a plurality of compression techniques to be chosen from.
a subgraph dividing section which divides the neural network into subgraphs (Zejda) [pg. 10, col.2, pp. 2, lines 25-27] “the neural network may, in various embodiments, be partitioned into different subgraphs,” As anticipated, Zejda teaches the division of a neural network into different subgraphs.
and an optimizing section which outputs a compression configuration in which one compression technique selected from a plurality thereof is associated with each of the subgraphs. (Zejda) [ col.2, lines 27-31]"the neural network may, in various embodiments, be partitioned into different subgraphs, which may then be assigned to different processing units" [pg. 15, col.11, pp. 4, lines 45-47] "For example, operations may include convolution, pooling, normalization, and/or activation operations, in some embodiments. anticipated, Zejda teaches the partitioning of a neural network into subgraphs. It would be obvious that said subgraph would be associated with one of the mentioned compression techniques mentioned by Zejda, as anticipated.
Claim 11
Zejda anticipates A neural network compression method comprising: a first step of dividing a neural network into subgraphs. (Zejda) [col.2, lines 25-27] “the neural network may, in various embodiments, be partitioned into different subgraphs,” As anticipated, Zejda teaches the division of a neural network into different subgraphs.
and a second step of performing tentative compression by associating one compression technique selected from a plurality thereof with each of the subgraphs. (Zejda) [pg. 10, col.2, pp. 2, lines 27-31]"the neural network may, in various embodiments, be partitioned into different subgraphs, which may then be assigned to different processing units" [pg. 15, col.11, pp. 4, lines 45-47] For example, operations may include convolution, pooling, normalization, and/or activation operations, in some embodiments. As anticipated, Zejda teaches the partitioning of a neural network into subgraphs by selecting from a plurality of compression techniques and assigning each subgraph to a different processing unit. It would be obvious that said subgraph would be associated with one of the mentioned compression techniques mentioned by Zejda, as anticipated.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Zejda as set forth above in view of Jin et al (US 20200082243 A1, referred to as Jin, hereinafter).
Regarding claim 2
Jin teaches:
The information processing device according to claim 1, wherein the subgraph dividing section divides the neural network having a hierarchical structure in units of layers. (Jin) [pg. 1, pp 4] “neural network, including at least one of partitioning a network into at least one subgraph node, determining a layer order” As anticipated, Jin teaches the partitioning of a subgraph and ordering them in layers, i.e. a hierarchical structure.
Before the effective filing date, it would have been obvious to one of ordinary skill of the art to combine the partitioning of a neural network of Zejda, with the use of hierarchal structure when dividing a neural network as taught by Jin. A person having ordinary skill of the art would be motivated to do so to determine the layer order of execution (Jin) [col.2, pp. 33] and to reduce the complexity of the layers (Jin) [col.1, pp. 3]
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Zejda et al., Jin et al., and Wang et al., and in further view of Kapoor et al (US 20140074805 A1, Kapoor hereinafter).
Regarding claim 4
Zejda, in addition to Vikram teach:
wherein the tentative compressing section refers to a compression table in which a type of a device on which the neural network is to be implemented and priority of compression techniques to be applied are associated with each other, and selects one compression method from a plurality thereof. (Zejda) [pg.18, col.17, pp.1] “At step 714, it is determined whether a size of subgraph operation weights (e.g., the subgraph operation weights 422) associated with the plurality of operations included in each of the set of subgraphs”. (Vikram) [pg.3, pp.46]” A table, or data from a plurality of rows thereof, is compressed into a "compression unit" using any of a wide variety of compression techniques.” As anticipated, Zejda in combination with Vikram teaches the use of multiple compression techniques, and a compression table in which a variety of compression techniques are applied, rendering claim 4 obvious.
Before the effective filing date, it would have been obvious to one of ordinary skill in the art to combine the process of associating subgraph weights with each compression technique, as taught by Zejda, with the compression table taught by Vikram. Such person would be motivated to do so, because compression units may thus co-exist with uncompressed data blocks and data block rows. (Vikram) [pg.3, col.2, pp.47]”
Claims 3, 5, 9, and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Zejda et Al, in view of Jin, in further view of Wang et al. (Hardware-Aware Autonomous Quantization, CVPR 2019, referred to as Wang hereinafter)., and in further view of Jiang et, al. (IJCAI-2018, Efficient DNN Neuron Pruning by Minimizing Layer-wise Nonlinear Reconstruction Error, Jiang Hereinafter).
Regarding claim 3
Zejda and Jin teach the limitations of claim 2, in addition Zejda teaches:
a tentative compressing section. (Zejda) [col. 11, pp.4] "As discussed herein, different partitioning schemes may be applied to identify different subgraphs of the neural network graph 400, such as subgraphs 416a-416n” Zejda teaches the use of multiple different partitioning or compression techniques as mentioned by claim 3.
and a first perturbation calculating section, wherein the tentative compressing section outputs tentatively compressed subgraphs by compressing the subgraphs respectively associated with one compressing technique selected from a plurality thereof. (Zejda) [col. 11, pp.4] "As discussed herein, different partitioning schemes may be applied to identify different subgraphs of the neural network graph 400, such as subgraphs 416a-416n” Zejda teaches the use of multiple different partitioning or compression techniques as mentioned by claim 3. The claim does not describe the tentatively compressed subgraphs are created, therefore under the broadest reasonable interpretation, subgraphs 416a-416n depict the tentatively compressed subgraphs.
the first perturbation calculating section outputs a perturbation of the neural network based on a difference between a forward feeding value of the neural network and a forward feeding value(Jiang) [p.2299 Section 2.2 – Equation (1)] “o^(l+1) – a (l+1)”. o^(l+1) is the non-linear output of the original (uncompressed) network, where a^(l+1) is the nonlinear output of the pruned (compressed) network. Here, Jiang describes the difference between forward feeding values of an uncompressed and compressed subgraph, the outcome of the equation is the perturbation of each subgraph. (Jiang) [Algorithm 1 – Step 7]” Forward propagation and compute E_l+1 by Eq. (7).” A perturbation is computed for each subgraph (layer) during iteration, thus teaching a “first perturbation calculating section”.
obtained by combining a plurality of the tentatively compressed subgraphs in series. (Zejda) [pg.16, col. 14, pp.4] “Accordingly, the compiler may select the second partitioning scheme (subgraph assignment 510-2) that exhibits the next lowest total time and also satisfies the maximum operation weight limit.”
However, the combination of Zejda and Jin fail to teach:
“And the optimizing section outputs a compression configuration in which the perturbation of the neural network satisfies a predetermined value.”
However, Wang teaches:
and the optimizing section (Wang)[pg. 3, section 3. Approach]the quantization task as a reinforcement learning problem (Figure 2). We use the actor-critic model with DDPG agent to give the action: bits for each layer. We collect hardware counters as constraints, together with accuracy as rewards to search the optimal quantization policy. Here, Wang teaches of an optimizing section in the form of a reinforcement learning agent configured to recursively search for an optimize a quantization policy based on performance feedback, thus rendering the limitation obvious.
outputs a compression configuration (Wang)[Pg. 2, 1. Introduction]“the agent… outputs the action which is the bitwidth of weights and activations.” Here, Wang discloses that the optimizing agent outputs a quantization policy specifying bitwidths for weights and activations of each layer, which constitutes a compression configuration of the neural network, thereby satisfying the claimed limitation.
in which the perturbation of the neural network satisfies a predetermined value. (Wang) [p.4, 3.2]” If the current policy exceeds our resource budget (on latency, energy or model size), we will sequentially decrease the bit width of each layer until the constraint is finally satisfied.” Wang teaches decreasing the bit width of each partitioned layer of a neural network, until a constraint is met. Similarly, the claim recites a perturbation of neural network satisfying a predetermined value, thus rendering claim 3 obvious.
Regarding claim 5
Zejda as modified by Wang and Vikram teach all of the limitations of claim 3, in addition to:
The information processing device according to claim 3, further comprising a second perturbation calculating section. (Zejda) [pg.16, col. 13, pp.1] Under the second partitioning scheme, the first subgraph 516a is assigned to a first accelerator of an execution engine and the second subgraph 516b is assigned to a second accelerator of the execution engine. The second accelerator may be different from the first accelerator. Zejda teaches a second partitioning scheme and accelerator engine.
Zejda, Wang, and Vikram fail to teach:
wherein the second perturbation calculating section outputs a perturbation of each of the subgraphs based on a difference between a forward feeding value of each of the subgraphs and a forward feeding value of each of the tentatively compressed subgraphs,
and the subgraph dividing section selects a subgraph to be further divided based on perturbation of the subgraphs.
However, Jiang teaches:
wherein the second perturbation calculating section outputs a perturbation of each of the subgraphs based on a difference between a forward feeding value of each of the subgraphs and a forward feeding value of each of the tentatively compressed subgraphs, (Jiang) [p.2299 Section 2.2 - Formulation]” We use a^(l) to denote nonlinear response in layer (l)”. Here, a^(l) is the forward feeding value (activation output) of each subgraph or in this case, layer. (Jiang) [Section 2.2 - Formulation]” A pruned model can be represented as {W^(l), m^(l)}, where m^(l) is a binary mask indicating whether a neuron is pruned.” Applying mask m^(l) creates a tentatively compressed subgraph. (Jiang) [p.2299 Section 2.2 – Equation (1)] “o^(l+1) – a (l+1)”. o^(l+1) is the non-linear output of the original (uncompressed) network, where a^(l+1) is the nonlinear output of the pruned (compressed) network. Here, Jiang describes the difference between forward feeding values of an uncompressed and compressed subgraph, the outcome of the equation is the perturbation of each subgraph. (Jiang) [Algorithm 1 – Step 7]” Forward propagation and compute E_l+1 by Eq. (7).” A perturbation is computed for each subgraph (layer) during iteration, thus teaching a “second perturbation calculating section”.
and the subgraph dividing section selects a subgraph to be further divided based on perturbation of the subgraphs. (Jiang) [Section 2.3 – Greedy Selection p. 3] “To measure the importance of neurons, we first analyze the sensitivity of NRE with respect to each neuron…neurons with the k_i largest sensitivity values are selected.” Here, sensitivity is derived from perturbations (NRE). Selection of a subgraph is based on the magnitude of each perturbation as taught in claim 5. (Jiang) [Algorithm 1- Iterative loop (p. 3)] “In the next iteration, the sensitivity of neurons will be recalculated and their masks will be updated.” Here, Jiang teaches reprocessing and refining the selected subgraphs, this anticipates the limitation “selects a subgraph to be further divided”.
Before the effective filing date, it would have been obvious to one of ordinary skill of the art to combine the quantization of Neural networks of Zejda, Wang and Vikram with the process of outputting a perturbation based on a subgraphs data, as taught by Jiang. A person having ordinary skill of the art would be motivated to do so, to select a subgraph for further iterative processing based on perturbation-derived sensitivity values (IJCAI-18, pp. 2-3 EQ (1), Algorithm 1).
Regarding Claim 9:
Zejda, as modified by Jin, Wang and Jiang, teach all of the limitations of claim 5, in addition Wang teaches:
The perturbation of the subgraphs is held in memory as a perturbation log. (Wang) [p.4, Section 3.3 – RL-Based Search] “After quantization, we retrain the quantized model for one more epoch, and feed the validation accuracy after short-term retraining as the reward signal.” Wang teaches evaluating tentative compression configurations by calculating performance differences between original and compressed networks and using these differences as reward signals retraining framework. It would be obvious that such frameworks store metrics such as perturbations of a subgraph in memory across iterations to guide policy updates and calculations. As stated, Wang teaches holding perturbation records in memory, rendering claim 9 obvious.
Before the effective filing date, it would have been obvious to one of ordinary skill in the art to combine the neural network compression technique of Zejda, Jin, and Jiang, and with storing of perturbation logs in memory of Wang to optimize the quantization of neural networks. A person having ordinary skill in the art would be motivated to do so, to efficiently implement quantization on to hardware. (Wang) [P.4 ,3.4 Quantization]
Regarding Claim 10:
Zejda, as modified by Jin, Wang, Jiang teach all of the limitations of claim 9, in addition to:
Transmission of a perturbation log out of a computing device. (Wang) [p.3 section 3. Approach] “We use direct latency and energy feedback from the hardware accelerator to guide the agent…”Wang teaches using direct latency to transmit information out of the hardware accelerator. It would be obvious that In such frameworks, performance records measured on target devices such as a log that stores the perturbations of a subgraph, are regularly transmitted to an external controller or optimization engine for policy updates. As mentioned, transmitting perturbation information outside the computing device is an obvious implementation detail, rendering claim 10 obvious.
Before the effective filing date, it would have been obvious to one of ordinary skill in the art to combine the neural network pruning of Zejda with the transmission of perturbation information of Wang to improve the method of quantizing a neural network. A person having ordinary skill of the art would be motivated to do so, to allow the agent to determine the bit width allocation policy from the differences between the plurality of layers, thus optimizing the quantization policy (Wang) [P.4 3.3 Direct Feedback from Hardware Accelerators]
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Zejda, in view of Jiang, and in further view of Deolalikar (US 20080180247 A1, Deolalikar hereinafter).
Regarding Claim 6:
Zejda, Jin, Wang, Vikram and Jiang teach the limitations of claim 5, except:
“The subgraph dividing section selects a subgraph with a presently largest perturbation as a subgraph to be further divided.”
However, Deolalikar teaches:
the subgraph dividing section selects a subgraph with a presently largest perturbation as a subgraph to be further divided. (Deolalikar) [pg.7, col.2, pp. 67]an edge-removal perturbation, an edge with a largest weight of the edges that interconnect nodes within the defect set is chosen for removal. Deolalikar teaches an edge removal system that removes the edge with the largest weight. Similarly, claim 6 teaches dividing the subgraph with the largest perturbation. The method of selecting the largest value or perturbation would be obvious to a person having ordinary skill of the art, rendering claim 6 obvious.
Before the effective filing date, it would have been obvious to one of ordinary skill of the art to combine the quantization process of algorithms of Zejda, Jin, Wang, Vikram and Jiang with the removal process of Deolalikar. Such person would be motivated to do so, to alter a collision graph to produce a lower-partite collision graph that achieves greater efficiency. (Deolalikar) [pg.7, col.1, pp. 65]
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Zejda, in further view of Jin, in further view of Wang and in further view of Roy (US 20140019490 A1, hereinafter Roy).
Regarding Claim 7:
Zejda teaches all of the limitations of claim 1 except:
further comprising a compressing section,
wherein the compressing section includes a computation reducing section and a subgraph combining section,
the computation reducing section receives the subgraphs and the compression configuration as inputs, applies a compression technique described in the compression configuration to each of the subgraphs,
and outputs compressed subgraphs,
and the subgraph combining section receives an output of the computation reducing section as an input,
combines the compressed subgraphs, and outputs a compressed neural network.
However, Roy in combination with Zejda and Wang teach:
further comprising a compressing section, (Roy)[pp. 0022]Graph views for event processing can extend the graph data store with dynamic partitioning and management mechanisms. Roy teaches dynamically partitioning subgraphs which is a method of compressing a subgraph, as taught by the claim.
wherein the compressing section includes a computation reducing section (Roy)[pp. 0019]Graph views can reduce the computation overhead on the system, for example, by limiting event handlers to the components, e.g., vertices and/or edges, of a subgraph. Roy teaches of reduction system that reduces computation overhead as taught by the claim.
and a subgraph combining section, , (Roy) [pp. 0068]” The subgraph and the second subgraph can be combined into a composite subgraph. Here Roy explicitly describes the functionality of the subgraph combining section mentioned by the claim.
the computation reducing section receives the subgraphs and the compression configuration as inputs, (Zejda)[Abstract] “ A partitioning scheme that divides the plurality of operations into a set of subgraphs” Zejda teaches of using a portioning scheme to divide a graph into subgraphs. (Wang)[pg.2 1- Introduction] ”of both weights and activations for each layer on different hardware accelerators.” Wang discloses determining a bitwidth for each layer of a neural network and quantizing each layer using its assigned bitwidth. The collection of bitwidth assignments across layers constitutes a set of parameters that control the degree of compression applied to the network, corresponding to a compression configuration. It would be obvious to a person having ordinary skill of the art to apply Wang’s compression configuration to the subgraphs produced by Zejda, which would in turn require the user to receive both subgraphs and the compression configuration as inputs to perform the compression, as taught by the claim.
applies a compression technique described in the compression configuration to each of the subgraphs, and outputs compressed subgraphs, (Wang)[pg.1 1 - Introduction] “Conventional quantization methods use the same number of bits for all layers [2, 14], but as different layers have different redundancy and behave differently on the hardware (computation bounded or memory bounded), it is necessary to use mixed precision for different layers” (Wang)[pg.2 1- Introduction] “For each layer, the agent receives the layer configuration and statistics as observation, and it then outputs the action which is the bitwidth of weights and activations.”
and the subgraph combining section receives an output of the computation reducing section as an input, combines the compressed subgraphs, and outputs a compressed neural network. , (Roy) [pp. 0068]” The subgraph and the second subgraph can be combined into a composite subgraph. Redundant vertex, edge, and/or property graph elements can be removed from the composite subgraph that are common to each of the subgraph and the second subgraph, thereby providing a union of the two views.” Roy teaches combining multiple subgraphs into a composite subgraph and removing redundant graph elements common to the subgraphs. When applied to compressed neural-network subgraphs, this directly corresponds to a subgraph combining section that receives compressed subgraphs, merges them, removes redundant, computation and connectivity, and outputs a compressed neural network as recited in claim 7.
Before the effective filing date, it would have been obvious to one of ordinary skill in the art to combine the algorithm computation reducing section of Zejda, Jin, and Wang with the subgraph combination taught by Roy to improve the neural network compressing model. A person having ordinary skill of the art would be motivated to do so, to optimize the processing of graph-structured data (Roy) [pg.6, col.1, pp.68].
Claim 8:
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Zejda, in view of Roy, and in further view of Lo (US 20200242474 A1, hereinafter Lo).
Regarding claim 8:
Zejda as modified by Roy, teaches all of the limitations of claim 7 except:
Retraining section, wherein the retraining section receives the compressed neural network and a training data set, and trains the compressed neural network.
However, Lo teaches:
Retraining section, wherein the retraining section receives the compressed neural network and a training data set, and trains the compressed neural network. (Lo) [pg.5, col.1, pp.48]” The neural network module 130 includes software interfaces that allow the system to be programmed to implement various types of neural networks… The neural network module can further provide utilities to allow for training and retraining of a neural network implemented with the module.” As anticipated, Lo teaches the implementation, training and retraining of a neural network, as taught in the limitation of claim 8. It would be obvious to a person having ordinary skill of the art that one type of a neural network that can be trained is a compressed neural network, thus rendering the claim obvious.
Before the effective filing date, it would have been obvious to one of ordinary skill in the art to combine the algorithm quantization of Zejda with the receiving of a compressed neural network of Wang with the training and retraining of a neural network of Lo. A person having ordinary skill of the art would be motivated to do so, to allow applications to define neural networks including weights, biases, activation functions, node values, and interconnections between layers of a neural network (Lo)[Description-0048].
Claim 12:
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Zejda and in further view of Wang et, al.
Regarding claim 12:
Zejda teaches all limitations of claim 11 except:
“…a third step of calculating a perturbation of a tentatively compressed neural network. and a fourth step of changing a correspondence of compression techniques to the subgraphs based on the perturbation of the neural network”
However, Wang teaches:
A third step of calculating a perturbation of a tentatively compressed neural network. and a fourth step of changing a correspondence of compression techniques to the subgraphs based on the perturbation of the neural network (Wang)[P.4 3.4 Quantization]"We linearly quantize the weights and activations of each layer using the action, a_k given by our agent" (Wang)[3.5 Reward Function]"After quantization, we retrain the quantized model for one or more epoch, and feed the validation accuracy after short term retraining as the reward signal" (Wang)[Section 3.4 - Constraints]”If the current policy exceeds our resource budget, we will sequentially decrease the bit width of each layer until the constraint is finally satisfied.” Wang teaches tentatively compressing a neural network by assigning bit widths to each layer, evaluating the compressed network by calculating a difference in inference performance relative to the original network, and modifying the layer-wise compression assignments based on this evaluation and predefined constraints. Wang also discloses calculating a perturbation caused by tentative compression and adjusting the correspondence of compression techniques accordingly.
Before the effective filing date, it would have been obvious to one of ordinary skill in the art to combine the quantization of neural networks of Zejda with tentative compression and adjusting of compression techniques of Wang. A person having ordinary skill in the art would be motivated to do so, since linearly quantized models are more efficient to implement on hardware (Wang) [P.4 Section 3.4 Quantization].
Claim 13
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Zejda, in view of Wang, and in further view of Jiang et, al.
Regarding claim 13:
Zejda and Wang teach all of the limitations of claim 11 except:
a fifth step of calculating, for each of the subgraphs, a perturbation of the subgraph in units of tentatively compressed subgraphs; and a sixth step of selecting a subgraph to be further divided based on the perturbation of the subgraph.
However, Jiang teaches:
a fifth step of calculating, for each of the subgraphs, a perturbation of the subgraph in units of tentatively compressed subgraphs; (Jiang) [P. 2 – Definition of Nonlinear Reconstruction Error (NRE)] “Definition of Nonlinear Reconstruction Error (NRE), which computes the Euclidean distance between nonlinear activation values of the unpruned model, can be a more reasonable metric when performing layer-wise pruning” (Jiang)[P. 2-3 layer subnetwork extraction] ”For each single layer pruning, a corresponding 3-layer subnetwork… is extracted and we prune neurons in the hidden layer by minimizing NRE of the output layer” Jiang teaches calculating a perturbation for each subgraph corresponding to a neural network layer by measuring nonlinear reconstruction error between forward-feeding activation values of an original and a tentatively pruned network (pg. 2-3).
And a sixth step of selecting a subgraph to be further divided based on the perturbation of the subgraph. (Jiang) [Pg. 3 – Greedy Selection Procedure] “After computing each neuron’s sensitivity, neurons with the k_i largest sensitivity values are selected” (Jiang) [P. 3 – Iterative pruning] “We adopt an iterative greedy selection procedure… After updating… the sensitivity of neurons will be recalculated and their masks will be updated.” Jiang further teaches selecting neurons and layers for further processing using a greedy strategy based on the largest perturbation (sensitivity) values. Therefore, Jiang teaches all of the limitations of claim 13.
Before the effective filing date, it would have been obvious to one of ordinary skill in the art to combine the quantization of neural networks of Zejda and Wang with the recursive storing of computations taught by Jiang. A person having ordinary skill of the art would be motivated to do so, to gain a more reasonable metric when performing layer-wise pruning (Jiang) [Introduction p.2298].
Claim 14
Claims 14 is rejected under 35 U.S.C. 103 as being unpatentable over Zejda, in view of Wang 2018, and in further view of MetaFlow (Jia et al., MLSys 2019, Jia hereinafter).
Regarding claim 14:
Zejda and Wang (2019) teach all limitations of claim 12 except:
“The neural network compression method according to claim 12, wherein in the first step, a division method into the subgraphs is managed by an m-branch tree, and the neural network is stored at a root of the m-branch tree, the neural network stored at the root of the m-branch tree is divided into m, m subgraphs are stored at a node of depth 1 that is a child of the root, and the subgraphs stored at leaves of the m-branch tree are output.”
However, Jia teaches:
The neural network compression method according to claim 12, wherein in the first step, a division method into the subgraphs is managed by an m-branch tree, and the neural network is stored at a root of the m-branch tree, (Jia) [p.3, Section 2: “Overview”]"Metaflow uses a computation graph G to define computation and state in a DNN model." Jia refers to the entire neural network as a graph, g. This graph is the starting object, or root.
the neural network stored at the root of the m-branch tree is divided into m, m subgraphs are stored at a node of depth 1 that is a child of the root, and the subgraphs stored at leaves of the m-branch tree are output. (Jia) [Figure 3 – Metaflow overview p.3]"Input Comp. graph…independent subgraphs… Flow-based graph split” Figure 3 explicitly depicts the division of the root graph into subgraphs, replicating the division method that is mention in claim 14. (Jia) [Section 4.3 – Flow-Based Recursive Graph Split p.5] "We use a flow-based graph split algorithm to recursively divide a computation graph into smaller individual subgraphs…" Here, Jia mentions recursive division which would be obvious to a person having ordinary skill of the art that a tree-based structure is being utilized.
Jia teaches managing division of a neural network using a recursive graph-splitting algorithm that forms a tree structure, with the original computation graph at the root and recursively generated subgraphs at child nodes. Leaf nodes correspond to final subgraphs output by the algorithm. Although Jia illustrates binary splitting, extending the branching factor to m is an obvious generalization of the disclosed recursive partitioning scheme, rendering claim 14 obvious.
Before the effective filing date, it would have been obvious to one of ordinary skill in the art to combine the quantization of neural networks of Zejda and Wang with the recursive storing of computations taught by Jia. A person having ordinary skill of the art would be motivated to do so, because the graph substitutions are performed locally, and splitting a computation graph can preserve graph substitutions. (Jia) [Section 4.3, p.5, Flow-Based Recursive Graph Split]
Claim 15
Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Zejda, in view of Wang et al., in view of Jia et al., and in further view of Jiang et, al.
Regarding Claim 15:
Zejda, Wang and Jia teach all of the limitations of claim 14, in addition to:
wherein a subgraph stored in a node having a largest perturbation among the m subgraphs stored in the node of depth 1 that is a child of the root is divided into m, (Jia) [p.3 Section 2: “Overview”]"Metaflow uses a computation graph G to define computation and state in a DNN model." Jia refers to the entire neural network as a graph, g. This graph is the starting object, or root.
However, Jiang teaches:
and the subgraph is stored in a child of a node having the largest perturbation of the subgraphs.
and the subgraph is stored in a child of a node having the largest perturbation of the subgraphs. (Jiang) [p. 3 – Sensitivity analysis] “To measure the importance of neurons, we first analyze the sensitivity of NRE with respect to each neuron” (Jiang) [Pg. 3 – Greedy selection rule] “After computing each neuron’s sensitivity, neurons with the k_i largest sensitivity values are selected.”
Jiang teaches selecting the subgraph components exhibiting the largest perturbation by computing sensitivity values derived from layer-wise nonlinear reconstruction error and greedily selecting those with the highest values for further processing (pg. 2-3). This directly corresponds to selecting a subgraph with the largest perturbation as a subgraph to be further divided, as taught in claim 15.
Before the effective filing date, it would have been obvious to one of ordinary skill in the art to combine the quantization of neural networks of Zejda and Jia with the selection of a largest perturbation taught by Jiang. A person having ordinary skill of the art would be motivated to do so, to optimize the Nonlinear reconstruction error, i.e. the difference between forward feeding values. (Jiang) [Section 2.3, p.2300, - Optimization]
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AYAAN AYAZ SHEIKH whose telephone number is (571)272-4643. The examiner can normally be reached MON-FRI 7:30-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Omar Fernandez can be reached at (571) 272-2589. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AYAAN AYAZ SHEIKH/Examiner, Art Unit 2128
/OMAR F FERNANDEZ RIVAS/Supervisory Patent Examiner, Art Unit 2128