DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-21 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1:
According to the first part of the analysis, claim 1 is a method claim. Claim 11 is directed to a parallel processing device and claim 21 is directed to a storage medium storing the program for execution. Thus, claim 1, 11 and 21 does fall into any one of the four statutory categories (i.e. process, machine, manufacture, or composition of matter).
In regard to claim 1:
Step 2A Prong 1:
“ partitioning parallelizable parameters included in the deep learning model” is a mental step of data split.
“ calculating partition parameters partitioned from the parallelizable parameters” is a mental step of data calculation.
“ obtaining a calculation result of the partition parameters” is a mental step of data calculation.
Additional Elements
Step 2A Prong 2:
“ A deep learning model parallel processing method that is performed by a deep learning model parallel processing device, comprising: “ recited in the preamble do not integrate the judicial exception into a practical application. These additional elements are merely directed to using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
“ loading a deep learning model to a main process by a central processing unit (CPU)” do not integrate the judicial exception into a practical application. These additional elements are merely directed to using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
“ by the CPU and storing the partitioned parallelizable parameters in a shared memory” do not integrate the judicial exception into a practical application. These additional elements are merely directed to using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
“ by a sub-process in each of a plurality of graphic processing units (GPUs) while access of the main process to the shared memory is stopped” do not integrate the judicial exception into a practical application. These additional elements are merely directed to using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
“ from the shared memory by the CPU; and outputting the calculation result by the CPU” do not integrate the judicial exception into a practical application. These additional elements are merely directed to using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
“ and outputting the calculation result by the CPU” do not integrate the judicial exception into a practical application. These additional elements are merely directed to using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
Step 2B:
“ A deep learning model parallel processing method that is performed by a deep learning model parallel processing device, comprising: “ recited in the preamble is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
“ loading a deep learning model to a main process by a central processing unit (CPU)” is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
“ by the CPU and storing the partitioned parallelizable parameters in a shared memory” is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
“ by a sub-process in each of a plurality of graphic processing units (GPUs) while access of the main process to the shared memory is stopped” is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
“ from the shared memory by the CPU; and outputting the calculation result by the CPU” is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
“ and outputting the calculation result by the CPU” is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
In regard to claim 2:
Step 2A Prong 1:
“ partitioning parameters corresponding to the first layer set by columns into first partition parameters and second partition parameters” is a mental step of data split.
Additional Elements
Step 2A Prong 2:
“wherein the deep learning model includes a first layer set and a second layer set that is subsequent to the first layer set, and the calculating partition parameters includes:” do not integrate the judicial exception into a practical application. These additional elements are merely directed to using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
“ and storing the first partition parameters and the second partition parameters in the shared memory” is directed to Storing and Retrieval from Memory and does not integrate the judicial exception into a practical application. See MPEP 2106.05(d)(ii)(iv)
Step 2B:
“wherein the deep learning model includes a first layer set and a second layer set that is subsequent to the first layer set, and the calculating partition parameters includes:” is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
“ and storing the first partition parameters and the second partition parameters in the shared memory” is directed to Storing and Retrieval from Memory and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(d)(ii)(iv)
In regard to claim 3:
Step 2A Prong 1:
“ performing calculation on the first partition parameters by a first sub-process “” is a mental step of data calculation.
“ performing calculation on the second partition parameters by a second sub-process ” is a mental step of data calculation.
Additional Elements
Step 2A Prong 2:
“ in a first GPU among the plurality of GPUs and producing first output parameters” do not integrate the judicial exception into a practical application. These additional elements are merely directed to using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
“ in a second GPU among the plurality of GPUs and producing second output “do not integrate the judicial exception into a practical application. These additional elements are merely directed to using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
“ and storing the first output parameters and the second output parameters in the shared memory” is directed to Storing and Retrieval from Memory and does not integrate the judicial exception into a practical application. See MPEP 2106.05(d)(ii)(iv)
Step 2B:
“ in a first GPU among the plurality of GPUs and producing first output parameters” is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
“ in a second GPU among the plurality of GPUs and producing second output “is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
“ and storing the first output parameters and the second output parameters in the shared memory” is directed to Storing and Retrieval from Memory and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(d)(ii)(iv)
In regard to claim 4:
Step 2A Prong 1:
“ partitioning parameters corresponding to the second layer set by rows into third partition parameters and fourth partition parameters” ” is a mental step of data split.
Additional Elements
Step 2A Prong 2:
“ and storing the third partition parameters and the fourth partition parameters in the shared memory” is directed to Storing and Retrieval from Memory and does not integrate the judicial exception into a practical application. See MPEP 2106.05(d)(ii)(iv)
Step 2B:
“ and storing the third partition parameters and the fourth partition parameters in the shared memory” is directed to Storing and Retrieval from Memory and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(d)(ii)(iv).
In regard to claim 5:
Step 2A Prong 1:
“ performing calculation on the first output parameters and the third partition parameters by the first sub-process” ” is a mental step of data calculation.
“ performing calculation on the second output parameters and the fourth partition parameters by the second sub-process” ” is a mental step of data calculation.
“ and producing a final calculation result by allowing the third output parameters and the fourth output parameters “is a mental step of data calculation.
Additional Elements
Step 2A Prong 2:
“ in the first GPU among the plurality of GPUs and producing third output parameters;” do not integrate the judicial exception into a practical application. These additional elements are merely directed to using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
“ in the second GPU among the plurality of GPUs and producing fourth output parameters” do not integrate the judicial exception into a practical application. These additional elements are merely directed to using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
“ to be shared between the first GPU and the second GPU through the shared memory” do not integrate the judicial exception into a practical application. These additional elements are merely directed to using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
Step 2B:
“ in the first GPU among the plurality of GPUs and producing third output parameters;” is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
“ in the second GPU among the plurality of GPUs and producing fourth output parameters” is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
“ to be shared between the first GPU and the second GPU through the shared memory” is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
In regard to claim 6:
Step 2A Prong 2:
“ wherein the outputting a calculation result includes:” do not integrate the judicial exception into a practical application. These additional elements are merely directed to using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
“ outputting the calculation result based on the final calculation result in the CPU” do not integrate the judicial exception into a practical application. These additional elements are merely directed to using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
Step 2B:
“ wherein the outputting a calculation result includes:” is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
“ outputting the calculation result based on the final calculation result in the CPU” is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
In regard to claim 7:
Step 2A Prong 2:
“ wherein the deep learning model includes a self-attention layer and a multilayer perceptron (MLP) layer, and the parallelizable parameters correspond to the self-attention layer and the MLP layer” do not integrate the judicial exception into a practical application. These additional elements are merely directed to using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
Step 2B:
“ wherein the deep learning model includes a self-attention layer and a multilayer perceptron (MLP) layer, and the parallelizable parameters correspond to the self-attention layer and the MLP layer” is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
In regard to claim 8:
Step 2A Prong 2:
“ wherein the storing in the shared memory includes: partitioning parameters corresponding to a first linear projection part of the self-attention layer by columns and partitioning parameters corresponding to a first output projection part subsequent to the first linear projection part by rows” do not integrate the judicial exception into a practical application. These additional elements are merely directed to using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
Step 2B:
“ wherein the storing in the shared memory includes: partitioning parameters corresponding to a first linear projection part of the self-attention layer by columns and partitioning parameters corresponding to a first output projection part subsequent to the first linear projection part by rows” is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
In regard to claim 9:
Step 2A Prong 2:
“ wherein the storing in the shared memory includes: partitioning parameters corresponding to a second linear projection part of the MLP layer by columns and partitioning parameters corresponding to a second output projection part subsequent to the second linear projection part by rows” do not integrate the judicial exception into a practical application. These additional elements are merely directed to using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
Step 2B:
“ wherein the storing in the shared memory includes: partitioning parameters corresponding to a second linear projection part of the MLP layer by columns and partitioning parameters corresponding to a second output projection part subsequent to the second linear projection part by rows” is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
In regard to claim 10:
Step 2A Prong 2:
“ constructing a library including a plurality of programs and a plurality of routines for the deep learning model, wherein a user code is operated in the main process, and a framework code for the plurality of programs and the plurality of routines is operated in the sub-process” do not integrate the judicial exception into a practical application. These additional elements are merely directed to using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
Step 2B:
“ constructing a library including a plurality of programs and a plurality of routines for the deep learning model, wherein a user code is operated in the main process, and a framework code for the plurality of programs and the plurality of routines is operated in the sub-process” is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
In regard to claim 11:
Step 2A Prong 1:
“partitions parallelizable parameters included in the deep learning model” is a mental step of data split.
Additional Elements
Step 2A Prong 2:
“ A deep learning model parallel processing device, comprising: a CPU that loads a deep learning model to a main process” recited in the preamble do not integrate the judicial exception into a practical application. These additional elements are merely directed to using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
“ and outputs a calculation result of partition parameters partitioned from the parallelizable parameters” do not integrate the judicial exception into a practical application. These additional elements are merely directed to using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
“ and a shared memory that is accessible by the main process and a plurality of sub-process and is able to store the partitioned parallelizable parameters for a predetermined period of time and store the calculation result for a predetermined period of time, wherein while access of the main process to the shared memory is stopped, the partition parameters are calculated by the sub-process in each of a plurality of graphic processing units (GPUs).” do not integrate the judicial exception into a practical application. These additional elements are merely directed to using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
Step 2B:
“ A deep learning model parallel processing device, comprising: a CPU that loads a deep learning model to a main process” recited in the preamble is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
“ and outputs a calculation result of partition parameters partitioned from the parallelizable parameters” is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
“ and a shared memory that is accessible by the main process and a plurality of sub-process and is able to store the partitioned parallelizable parameters for a predetermined period of time and store the calculation result for a predetermined period of time, wherein while access of the main process to the shared memory is stopped, the partition parameters are calculated by the sub-process in each of a plurality of graphic processing units (GPUs).” is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
In regard to claim 12:
Step 2A Prong 1:
“ CPU partitions the parameters corresponding to the first layer set by columns into first partition parameters and second partition parameters (except CPU)” is a mental step of data split.
Additional Elements
Step 2A Prong 2:
“wherein the deep learning model includes a first layer set and a second layer set that is subsequent to the first layer set, and the calculating partition parameters includes:” do not integrate the judicial exception into a practical application. These additional elements are merely directed to using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
“CPU” do not integrate the judicial exception into a practical application. These additional elements are merely directed to using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
“ and stores the first partition parameters and the second partition parameters in the shared memory” is directed to Storing and Retrieval from Memory and does not integrate the judicial exception into a practical application. See MPEP 2106.05(d)(ii)(iv)
Step 2B:
“wherein the deep learning model includes a first layer set and a second layer set that is subsequent to the first layer set, and the calculating partition parameters includes:” is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
“CPU” is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
“ and stores the first partition parameters and the second partition parameters in the shared memory” is directed to Storing and Retrieval from Memory and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(d)(ii)(iv)
In regard to claim 13:
Step 2A Prong 1:
“ performs calculation on the first partition parameters by a first sub-process “” is a mental step of data calculation.
“ performs calculation on the second partition parameters by a second sub-process ” is a mental step of data calculation.
Additional Elements
Step 2A Prong 2:
“ wherein a first sub-process in a first GPU among the plurality of GPUs and producing first output parameters” do not integrate the judicial exception into a practical application. These additional elements are merely directed to using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
“ wherein a second sub-process in a second GPU among the plurality of GPUs and producing second output “do not integrate the judicial exception into a practical application. These additional elements are merely directed to using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
“ the first output parameters and the second output parameters are stored in the shared memory” is directed to Storing and Retrieval from Memory and does not integrate the judicial exception into a practical application. See MPEP 2106.05(d)(ii)(iv)
Step 2B:
“ the first sub-process in a first GPU among the plurality of GPUs and producing first output parameters” is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
“ the second sub-process in a second GPU among the plurality of GPUs and producing second output “is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
“the first output parameters and the second output parameters are stored in the shared memory” is directed to Storing and Retrieval from Memory and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(d)(ii)(iv)
In regard to claim 14:
Step 2A Prong 1:
“ partitions parameters corresponding to the second layer set by rows into third partition parameters and fourth partition parameters (except wherein the CPU)” is a mental step of data split.
Additional Elements
Step 2A Prong 2:
“ wherein the CPU” do not integrate the judicial exception into a practical application. These additional elements are merely directed to using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
“ stores the third partition parameters and the fourth partition parameters in the shared memory” is directed to Storing and Retrieval from Memory and does not integrate the judicial exception into a practical application. See MPEP 2106.05(d)(ii)(iv)
Step 2B:
“ wherein the CPU” is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
“ stores the third partition parameters and the fourth partition parameters in the shared memory” is directed to Storing and Retrieval from Memory and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(d)(ii)(iv).
In regard to claim 15:
Step 2A Prong 1:
“ performs calculation on the first output parameters and the third partition parameters by the first sub-process” ” is a mental step of data calculation.
“ performs calculation on the second output parameters and the fourth partition parameters by the second sub-process” ” is a mental step of data calculation.
“ and produced a final calculation “is a mental step of data calculation.
Additional Elements
Step 2A Prong 2:
“wherein the first sub-process in the first GPU among the plurality of GPUs and producing third output parameters;” do not integrate the judicial exception into a practical application. These additional elements are merely directed to using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
“ wherein the second sub-process in the second GPU among the plurality of GPUs and producing fourth output parameters” do not integrate the judicial exception into a practical application. These additional elements are merely directed to using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
“ produces the third output parameters “ do not integrate the judicial exception into a practical application. These additional elements are merely directed to using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
Step 2B:
“wherein the first sub-process in the first GPU among the plurality of GPUs and producing third output parameters;” is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
““ wherein the second sub-process in the second GPU among the plurality of GPUs and producing fourth output parameters” is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
“ produces the third output parameters” is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
In regard to claim 16:
Step 2A Prong 2:
“ wherein the CPU outputs the calculation result based on the final calculation result:” do not integrate the judicial exception into a practical application. These additional elements are merely directed to using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
Step 2B:
“ wherein the CPU outputs the calculation result based on the final calculation result” is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
“ outputting the calculation result based on the final calculation result in the CPU” is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
In regard to claim 17:
Step 2A Prong 2:
“ wherein the deep learning model includes a self-attention layer and a multilayer perceptron (MLP) layer, and the parallelizable parameters correspond to the self-attention layer and the MLP layer” do not integrate the judicial exception into a practical application. These additional elements are merely directed to using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
Step 2B:
“ wherein the deep learning model includes a self-attention layer and a multilayer perceptron (MLP) layer, and the parallelizable parameters correspond to the self-attention layer and the MLP layer” is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
In regard to claim 18:
Step 2A Prong 2:
“ wherein CPU partitions the parameters corresponding to a first linear projection part of the self-attention layer by columns and partitioning parameters corresponding to a first output projection part subsequent to the first linear projection part by rows” do not integrate the judicial exception into a practical application. These additional elements are merely directed to using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
Step 2B:
“ wherein the CPU partitions the parameters corresponding to a first linear projection part of the self-attention layer by columns and partitioning parameters corresponding to a first output projection part subsequent to the first linear projection part by rows” is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
In regard to claim 19:
Step 2A Prong 2:
“ wherein the CPU partitions the parameters corresponding to a second linear projection part of the MLP layer by columns and partitioning parameters corresponding to a second output projection part subsequent to the second linear projection part by rows” do not integrate the judicial exception into a practical application. These additional elements are merely directed to using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
Step 2B:
“ wherein the CPU partitions the parameters corresponding to a second linear projection part of the MLP layer by columns and partitioning parameters corresponding to a second output projection part subsequent to the second linear projection part by rows” is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
In regard to claim 20:
Step 2A Prong 2:
“a library including a plurality of programs and a plurality of routines for the deep learning model, wherein a user code is operated in the main process, and a framework code for the plurality of programs and the plurality of routines is operated in the sub-process” do not integrate the judicial exception into a practical application. These additional elements are merely directed to using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
Step 2B:
“a library including a plurality of programs and a plurality of routines for the deep learning model, wherein a user code is operated in the main process, and a framework code for the plurality of programs and the plurality of routines is operated in the sub-process” is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
In regard to claim 21:
Step 2A Prong 1:
“ partition parallelizable parameters included in the deep learning model” is a mental step of data split.
“ calculate partition parameters partitioned from the parallelizable parameters is a mental step of data calculation.
“ obtain a calculation result of the partition parameters” is a mental step of data calculation.
Additional Elements
Step 2A Prong 2:
“ A non-transitory computer-readable storage medium storing a computer program including a sequence of instructions to perform parallel processing of a deep learning model, wherein the computer program includes a sequence of instructions that, when executed by a computing device, cause the computing device to:” recited in the preamble do not integrate the judicial exception into a practical application. These additional elements are merely directed to using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
“ load a deep learning model to a main process by a central processing unit (CPU)” do not integrate the judicial exception into a practical application. These additional elements are merely directed to using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
“ by the CPU and store the partitioned parallelizable parameters in a shared memory” do not integrate the judicial exception into a practical application. These additional elements are merely directed to using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
“ by a sub-process in each of a plurality of graphic processing units (GPUs) while access of the main process to the shared memory is stopped” do not integrate the judicial exception into a practical application. These additional elements are merely directed to using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
“ from the shared memory by the CPU; and outputting the calculation result by the CPU” do not integrate the judicial exception into a practical application. These additional elements are merely directed to using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
“ and output the calculation result by the CPU” do not integrate the judicial exception into a practical application. These additional elements are merely directed to using a computer as a tool to perform an abstract idea. See MPEP 2106.05(h).
Step 2B:
“ A non-transitory computer-readable storage medium storing a computer program including a sequence of instructions to perform parallel processing of a deep learning model, wherein the computer program includes a sequence of instructions that, when executed by a computing device, cause the computing device to:” recited in the preamble is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
“ load a deep learning model to a main process by a central processing unit (CPU)” is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
“ by the CPU and store the partitioned parallelizable parameters in a shared memory” is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
“ by a sub-process in each of a plurality of graphic processing units (GPUs) while access of the main process to the shared memory is stopped” is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
“ from the shared memory by the CPU; and outputting the calculation result by the CPU” is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
“ and output the calculation result by the CPU” is directed to a generic computer function and does not amount to significantly more than the judicial exception in the claim. See MPEP 2106.05(h).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over
Juniping Zhao et. al. (hereinafter Zhao) US 2019/0324856 A1,
in view of Mario ALMEIDA et. al. (hereinafter ALME) US 2022/0083386 A1,
in view of Mohamed Shahim et.al (hereinafter Shahim) US 2021/0303346 A1.
In regard to claim 1:
Zhao discloses:
- A deep learning model parallel processing method that is performed by a deep learning model parallel processing device, comprising:
loading a deep learning model to a main process by a central processing unit (CPU);
in [0026]:
A distributed SGD DL training process can be implemented by the deep learning computing platform 110 in the HPC system 100 using a data-parallel programming model in which the SGD training process is executed in parallel on a plurality of GPU devices 160 that are distributed over one or more compute nodes of the HPC system 100.
in [0064]:
In one embodiment, the various system components 110, 120 and 130 of the deep learning system 630 comprise software modules that are persistently stored in a storage device and loaded into the system memory resources
In [0015]:
HPC task running on a plurality of hardware accelerator devices (e.g., GPU devices) is copied to host system memory through a host-device (e.g., CPU-GPU) coordinated checkpoint protocol
(BRI: A CPU coordinating with a GPU via the system memory implies that a model is loaded into system memory by CPU)
Zhao does not explicitly disclose:
- partitioning parallelizable parameters included in the deep learning model by the CPU and storing the partitioned parallelizable parameters in a shared memory;
However, ALME discloses:
- partitioning parallelizable parameters included in the deep learning model by the CPU and storing the partitioned parallelizable parameters in a shared memory;
In [0005]:
partitioning the neural network into a number of partitions based on the determined subset of computing resources that are able to satisfy the at least one optimisation constraint; assigning each partition of the neural network to be executed by one of the determined subset of computing resources; and scheduling the execution of the partitions of the neural network by each computing resource assigned to execute a partition.,
in [0061] :
The partitions may be of equal or unequal sizes. As explained more below, in some cases, a computing resource may itself be able to share part of the computation of a partition of the neural network with a further computing resource
(BRI: A computing resource may include shared memory)
In [0015]:
communicate or send data between the user device and each computing resource, and use this information to determine if any data transfer optimisation process is required to transfer data to the computing resource for executing the neural network.
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Zhao, and ALME.
Zhao teaches the parallel processing the model using a CPU-GPU platform.
ALME teaches portioning the model.
One of ordinary skill would have motivation to combine Zhao, and ALME that can use tensor quantization to reduce the model size and provide speed-up (ALME [0018]).
Zhao and ALME do not explicitly disclose:
- calculating a partitionable parameter from a partitioned from the partitionable parameters by a sub-process in each of a plurality of graphic processing units (GPUs) while access of the main process to the shared memory is stopped;
- obtaining a calculation result of the partition parameters from the shared memory by the CPU
- and outputting the calculation result by the CPU.
However, Shahim discloses:
- calculating a partitionable parameter from a partitioned from the partitionable parameters by a sub-process in each of a plurality of graphic processing units (GPUs) while access of the main process to the shared memory is stopped;
In [0015]:
The queue processor 100 can interface with components and/or resources of the multicore processor system 200, such as a main memory 230, a control processor, a set of direct memory access controllers or engines (hereinafter “DMA engines 220”), a set of processing units 210—such as general purpose processing units (hereinafter “CPUs”), graphics processing units (hereinafter “GPUs”),
(BRI: GPU execution is a sub-process)
In [0015]:
Additionally, the queue processor 100 itself includes a set of subcomponents such as a set of command queues 110, command logic module 130 for each command queue 110, a reorder buffer 140 for each command queue 110, a signal-wait counter matrix 120
In [0016]:
the scheduler and software compiler can convert a DAG representation of a parallel process (e.g., an inference of a CNN based on an input image) to command streams that can be input to the queue processor 100 for decoding and routing to the various resources of a multicore processor system 200, while maintaining dependencies between commands in disparate command streams.
In [0020] :
Upon detecting a wait primitive via the command logic module 130, the queue processor 100 can interface with the signal-wait counter matrix 120 to properly handle the dependencies represented by these wait primitives.
In [0014]:
the queue processor 100 can leverage a schedule, pre-generated by a scheduler (or dynamically generated by an on-chip control processor) and including signal primitives and wait primitives, to: receive a set of command streams from main memory 230; populate a set of command queues 110 with the set of command streams; and route control signals to resources of the multicore processor system 200 specified by the set of control streams while tracking dependencies between the set of control streams via a signal-wait counter matrix 120 implemented in hardware.
In [0058]:
by allocating command streams according to resource type of the instructions in the command stream, the scheduler can reduce conflicts between command streams for the same set of resources of the multicore processor system 200.
(BRI: With a “ wait matrix" dependencies for a memory stream can be managed which is particularly relevant in parallel computing environments like GPUs. The system 200 includes GPUs (Performing sub-process and Memory)
- obtaining a calculation result of the partition parameters from the shared memory by the CPU
In [0037]:
The queue processor 100 interfaces with a set of processing units 210 of the multicore processor system 200, such as CPUs, GPUs, and/or specialized DLPs, in order to execute the statically scheduled parallel process.
In [0037]:
the queue processor 100 does not restrict particular queues to a particular subset of processing units 210, thereby enabling dynamic mappings of queues to resources in the static schedule for the parallel process.
In [0085] :
As shown in FIG. 4, in one variation, the queue processor 100 can include multiple groups of command queues 110, where each group: includes a subset of command queues 110 in the set of command queues 110; shares a single dequeuing arbiter 132; shares a group reorder buffer 140; and accesses a set of signal-wait counter matrix 124 partitions, where each partition includes registers representing dependencies between the command queues 110 corresponding to a pair of command queue 110 groups
In [0085]:
by including multiple groups of command queues 110 that share components, the queue processor 100 can reduce the number of read-modify-write ports between the command logic module 130, reorder buffers 140, and the various partitions of the signal-wait counter matrix 120, thereby reducing the on-chip area overhead of the signal-wait counter matrix 120.
In [0086] :
For example, in variations of the queue processor 100 that do not include command queue 110 grouping and that includes a signal-wait counter matrix 120 implemented with flops, each command queue 110 and corresponding reorder buffer 140 interfaces with each register of the signal-wait counter matrix 120 with a read-modify-write port resulting in a set of 2.sup.n read-modify-write ports, where n represents the number of command queues 110. However, in the command queue 110 grouping variation, the queue processor 100 includes a partitioned signal-wait counter matrix 120, where each partition of the signal-wait counter matrix 120 represents dependencies of the command queues 110 of a first group on the command queues 110 on a second group in the set of groups
(BRI: a command queue that shares components can include shared memory. A Reorder Buffer (ROB) is a CPU hardware component that allows instructions to execute out-of-order but ensures that their results are committed to the architectural state (registers and memory) in their original program order
- and outputting the calculation result by the CPU.
In [0045]:
Each command logic module 130 takes as input the value of the last command container or the specific decode container in addition to values stored in registers of the signal-wait counter matrix 120 and outputs values to the reorder buffers 140 of the queue processor 100 and to the signal-wait counter matrix 120. Thus, the queue processor 100 includes command logic module 130 that is communicatively coupled to a set of reorder buffers 140 for each command queue 110 of the queue processor 100.
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Zhao, ALME and Shahim.
Zhao teaches the parallel processing the model using a CPU-GPU platform.
ALME teaches
Shahim teaches partitioning the parameters and storing in the shared memory and obtaining calculation results and outputting the results by CPU.
One of ordinary skill would have motivation to combine Zhao, and Shahim that can improve the performance of the multicore system by using a queue processor (Shahim [0026]).
In regard to claim 21:
Zhao discloses:
- A non-transitory computer-readable storage medium storing a computer program including a sequence of instructions to perform parallel processing of a deep learning model, wherein the computer program includes a sequence of instructions that, when executed by a computing device, cause the computing device to:
Load a deep learning model to a main process by a central processing unit (CPU); partition parallelizable parameters included in the deep learning model by the CPU; store the partitioned parallelizable parameters in a shared memory;
In [0064], in [0038],
in [0026]:
A distributed SGD DL training process can be implemented by the deep learning computing platform 110 in the HPC system 100 using a data-parallel programming model in which the SGD training process is executed in parallel on a plurality of GPU devices 160 that are distributed over one or more compute nodes of the HPC system 100.
in [0064]:
In one embodiment, the various system components 110, 120 and 130 of the deep learning system 630 comprise software modules that are persistently stored in a storage device and loaded into the system memory resources
In [0015]:
HPC task running on a plurality of hardware accelerator devices (e.g., GPU devices) is copied to host system memory through a host-device (e.g., CPU-GPU) coordinated checkpoint protocol
(BRI: A CPU coordinating with a GPU via the system memory implies that a model is loaded into system memory by CPU)
Zhao does not explicitly disclose:
- partition parallelizable parameters included in the deep learning model by the CPU; store the partitioned parallelizable parameters in a shared memory;
However, ALME discloses:
- partition parallelizable parameters included in the deep learning model by the CPU; store the partitioned parallelizable parameters in a shared memory;
In [0005]:
partitioning the neural network into a number of partitions based on the determined subset of computing resources that are able to satisfy the at least one optimisation constraint; assigning each partition of the neural network to be executed by one of the determined subset of computing resources; and scheduling the execution of the partitions of the neural network by each computing resource assigned to execute a partition.,
in [0061] :
The partitions may be of equal or unequal sizes. As explained more below, in some cases, a computing resource may itself be able to share part of the computation of a partition of the neural network with a further computing resource
(BRI: A computing resource may include shared memory)
In [0015]:
communicate or send data between the user device and each computing resource, and use this information to determine if any data transfer optimisation process is required to transfer data to the computing resource for executing the neural network.
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Zhao, and ALME.
Zhao teaches the parallel processing the model using a CPU-GPU platform.
ALME teaches portioning the model.
One of ordinary skill would have motivation to combine Zhao, and ALME that can use tensor quantization to reduce the model size and provide speed-up (ALME [0018]).
Zhao and ALME do not explicitly disclose:
- calculate partition parameters partitioned from the parallelizable parameters by a sub-process in each of a plurality of graphic processing units (GPUs) while access of the main process to the shared memory is stopped;
- obtain a calculation result of the partition parameters from the shared memory by the CPU; and output the calculation result by the CPU.
- and output the calculation result by the CPU.
However, Shahim discloses:
- calculate partition parameters partitioned from the parallelizable parameters by a sub-process in each of a plurality of graphic processing units (GPUs) while access of the main process to the shared memory is stopped;
In [0015]:
The queue processor 100 can interface with components and/or resources of the multicore processor system 200, such as a main memory 230, a control processor, a set of direct memory access controllers or engines (hereinafter “DMA engines 220”), a set of processing units 210—such as general purpose processing units (hereinafter “CPUs”), graphics processing units (hereinafter “GPUs”),
In [0015]:
Additionally, the queue processor 100 itself includes a set of subcomponents such as a set of command queues 110, command logic module 130 for each command queue 110, a reorder buffer 140 for each command queue 110, a signal-wait counter matrix 120
In [0016]:
the scheduler and software compiler can convert a DAG representation of a parallel process (e.g., an inference of a CNN based on an input image) to command streams that can be input to the queue processor 100 for decoding and routing to the various resources of a multicore processor system 200, while maintaining dependencies between commands in disparate command streams.
In [0020] :
Upon detecting a wait primitive via the command logic module 130, the queue processor 100 can interface with the signal-wait counter matrix 120 to properly handle the dependencies represented by these wait primitives.
In [0014]:
the queue processor 100 can leverage a schedule, pre-generated by a scheduler (or dynamically generated by an on-chip control processor) and including signal primitives and wait primitives, to: receive a set of command streams from main memory 230; populate a set of command queues 110 with the set of command streams; and route control signals to resources of the multicore proces