DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claims 1-20 are objected to because of the following informalities.
Claim 19 lines 2-3 recite “the second matrix stored while they first matrix and second matrix are stored in the memory portion”. This appears to include a typographical error and should possibly recite “the second matrix stored while the first matrix and second matrix are stored in the memory portion”.
Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-19 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
Regarding claim 1, under the Alice framework Step 1, the claim falls within the four statutory categories of patentable subject matter identified by 35 USC 101: a process, machine, manufacture or a composition of matter.
Under the Alice framework Step 2A prong 1, the claim recites mental steps comprising determination steps with respect to performing an emulation or not, and mathematical concepts of mathematical calculations and mathematical relationships comprising matrix multiplication using floating-point. Specifically, claim 1 recites the following mental steps and mathematical calculations and mathematical relationships:
A method of performing a floating-point operation, the floating-point operation being a multiplication of a first matrix and a second matrix that are double-precision floating-point matrices (mathematical calculations and mathematical relationships), the method comprising:
determining whether an emulation is to be used to perform the floating-point operation (mental step);
based on a result of the determining whether the emulation is to be used, determining whether to use the memory processor for the emulation, the emulation comprising stages (mental step);
based on a result of the determining whether to use the memory processor for the emulation, individually determining whether to use the memory processor for each stage of the emulation (mental step); and
multiplying the first matrix and the second matrix based on a result of the individually determining whether to use the memory processor (mathematical calculation and mental step).
Under the Alice framework Step 2A prong 2 analysis, claim 1 recites the following additional elements: performing the floating-point operation using a memory processor. These elements are recited at a very high level of generality, wherein mathematical calculations and mathematical relationships are merely “applied” in a generically recited memory processor, or generally linked to a particular technological environment, without specifically limiting the functions performed by the memory processor or an emulator to actually perform the specific steps beyond the mental step of determining. For these reasons, claim 1 is not integrated into a practical application.
Under the Alice Framework Step 2B analysis, claim 1 considered individually and as an ordered combination does not include additional elements that are sufficient to amount to significantly more than the abstract idea. As stated in the Step 2A prong 2 analysis, the claim does no more than generally link generically recited memory processor to a particular technological environment. For these reasons claim 1 does not amount to significantly more than the abstract idea.
Claims 2-14 are rejected for at least the reasons set forth with respect to claim 1. Claims 2-12, and 14-15 merely further limit the mental steps with further details of the mental steps of claim 1. Claims 2-12, and 14-15 contain no further additional elements beyond those recited in claim 1 that would require further analysis under step 2A prong 2 and step 2B.
Claim 13 recites the following further additional element: wherein the multiplying of the first matrix and the second matrix is controlled by at least one memory processor through a direct memory access (DMA) when the memory processor is used for the emulation. Under the step 2A prong 2 analysis, this step continues to merely generally link a generically recited DMA for controlling of the matrix multiplication. For these reasons claim 13 is not integrated into a practical application. Furthermore under the step 2B analysis, the control of mathematical operations by a DMA is well understood, routine and conventional activity. See e.g., J.L. Hennessy et al., Computer architecture: A quantitative approach, Elsevier Science & Technology, 2014 (hereinafter “Hennessy”), describing use of DMA as a hardware used in many computers to allow transfers to and from memory without interventions from the CPU (p. 700-701)
Claim 15 is directed to a non-transitory computer-readable storage medium storing instructions that when executed by a processor, cause the processor to perform the method as in claim 1. The claim 1 analysis applies equally to claim 15.
Regarding claim 16, under the Alice framework Step 1, the claim falls within the four statutory categories of patentable subject matter identified by 35 USC 101: a process, machine, manufacture or a composition of matter.
Under the Alice framework Step 2A prong 1, the claim recites mental steps comprising determination steps with respect to performing an emulation or not, and mathematical concepts of mathematical calculations and mathematical relationships comprising matrix multiplication using floating-point. Specifically, claim 16 recites the following mental steps and mathematical calculations and mathematical relationships:
performing a floating-point operation, comprising:
determine whether an emulation is to be used to perform the floating-point operation (mental step);
based on a result of the determining whether the emulation is to be used, determining whether to use the memory processor for the emulation, the emulation comprising stages (mental step);
based on a result of the determining whether to use the memory processor for the emulation, individually determining whether to use the memory processor for each stage of the emulation (mental step); and
multiply a first matrix and the second matrix based on a result of the individually determining whether to use the memory processor (mathematical calculation and mental step).
Under the Alice framework Step 2A prong 2 analysis, claim 1 recites the following additional elements: an accelerator core; a memory system; and a memory processor included in the memory system, and receiving a floating-point operation request from a processor. These elements are recited at a very high level of generality, wherein mathematical calculations and mathematical relationships are merely “applied” in a generically recited accelerator, and memory processor, or generally linked to a particular technological environment, without specifically limiting the functions performed by the memory processor or an emulator to actually perform the specific steps beyond the mental step of determining. Furthermore, the step of receiving a floating-point operation request from a processor comprises an insignificant extra solution activity. For these reasons, claim 16 is not integrated into a practical application.
Under the Alice Framework Step 2B analysis, claim 16 considered individually and as an ordered combination does not include additional elements that are sufficient to amount to significantly more than the abstract idea. As stated in the Step 2A prong 2 analysis, the claim does no more than generally link generically recited accelerator and memory processor to a particular technological environment. Furthermore the receiving a floating-point operation request from a process comprises well-understood, routine and conventional activity. See MPEP 2106.05(d).II.i. For these reasons claim 16 does not amount to significantly more than the abstract idea.
Regarding claim 17, under the Alice framework Step 1, the claim falls within the four statutory categories of patentable subject matter identified by 35 USC 101: a process, machine, manufacture or a composition of matter.
Under the Alice framework Step 2A prong 1, the claim recites mental steps comprising determination steps with respect to performing an emulation or not, and mathematical concepts of mathematical calculations and mathematical relationships comprising matrix multiplication using floating-point. Specifically, claim 17 recites the following mental steps and mathematical calculations and mathematical relationships:
performing a floating-point operation, comprising:
determine whether an emulation is to be used to perform the floating-point operation (mental step);
based on a result of the determining whether the emulation is to be used, determining whether to use the memory processor for the emulation, the emulation comprising stages (mental step);
based on a result of the determining whether to use the memory processor for the emulation, individually determining whether to use the memory processor for each stage of the emulation (mental step); and
multiply a first matrix and the second matrix based on a result of the individually determining whether to use the memory processor (mathematical calculation and mental step).
Under the Alice framework Step 2A prong 2 analysis, claim 1 recites the following additional elements: a computing device comprising a processor; a memory; and a memory processor included in the memory. These elements are recited at a very high level of generality, wherein mathematical calculations and mathematical relationships are merely “applied” in a generically recited processor, and memory processor, or generally linked to a particular technological environment, without specifically limiting the functions performed by the memory processor or an emulator to actually perform the specific steps beyond the mental step of determining. For these reasons, claim 17 is not integrated into a practical application.
Under the Alice Framework Step 2B analysis, claim 17 considered individually and as an ordered combination does not include additional elements that are sufficient to amount to significantly more than the abstract idea. As stated in the Step 2A prong 2 analysis, the claim does no more than generally link generically recited accelerator and memory processor to a particular technological environment. For these reasons claim 17 does not amount to significantly more than the abstract idea.
Claims 18-19 are rejected for at least the reasons set forth with respect to claim 17. Claims 18-19 merely further limit the mental steps with further details of the mental steps of claim 17. Claims 18-19 contain no further additional elements beyond those recited in claim 1 that would require further analysis under step 2A prong 2 and step 2B.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 8, and 13-19 are rejected under 35 U.S.C. 103 as being unpatentable over S. Lee et al., Hardware Architecture and Software Stack for PIM Based on Commercial DRAM Technology, 2021 ACM/IEEE 48th Annual International Symposium on Computer Architecture (ISCA), 2021 (hereinafter “Lee”) in view of J.H. Kim et al., Aquabolt-X: HBM2-PIM, LPDDR5-PIM With In-Memory Processing, and AXDIMM With Acceleration Buffer, Theme Article: HOT CHIPS 33, 5 April 2022 in view of B. Cho et al., Accelerating Bandwidth-Bound Deep Learning Inference with Main-Memory Accelerators, ACM SC ’21, 2021 (hereinafter “Cho”) in view of D. Mukunoki et al., DGEMM Using Tensor Cores, and Its Accurate and Reproducible Versions, High Performance Computing, 35th International Conference, ISC High Performance 2020 Proceedings, LNCS 12151, 2020 (hereinafter “Mukunoki”) .
Regarding claim 1, Lee teaches the following:
a method of performing a floating-point operation using a memory processor, the floating-point operation being a multiplication of a first matrix and a second vector that are floating-point (abstract, fig 1, section III.A, SIMD floating-point unit (FPU), section III.C. Data Operation Types including GEMV- matrix vector multiplication ), the method comprising:
determining the processing is to be used to perform the floating-point operation (Section V.A p. 49 right column, PIM preprocessor analyzes the source code of applications and finds TensorFlow (TF) ops suitable for PIM acceleration at runtime, section VIII, p. 54 middle, PIM-HBM can support virtualization allowing a processor to independently control PIM operations of each memory channel);
based on a result of the determining, determining whether to use the memory processor, comprising stages (Section V.A. p. 49 after identifying such TF ops, it maps associated operand data to memory space in a PIM-friendly way and generates the PIM microkernel code, Section IV.B. Pipeline the PIM execution unit is divided into 5 pipeline stages, Section III.C.Instruction Format discussing stages of pipeline and execution) ;
based on a result of the determining whether to use the memory processor, individually determining whether to use the memory processor for each stage (Section VII.B. performance, PIM-HBM improves the performance of GEMV compared to HBM, PIM-HBM improves the performance of ADD by 1.6 over HBM, as batch size increases performance improvement with PIM-HBM decreases and does not improve performance if batch size is 4 or larger, X conclusion PIM reduced end-to end execution time of memory bound neural network kernels and applications); and
multiplying the first matrix and the second vector based on a result of the individually determining whether to use the memory processor (section II.A, fig 1(c), Section III.C. Data and Operation Types, GEMV, Section V.A third paragraph GEMV).
Lee does not explicitly disclose wherein the floating-point operation is a multiplication of a first matrix and a second matrix that are double-precision floating-point matrices, or explicitly disclose determining whether an emulation is to be used, or performing based on the emulation.
Lee does, however, disclose a preprocessor that analyzes source code of applications, finding TF ops suitable for PIM acceleration at runtime, mapping associated operand data to memory space in a PIM-friendly way and generating PIM microkernel code, configuring and invoking a PIM kernel. However, in the same field of endeavor Kim discloses additional aspects of the apparatus of Lee (abstract, fig 4-6). Kim further discloses use of an emulator to confirm system-level performance and energy efficiency improvements (Kim Conclusion and Beyond). It would have been obvious to one of ordinary skill in the art before the effective filing date for the analysis performed by the preprocessor at runtime to include the emulator of Kim to determine whether an emulation is to be used to perform he floating-point operation, and based on the determining to determine whether to use the memory processor for emulation, and to perform the emulation according to the claim mapping above. It would have been obvious to achieve the benefit of determining and executing layers of a neural network that are memory-bound versus compute-bound and executing via PIM acceleration versus processor to achieve the benefit of a cost-effective way of increasing bandwidth and energy efficiency for neural network applications (Lee abstract, Section I Introduction, Section II.A).
Furthermore, Lee does disclose various floating point format precisions such as BFLOAT16, FP16, and FP32 in support of operations included in matrix multiplications such as MUL, ADD, MAC (Section III.C). Furthermore, in the same field of endeavor, Cho discloses an apparatus similar to Lee that includes PIM units for performing matrix multiplications (GEMM) for deep learning applications (abstract, introduction). Cho further discloses performing GEMM with a vector oriented approach that splits the GEMM into multiple matrix-vector operations (Section 2 last paragraph). It would have bene obvious to one of ordinary skill in the art before the effective filing date for Lee to perform matrix multiplications by using Lee’s emulator to split the GEMM operation into multiple GEMV operations. It would have been obvious to achieve the benefit of using Lee’s architecture to perform GEMM operations to accelerate the memory-bound operation by using the GEMV TF operation supported by Lee (Section V.A).
Furthermore, in the same field of endeavor Mukunoki discloses a method for implementing matrix multiplication on double-precision floating point matrices using the Ozaki scheme (abstract). Mukunoki further discloses the Ozaki scheme as splitting a double-precision vector into several FP16 vectors, computing the inner product therefrom and summing the inner products to perform the double-precision matrix multiplication (section 3.1). It would have been obvious to one of ordinary skill in the art before the effective filing date for Lee in view of Cho to use Mukunoki’s scheme to split the double-precision vectors into several FP16 vectors to perform the matrix multiplication, thereby using Lee’s 16b FPU PIM datapath (Lee Section III.A,C, fig 1(c)). It would have been obvious to achieve the benefit of using existing cublas routines using TF operations (Mukunoki abstract).
Regarding claim 2, Lee in view of Cho in view of Mukunoki teach the claim 1 limitations. As in the claim 1 mapping, Lee does not explicitly disclose supporting a double-precision floating-point operation. However, furthermore as in the claim 1 mapping Mukunoki discloses supporting a double-precision floating-point operation by using the Ozaki scheme. It would have been obvious to one of ordinary skill in the art before the effective filing date to include in the emulator of Lee, the determining whether the emulation is to be used is based on whether an electronic device supports a double-precision floating-point operation. It would have been obvious to use the emulator of Lee to support the Ozaki scheme of Mukunoki.
Regarding claim 3, Lee in view of Cho in view of Mukunoki teach the claim 1 limitations. Furthermore Mukunoki discloses:
wherein the stages comprise a splitting stage, a matrix multiplication operation stage, and a summation stage (Mukinoki section 3.1).
The motivation to combine provided with respect to claim 1 applies equally to claim 3.
Regarding claim 4, Lee in view of Cho in view of Mukunoki teach the claim 3 limitations. Furthermore Mukunoki discloses:
the splitting stage comprises splitting the first matrix into a plurality of first sub-matrices and splitting the second matrix into a plurality of second sub-matrices (Section 3.1 step 1);
the matrix multiplication operation stage comprises calculating matrix products between the first sub-matrices and the second sub-matrices (section 3.1 step 2); and
the summation stage comprises summing the matrix products. (Mukinoki section 3.1 step 3).
The motivation to combine provided with respect to claim 1 applies equally to claim 4.
Regarding claim 8, Lee in view of Cho in view of Mukunoki teach the claim 1 limitations. Lee Further discloses various pipeline stages including MULT, ADD, MAC, and determining whether to use the memory based processor for the emulation as in the claim 1 mapping. Furthermore Mukinoki discloses the splitting stage comprises splitting the first matrix into a plurality of first sub-matrices and splitting the second matrix into a plurality of second sub-matrices (Section 3.1 step 1); the matrix multiplication operation stage comprises calculating matrix products between the first sub-matrices and the second sub-matrices (section 3.1 step 2); and the summation stage comprises summing the matrix products. (Mukinoki section 3.1 step 3). The motivation to combine Mukinoki provided in claim 1 applies equally to claim 8 Mukinoki performing the above stages. Furthermore, it would have been obvious to one of ordinary skill in the art before the effective filing date to include in the emulator of Lee, determining whether to use the memory processor in a split stage; determining whether to use the memory processor in a matrix multiplication operation stage; or determining whether to use the memory processor in a summation stage. It would have been obvious to use the emulator of Lee to support the Ozaki scheme of Mukunoki.
Regarding claim 14, Lee in view of Cho in view of Mukunoki teach the claim 1 limitations. Mukunoki further discloses wherein
the first matrix and the second matrix correspond to 64-bit floating point (FP64) (section 3.1), and
a sub-matrix obtained by splitting the matrix corresponds to at least one of 32-bit floating point (FP32), 16-bit floating point (FP16), 16-bit brain floating point (BF16), or tensor-float-32 (TF32) (section 3.1 FP16).
The motivation to combine provided with respect to claim 1 applies equally to claim 14.
Claim 15 is directed to a non-transitory computer-readable storage medium storing instructions that, when executed by a processor cause the processor to perform the method of claim 1. All steps recited in the computer-readable storage medium of claim 15 that when executed by a processor cause the processor to perform all steps recited in the method of claim 1. The claim 1 analysis applies equally to claim 15.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Lee in view of Cho in view of Mukunoki in view of (hereinafter “Kim”).
Regarding claim 13, Lee in view Kim in view of Cho in view of Mukunoki teach the claim 1 limitations. Kim further discloses controlling at least one memory processor through a direct memory access (DMA) (fig 13). It would have been obvious to one of ordinary skill in the art before the effective filing date to use the evaluation architecture of Kim in the apparatus of Lee control the at least one memory processor through a direct memory access (DMA) when the memory processor is used for emulation when multiplying the first matrix and the second matrix. It would have been obvious to achieve the benefit of confirming the system level performance and energy efficiency improvements when evaluation is performed in a Xilinx system as in figure 13 of Kim.
Claims 16-19 are rejected under 35 U.S.C. 103 as being unpatentable over Lee in view of Kim in view of Cho.
Regarding claim 16, Lee teaches the following:
an accelerator for performing a floating-point operation by receiving a floating-point operation request from a processor,(abstract, fig 1, section III.A, SIMD floating-point unit (FPU), section III.C. Data Operation Types including GEMV- matrix vector multiplication Section III.A. last paragraph PIM instruction execution includes receiving an operation request, Section III.C ISA includes floating point operations), the accelerator comprising:
an accelerator core (fig 1 PIM unit, including FPUs);
a memory system (fig 1 HBM); and
a memory processor included in the memory system (fig 1PIM unit, including FPUs),
wherein the accelerator is configured to:
determine the processing is to be used to perform the floating-point operation (Section V.A p. 49 right column, PIM preprocessor analyzes the source code of applications and finds TensorFlow (TF) ops suitable for PIM acceleration at runtime, section VIII, p. 54 middle, PIM-HBM can support virtualization allowing a processor to independently control PIM operations of each memory channel);
based on a result of the determining, determining whether to use the memory processor, comprising stages (Section V.A. p. 49 after identifying such TF ops, it maps associated operand data to memory space in a PIM-friendly way and generates the PIM microkernel code, Section IV.B. Pipeline the PIM execution unit is divided into 5 pipeline stages, Section III.C.Instruction Format discussing stages of pipeline and execution) ;
based on a result of the determining whether to use the memory processor, individually determining whether to use the memory processor for each stage (Section VII.B. performance, PIM-HBM improves the performance of GEMV compared to HBM, PIM-HBM improves the performance of ADD by 1.6 over HBM, as batch size increases performance improvement with PIM-HBM decreases and does not improve performance if batch size is 4 or larger, X conclusion PIM reduced end-to end execution time of memory bound neural network kernels and applications); and
multiplying the first matrix and the second vector based on a result of the individually determining whether to use the memory processor (section II.A, fig 1(c), Section III.C. Data and Operation Types, GEMV, Section V.A third paragraph GEMV).
Lee does not explicitly disclose wherein the floating-point operation is a multiplication of a first matrix and a second matrix, or explicitly disclose determining whether an emulation is to be used, or performing based on the emulation.
Lee does, however, disclose a preprocessor that analyzes source code of applications, finding TF ops suitable for PIM acceleration at runtime, mapping associated operand data to memory space in a PIM-friendly way and generating PIM microkernel code, configuring and invoking a PIM kernel. However, in the same field of endeavor Kim discloses additional aspects of the apparatus of Lee (abstract, fig 4-6). Kim further discloses use of an emulator to confirm system-level performance and energy efficiency improvements (Kim Conclusion and Beyond). It would have been obvious to one of ordinary skill in the art before the effective filing date for the analysis performed by the preprocessor at runtime to include the emulator of Kim to determine whether an emulation is to be used to perform he floating-point operation, and based on the determining to determine whether to use the memory processor for emulation, and to perform the emulation according to the claim mapping above. It would have been obvious to achieve the benefit of determining and executing layers of a neural network that are memory-bound versus compute-bound and executing via PIM acceleration versus processor to achieve the benefit of a cost-effective way of increasing bandwidth and energy efficiency for neural network applications (Lee abstract, Section I Introduction, Section II.A).
Furthermore, Lee does disclose various floating point format precisions such as BFLOAT16, FP16, and FP32 in support of operations included in matrix multiplications such as MUL, ADD, MAC (Section III.C). Furthermore, in the same field of endeavor, Cho discloses an apparatus similar to Lee that includes PIM units for performing matrix multiplications (GEMM) for deep learning applications (abstract, introduction). Cho further discloses performing GEMM with a vector oriented approach that splits the GEMM into multiple matrix-vector operations (Section 2 last paragraph). It would have bene obvious to one of ordinary skill in the art before the effective filing date for Lee to perform matrix multiplications by using Lee’s emulator to split the GEMM operation into multiple GEMV operations. It would have been obvious to achieve the benefit of using Lee’s architecture to perform GEMM operations to accelerate the memory-bound operation by using the GEMV TF operation supported by Lee (Section V.A).
Regarding claim 17, Lee teaches the following:
a computing device for a floating point operation,(abstract, fig 1, section III.A, SIMD floating-point unit (FPU), section III.C. Data Operation Types including GEMV- matrix vector multiplication ), the computing device comprising:
a processor (fig 1 PIM unit, including FPUs);
a memory (fig 1 HBM); and
a memory processor included in the memory system (fig 1PIM unit, including FPUs),
wherein the processor, for performing the floating-point operation, is configured to:
determine the processing is to be used to perform the floating-point operation (Section V.A p. 49 right column, PIM preprocessor analyzes the source code of applications and finds TensorFlow (TF) ops suitable for PIM acceleration at runtime, section VIII, p. 54 middle, PIM-HBM can support virtualization allowing a processor to independently control PIM operations of each memory channel);
based on a result of the determining, determining whether to use the memory processor, comprising stages (Section V.A. p. 49 after identifying such TF ops, it maps associated operand data to memory space in a PIM-friendly way and generates the PIM microkernel code, Section IV.B. Pipeline the PIM execution unit is divided into 5 pipeline stages, Section III.C.Instruction Format discussing stages of pipeline and execution) ;
based on a result of the determining whether to use the memory processor, individually determining whether to use the memory processor for each stage (Section VII.B. performance, PIM-HBM improves the performance of GEMV compared to HBM, PIM-HBM improves the performance of ADD by 1.6 over HBM, as batch size increases performance improvement with PIM-HBM decreases and does not improve performance if batch size is 4 or larger, X conclusion PIM reduced end-to end execution time of memory bound neural network kernels and applications); and
multiplying the first matrix and the second vector based on a result of the individually determining whether to use the memory processor (section II.A, fig 1(c), Section III.C. Data and Operation Types, GEMV, Section V.A third paragraph GEMV).
Lee does not explicitly disclose wherein the floating-point operation is a multiplication of a first matrix and a second matrix, or explicitly disclose determining whether an emulation is to be used, or performing based on the emulation.
Lee does, however, disclose a preprocessor that analyzes source code of applications, finding TF ops suitable for PIM acceleration at runtime, mapping associated operand data to memory space in a PIM-friendly way and generating PIM microkernel code, configuring and invoking a PIM kernel. However, in the same field of endeavor Kim discloses use of an emulator to confirm system-level performance and energy efficiency improvements (Kim Conclusion and Beyond). It would have been obvious to one of ordinary skill in the art before the effective filing date for the analysis performed by the preprocessor at runtime to include the emulator of Kim to determine whether an emulation is to be used to perform he floating-point operation, and based on the determining to determine whether to use the memory processor for emulation, and to perform the emulation according to the claim mapping above. It would have been obvious to achieve the benefit of determining and executing layers of a neural network that are memory-bound versus compute-bound and executing via PIM acceleration versus processor to achieve the benefit of a cost-effective way of increasing bandwidth and energy efficiency for neural network applications (Lee abstract, Section I Introduction, Section II.A).
Furthermore, Lee does disclose various floating point format precisions such as BFLOAT16, FP16, and FP32 in support of operations included in matrix multiplications such as MUL, ADD, MAC (Section III.C). Furthermore, in the same field of endeavor, Cho discloses an apparatus similar to Lee that includes PIM units for performing matrix multiplications (GEMM) for deep learning applications (abstract, introduction). Cho further discloses performing GEMM with a vector oriented approach that splits the GEMM into multiple matrix-vector operations (Section 2 last paragraph). It would have bene obvious to one of ordinary skill in the art before the effective filing date for Lee to perform matrix multiplications by using Lee’s emulator to split the GEMM operation into multiple GEMV operations. It would have been obvious to achieve the benefit of using Lee’s architecture to perform GEMM operations to accelerate the memory-bound operation by using the GEMV TF operation supported by Lee (Section V.A).
Regarding claim 18, in addition to the teachings addressed in the claim 17 analysis, Lee teaches the following:
wherein the memory comprises a memory chip, and wherein the memory chip comprises a memory portion and the memory processor (fig 1-2).
Regarding claim 19, in addition to the teachings addressed in the claim 18 analysis, Lee teaches the following:
wherein the memory is configured such that the memory processor is capable of performing the multiplying on the first matrix and the second matrix stored while they first matrix and second matrix are stored in the memory portion (section III.A.B All Bank Mode, All Bank PIM Mode)
.
Allowable Subject Matter
Claims 5-7 and 9-12 would be allowable if rewritten to overcome the rejections under 35 USC 101, and rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter.
Applicant claims methods for performing a floating-point operation using a memory processor, wherein the method as in claim 1 comprises:
A method of performing a floating-point operation using a memory processor, the floating-point operation being a multiplication of a first matrix and a second matrix that are double-precision floating-point matrices, the method comprising:
determining whether an emulation is to be used to perform the floating-point operation; based on a result of the determining whether the emulation is to be used, determining whether to use the memory processor for the emulation, the emulation comprising stages;
based on a result of the determining whether to use the memory processor for the emulation, individually determining whether to use the memory processor for each stage of the emulation; and
multiplying the first matrix and the second matrix based on a result of the individually determining whether to use the memory processor.
Furthermore, dependent claims including allowable subject matter comprise:
Claim 5. The method of claim 1, wherein the determining of whether to use the memory processor for the emulation is based on at least one of a size of a matrix, a size of a sub-matrix, or a number of sub-matrices, the size of the matrix comprises at least one of a size of the first matrix or a size of the second matrix, and the number of sub-matrices is determined based on at least one of a number of first sub- matrices or a number of second sub-matrices.
Claim 9. The method of claim 8, wherein the determining of whether to use the memory processor in the split stage is based on a comparison between a size of a sub-matrix and a memory bandwidth.
Claim 11. The method of claim 8, wherein the determining of whether to use the memory processor in the matrix multiplication operation stage is based on at least one of a number of sub-matrices or floating-point operations per second (FLOPS).
Claim 12. The method of claim 8, wherein the determining of whether to use the memory processor in the summation stage is based on a comparison between a size of a sub-matrix and a memory bandwidth.
The primary reasons for indication of allowable subject matter is the limitations highlighted above in combination with the remaining limitations. Lee, Kim, Cho, and Mukunoki disclose the claimed invention according to the above claim mappings. Lee and Kim disclose emulation determinations generally, but do not teach or suggest the specific bases for the determining steps as in claims 5, 9, 11 and 12 with no specific discussion of determination based on matrix or submatrix size, or FLOPS, or memory bandwidth or matrix or submatrix size with respect to specific stages of operations such as splitting or multiplication stages.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to EMILY E LAROCQUE whose telephone number is (469)295-9289. The examiner can normally be reached on 10:00am - 1200pm, 2:00pm - 8pm ET M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor Andrew Caldwell can be reached on 571-272-3701. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/EMILY E LAROCQUE/Examiner, Art Unit 2182
Prosecution Insights