DETAILED ACTION
This action is responsive to the amendment filed on 01/31/2022. Claims 1-3, 6-9, 11-15, 18-20 are pending in the case. Claims 1, 9, and 18 are independent claims. Claims 1, 6, 9, 11, 15, 18, 19, 20 are amended.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 1-17, 19-20 rejected under 35 U.S.C. 101 because the claim are directed to an abstract idea without significantly more.
Claim 19 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim(s) does/do not fall within at least one of the four categories of patent eligible subject matter because the claim is directed to transitory storage medium, i.e signals per se.
Examiner notes specification paragraph 0257 describes the storage medium as any well known medium in the art, therefore the claim encompasses both transitory and non-transitory media. Because the claim includes signals, the claim is directed to non-statutory subject matter.
Regarding Claim 1/11/19/20:
Under step 1, claim 1 is directed to a method which is directed to a process, one of the statutory categories.
Under step 1, claim 11 is directed to an apparatus which is directed to a machine, one of the statutory categories.
Under step 1, claim 19 is directed to transitory storage medium, which is not one of the statutory categories
Under step 1, claim 20 is directed to an apparatus which is directed to machine, one of the statutory categories.
Under Step 2A Prong 1, the claim recites the following limitations which are considered mental evaluations:
to perform calculation on first target data having the first data shape.
, the second data shape of each of the at least two calculating units is combined based on the combination information… and a data length in any dimension obtained after the second data shape of each calculating unit is combined is greater than or equal to a data length of the first data shape in the same dimension
Calculations on abstract data is an evaluation which can be performed in the human mind. Further combining data based on information is similarly an evaluation made in the mind. Therefore, the claim recites an abstract idea
Under step 2A Prong 2, The claim recites the following additional element(s):
and invoking the at least two calculating units (which amounts to descriptions which makes use of or applies the abstract idea because under 2106.05(f)(1) “the claim fails to recite details of how a solution to a problem is accomplished”, as no details of the technical function of the calculating units is described.)
from claim 11: at least one processor,: and at least one memory storing instructions that, when executed by the at least one processor, cause the operator calculation apparatus to perform operations comprising (which amounts to descriptions which makes use of or applies the abstract idea because under 2106.05(f)(1) “the claim fails to recite details of how a solution to a problem is accomplished”)
from claim 19: wherein the computer storage medium stores instructions, and when the instructions are run on a computer, the computer is enabled to perform operations (which amounts to descriptions which makes use of or applies the abstract idea because under 2106.05(f)(1) “the claim fails to recite details of how a solution to a problem is accomplished”)
from claim 20: the interface is configured to provide program instructions or data for the at least one processor; and the at least one processor is configured to execute the program instructions (which amounts to descriptions which makes use of or applies the abstract idea because under 2106.05(f)(1) “the claim fails to recite details of how a solution to a problem is accomplished”)
obtaining parameter data of a first data shape of an artificial intelligence AI network (that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP 2106.05(g) )
wherein the first data shape is represented by a data length in each dimension that is supported by the Al network for processing, the parameter data comprises combination information of at least two calculating units, data that is supported by each of the at least two calculating units for processing is data having a second data shape, …, (amounts to generally linking the use of the judicial exception to a particular technological environment or field of use. The limitation is merely an “incidental or token addition to the claim that did not alter or affect how” the claimed steps are performed, see MPEP 2106.05(h))
Therefore, the claim is directed to a judicial exception.
Further, additional element obtaining parameter data of a first data shape of an artificial intelligence AI network is well understood, routine, and conventional activity because it amounts to “transmitting or receiving data over a network" (see MPEP 2106.05(d)(II)(i) )
Regarding Claim 2/12
The claim depends upon claim 1/11
The claim does not recite further abstract idea to consider, beyond those recited in the parent claim.
The claim recites the following additional element(s), in addition to those already identified in the parent claim:
wherein the at least two calculating units comprise same calculating units, or different calculating units, or a combination of same calculating units and different calculating units; and second data shapes of the same calculating units have a same data length in each dimension, and second data shapes of the different calculating units have different data lengths in at least one dimension (amounts to generally linking the use of the judicial exception to a particular technological environment or field of use. The limitation is merely an “incidental or token addition to the claim that did not alter or affect how” the claimed steps are performed, see MPEP 2106.05(h))
Regarding Claim 3/13
The claim depends upon claim 1/11
The claim does not recite further abstract idea to consider, beyond those recited in the parent claim.
The claim recites the following additional element(s), in addition to those already identified in the parent claim:
wherein the at least two calculating units each are a calculating unit of the Al network. (amounts to generally linking the use of the judicial exception to a particular technological environment or field of use. The limitation is merely an “incidental or token addition to the claim that did not alter or affect how” the claimed steps are performed, see MPEP 2106.05(h))
Regarding Claim 4/14
The claim depends upon claim 1/11
Under Step 2A Prong 1, The claim recites the limitations:
and a data length in any dimension obtained after the second data shape of each calculating unit is combined based on the combination mode is greater than or equal to a data length of the first data shape in a same dimension
which further describe the abstract ideas recited in the parent claims, under Step 2A Prong 1, in particular the limitations describe mental evaluations.
The claim recites the following additional element(s), in addition to those already identified in the parent claim:
wherein the combination information comprises a combination mode of the at least two calculating units (amounts to generally linking the use of the judicial exception to a particular technological environment or field of use. The limitation is merely an “incidental or token addition to the claim that did not alter or affect how” the claimed steps are performed, see MPEP 2106.05(h))
Regarding Claim 5/15
The claim depends upon claim 1/11
The claim does not recite further abstract idea to consider, beyond those recited in the parent claim.
The claim recites the following additional element(s), in addition to those already identified in the parent claim:
wherein the parameter data further comprises identification information for a specified calculating unit in the at least two calculating units, whose data that needs to be processed is data having a third data shape, and a data length of the third data shape in at least one dimension is less than a data length of the second data shape that is supported by the specified calculating unit for processing and that is in the same dimension. (amounts to generally linking the use of the judicial exception to a particular technological environment or field of use. The limitation is merely an “incidental or token addition to the claim that did not alter or affect how” the claimed steps are performed, see MPEP 2106.05(h))
Regarding Claim 6/16
The claim depends upon claim 1/11
The claim does not recite further abstract idea to consider, beyond those recited in the parent claim.
The claim recites the following additional element(s), in addition to those already identified in the parent claim:
wherein the parameter data comprises rank parameter data for supporting a data shape in a specified change range. (amounts to generally linking the use of the judicial exception to a particular technological environment or field of use. The limitation is merely an “incidental or token addition to the claim that did not alter or affect how” the claimed steps are performed, see MPEP 2106.05(h))
Regarding Claim 7/17
The claim depends upon claim 1/11
Under Step 2A Prong 1, The claim recites the limitations:
performing… calculation on the first target data having the first data shape.
which further describe the abstract ideas recited in the parent claims, under Step 2A Prong 1, in particular the limitations describe mental evaluations.
The claim recites the following additional element(s), in addition to those already identified in the parent claim:
by using the at least two calculating units (which amounts to descriptions which makes use of or applies the abstract idea because under 2106.05(f)(1) “the claim fails to recite details of how a solution to a problem is accomplished”, as no details of the technical function of the calculating units is described.)
obtaining the at least two calculating units from a calculating unit operator library; (that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP 2106.05(g) )
Therefore, the claim is directed to a judicial exception.
Further, additional element obtaining the at least two calculating units from a calculating unit operator library is well understood, routine, and conventional activity because it amounts to “transmitting or receiving data over a network" (see MPEP 2106.05(d)(II)(i) )
Regarding Claim 8
The claim depends upon claim 1
Under Step 2A Prong 1, The claim recites the limitations:
for each of the at least two calculating units, determining a target location, in the first target data, of second target data that needs to be processed … and performing calculation on the second target data
which further describe the abstract ideas recited in the parent claims, under Step 2A Prong 1, in particular the limitations describe mental evaluations.
The claim recites the following additional element(s), in addition to those already identified in the parent claim:
by the calculating unit… by using the calculating unit.(which amounts to descriptions which makes use of or applies the abstract idea because under 2106.05(f)(1) “the claim fails to recite details of how a solution to a problem is accomplished”, as no details of the technical function of the calculating units is described.)
obtaining, based on the target location, the second target data that needs to be processed by the calculating unit from memory space storing the first target data (that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP 2106.05(g) )
Therefore, the claim is directed to a judicial exception.
Further, additional element obtaining, based on the target location, the second target data that needs to be processed by the calculating unit from memory space storing the first target data is well understood, routine, and conventional activity because it amounts to “Storing and retrieving information in memory" (see MPEP 2106.05(d)(II)(ii))
Regarding Claim 9
The claim depends upon claim 1
The claim does not recite further abstract idea to consider, beyond those recited in the parent claim.
The claim recites the following additional element(s), in addition to those already identified in the parent claim:
wherein the at least two calculating units belong to different types of operators. (amounts to generally linking the use of the judicial exception to a particular technological environment or field of use. The limitation is merely an “incidental or token addition to the claim that did not alter or affect how” the claimed steps are performed, see MPEP 2106.05(h))
Regarding Claim 10
The claim depends upon claim 1
The claim does not recite further abstract idea to consider, beyond those recited in the parent claim.
The claim recites the following additional element(s), in addition to those already identified in the parent claim:
the calculating unit is a precompiled operator. (amounts to generally linking the use of the judicial exception to a particular technological environment or field of use. The limitation is merely an “incidental or token addition to the claim that did not alter or affect how” the claimed steps are performed, see MPEP 2106.05(h))
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 4 and 8 and 14 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 4 and 14 recites the limitation "the combination mode". There is insufficient antecedent basis for this limitation in the claim. Examiner notes the claim should likely recite “a combination mode”. Further, it is unclear from the context of the claim how a “combination mode” could be construed being “greater than or equal to a data length of the first data shape in a same dimension”. These two claim entities are not comparable. For purposes of examination, this phrase is understood to describe the claimed “data length”.
Claim 8 recites the limitation "for each of the at least two calculating units, determining a target location… needs to be processed by the calculation unit…by using the calculating unit". There is insufficient antecedent basis for this limitation in the claim. Examiner notes the claim does not delineate which of “the each calculating units” need to be processed and/or used. For the purposes of examination, the claim is understood as “for each of the at least two calculating units, determining a target location… needs to be processed by a respective calculation unit… by a respective calculating unit”
Claim 10 recites the limitation "the calculating unit". There is insufficient antecedent basis for this limitation in the claim. Examiner notes the claim describes at least two calculation units. It is not clear from the claim which calculation unit is being referred to. For purposes of examination, the claim is understood to describe at least on of the two calculating units.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-17, 19-20 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Chen “TVM: An Automated End-to-End Optimizing Compiler for Deep Learning”
Regarding claim 1
Chen Teaches, An operator calculation method, obtaining parameter data of a first data shape of an artificial intelligence AI network, wherein the first data shape is represented by a data length in each dimension that is supported by the Al network for processing, (Section 2 pg 3 “This section describes TVM…The system first takes as input a model from an existing framework and transforms it into a computational graph representation.” Section 3 pg 4 “Computational graphs are a common way to represent programs in DL [deep learning] frameworks…Figure 3 shows an example” Figure 3 citation pg 4 “Figure 3: Example computational graph of a two-layer convolutional neural network.
PNG
media_image1.png
166
537
media_image1.png
Greyscale
the figure shows the attributes, for example the conv2d attribute has a first data shape i.e 3x3 which represents a supported length in each dimension for processing. The DL or deep learning framework is a type of artificial intelligence AI network)the parameter data comprises combination information of at least two calculating units, data that is supported by each of the at least two calculating units for processing is data having a second data shape, the second data shape of each of the at least two calculating units is combined based on the combination information, and a data length in any dimension obtained after the second data shape of each calculating unit is combined is greater than or equal to a data length of the first data shape in the same dimension; (See figure 3, the graph comprises at least two operations or calculating units for combining data, the second data shape as a shape of 1,10 which is greater than the first dimension in at least one dimension.) and invoking the at least two calculating units to perform calculation on first target data having the first data shape. (pg 11 Section 6.1 “Figure 14 shows that TVM outperforms the base lines, with speed ups ranging from 1.6× to 3.8× due to both joint graph optimization and the automatic optimizer” the resulting TVM graph is invoked to assess performance.)
Regarding claim 2
Chen teaches claim 1
Chen Teaches, wherein the at least two calculating units comprise same calculating units, or different calculating units, or a combination of same calculating units and different calculating units; and second data shapes of the same calculating units have a same data length in each dimension, and second data shapes of the different calculating units have different data lengths in at least one dimension. ( pg 4 figure 3 caption “Example computational graph of a two-layer convolutional neural network. Each node in the graph represents an operation that consumes one or more tensors and produces one or more tensors” as previously states the softmax and conv2d are two operation which are different and operate on different dimension shapes. A calculating unit doing the same operation with the same data shape size is by definition the same, while a calculating unit with either the same operation or a different shape size is different.)
Regarding claim 3
Chen teaches claim 1
Chen Teaches, wherein the at least two calculating units each are a calculating unit of the Al network. (Figure 3 citation pg 4 “Figure 3: Example computational graph of a two-layer convolutional neural network.
PNG
media_image1.png
166
537
media_image1.png
Greyscale
the figure shows the attributes of different calculating units, they are of a convolutional AI neural network.)
Regarding claim 4
Chen teaches claim 1
Chen Teaches, wherein the combination information comprises a combination mode of the at least two calculating units; (pg 4-5 Section 3 “Operator fusion combines multiple operators into a single kernel without saving the intermediate results in memory… We can apply these rules to transform the computational graph into a fused version” a fused operator is the combination information of the operators or calculating units.) and a data length in any dimension obtained after the second data shape of each calculating unit is combined based on the combination mode is greater than or equal to a data length of the first data shape in a same dimension. (pg 4 “TVM exploits a computational graph representation to apply high-level optimizations: a node represents an operation on tensors or program inputs, and edges represent data dependencies between operations. It implements many graph-level optimizations, including: operator fusion, which fuses multiple small operations together” Operator fusion combines the operators such that the combined operator is greater than the original small operation, the fusion happens after receiving the first and second initial shape information)
Regarding claim 5
Chen teaches claim 1
Chen Teaches, wherein the parameter data further comprises identification information for a specified calculating unit in the at least two calculating units, whose data that needs to be processed is data having a third data shape, and a data length of the third data shape in at least one dimension is less than a data length of the second data shape that is supported by the specified calculating unit for processing and that is in the same dimension. (pg 9 Section 4.1 “The following code shows an example tensor expression to compute transposed matrix multiplication…
PNG
media_image2.png
102
392
media_image2.png
Greyscale
here each tensor placeholder is a calculating unit which comprising a name which identifies the unit. the data associated with k has at least one dimension which is 0, i.e smaller than a second data shape.)
Regarding claim 6
Chen teaches claim 1
Chen Teaches, wherein the parameter data comprises rank parameter data for supporting a data shape in a specified change range. Figure 3 citation pg 4 “Figure 3: Example computational graph of a two-layer convolutional neural network.
PNG
media_image1.png
166
537
media_image1.png
Greyscale
the figure shows the attributes the rank of the conv2d operation which is the specified change range of the data shape.)
Regarding claim 7
Chen teaches claim 1
Chen Teaches, wherein the invoking the at least two calculating units to perform calculation on first target data having the first data shape comprises: obtaining the at least two calculating units from a calculating unit operator library;(pg 11 Table 2 “Configurations of all conv2d operators in ResNet-18 and all depth wise conv2d operators in MobileNet used in the single kernel experiments” pg 14 “We used TVM to generate ResNet inference kernels on the PYNQ plat form and offloaded as many layers as possible to VDLA” the TVM generated kernels are according to the set of possible conv2d operators, therefore from an operator library.) and performing, by using the at least two calculating units, calculation on the first target data having the first data shape. (pg 14 “Figure 21 breaks down ResNet inference time into CPU-only execution and CPU+FPGA execution. Most computation was spent on the convolution layers that could be offloaded to VDLA.” Inference time refers to the time to perform calculation using the calculating units on data having a first data shape, which is first target data)
Regarding claim 8
Chen teaches claim 1
Chen Teaches,
calculating units to perform calculation on first target data having the first data shape comprises: for each of the at least two calculating units, determining a target location, in the first target data, of second target data that needs to be processed by the calculating unit; obtaining, based on the target location, the second target data that needs to be processed by the calculating unit from memory space storing the first target data; (pg 6-7 “Specifically, groups of threads can cooperatively fetch the data they all need and place it into a shared memory space… GPU memory hierarchy and enable data reuse across threads through shared memory regions. TVM supports this well-known GPU optimization using a schedule primitive to achieve optimal performance. The following GPU code example optimizes matrix multiplication… memory synchronization barriers must be properly inserted to guarantee that shared loaded data is visible to consumers.
PNG
media_image3.png
175
377
media_image3.png
Greyscale
as shown the location of the first data is shared for the plurality of computation units such that the first target data of the second target data is from memory space storing the first target data. This is ensured by the system determining the memory barrier for the threads.) and performing calculation on the second target data by using the calculating unit. (pg 14 “Figure 21 breaks down ResNet inference time into CPU-only execution and CPU+FPGA execution. Most computation was spent on the convolution layers that could be offloaded to VDLA.” The resulting neural network optimization is used to measure inference time.)
Regarding claim 9
Chen teaches claim 1
Chen Teaches, the at least two calculating units belong to different types of operators. “Figure 3: Example computational graph of a two-layer convolutional neural network.
PNG
media_image1.png
166
537
media_image1.png
Greyscale
the figure shows different operators)
Regarding claim 10
Chen teaches claim 1
Chen teaches, the calculating unit is a precompiled operator. (pg 4 “In a few lines of code, a user can take a model from existing deep learning frameworks and call the TVM API to get a deployable module… This compiled runtime module contains three components… These components can then be used to deploy the model to the target back-end:” the TVM API creates compiled operators which are considered precompiled operators to be executed on a target back end.)
Regarding claim 11
Chen teaches the common limitations described in claim 1
Further, Chen teaches, comprising at least one processor,: and at least one memory storing instructions that, when executed by the at least one processor, cause the operator calculation apparatus to perform operations comprising (pg 11 Section 6 “,we evaluated TVM on four types of platforms: (1) a server-class GPU, (2)an embedded GPU, (3) an embedded CPU, and (4) a DL accelerator implemented on a low-power FPGA SoC.”)
Regarding claim 12-17
Chen teaches claim 11
These claims 12-17 are rejected for the reasons previously stated in the rejection of claim 2-7
Regarding claim 19
Chen teaches the common limitations described in claim 1
Further, Chen teaches, A computer storage medium, wherein the computer storage medium stores instructions, and when the instructions are run on a computer, the computer is enabled to perform operations (pg 11 Section 6 “,we evaluated TVM on four types of platforms: (1) a server-class GPU, (2)an embedded GPU, (3) an embedded CPU, and (4) a DL accelerator implemented on a low-power FPGA SoC.”)
Regarding claim 20
Chen teaches the common limitations described in claim 1
Further, Chen teaches, A chip, comprising at least one processor and an interface, wherein the interface is configured to provide program instructions or data for the at least one processor; and the at least one processor is configured to execute the program instructions, (pg 11 Section 6 “,we evaluated TVM on four types of platforms: (1) a server-class GPU, (2)an embedded GPU, (3) an embedded CPU, and (4) a DL accelerator implemented on a low-power FPGA SoC.”)
Conclusion
Prior art:
Cowen et el. “Automating Generation of Low Precision Deep Learning Operators” describes optimized code for neural network scheduling
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOHNATHAN R GERMICK whose telephone number is (571)272-8363. The examiner can normally be reached M-F 9:30-4:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kakali Chaki can be reached on 571-272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.R.G./
Examiner, Art Unit 2122
/KAKALI CHAKI/Supervisory Patent Examiner, Art Unit 2122
Prosecution Insights