DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after allowance or after an Office action under Ex Parte Quayle, 25 USPQ 74, 453 O.G. 213 (Comm'r Pat. 1935). Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, prosecution in this application has been reopened pursuant to 37 CFR 1.114. Applicant's submission filed on 1/12/2026 has been entered.
No claims have been amended, canceled, or added. Claims 1-15 and 54-55 have been examined.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1 and 7-8 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by U.S. Patent Application Publication 20180121196 by Temam et al. ("Temam").
In regard to claim 1 Temam discloses:
1. A computer-implemented method for a conditional execution of an artificial neural network (ANN) comprising: See Temam, Fig. 6, broadly depicting a method.
storing, in a memory, a first data tile, wherein the first data tile: (i) holds a set of ANN data elements of the artificial neural network; Temam, ¶ 0005, “A tensor accessible from narrow memory and wide memory units, in a single compute tile, is traversed based on memory address values retrieved from registers.” ¶ 0019, “the layers of the neural network are arranged … in a directed graph.” Also ¶ 0073, “Activation values are stored in a narrow memory 210.”
(ii) is larger than a single ANN data element; and … ; Temam, ¶ 0060-0061, “FIG. 3 illustrates an example Tensor Traversal Unit (TTU) structure 300 comprising four tensors to track each having a depth of eight. … Accordingly, as discussed above, there are two sets of TTUs in compute tile 200: 1) TensorOp TTU 226; and 2) DMAOp TTU 228. In various implementations, TensorOp control 206 will cause TTU 300 to load TensorOp TTU counter 302, limit 308, and”
(iii) is smaller than a layer of the ANN. Temam, ¶ 0026, “Each tile is an individual computing unit that cooperates with other tiles in the system to accelerate computations across one or more layers of a multi-layer neural network.”
generating metadata for the first data tile during an execution of the ANN and based on the data in the first data tile; storing the metadata, in association with the first data tile in the memory. Temam, ¶ 0031, “In one example, a header (i.e., a bitmap) of the instruction indicates, to a receiving tile, that the receiving tile needs to consume a particular instruction based on a bitmap associated with the instruction.” Also see ¶ 0038-0039, “As discussed above, ring bus 128 also includes a bitmap header indicating the tiles that need to consume payload data comprising instructions or parameters communicated via ring bus 128. ¶ With regard to data (i.e., payload) received at a particular tile via ring bus 128, in response to receiving the information, each tile will zero (i.e., clear out) position data indicated in the bitmap header that is unique to the receiving tile before forwarding the data on to another tile.” A first tile modifies the payload data header which is then sent for subsequent execution to other tiles.
fetching an instruction for execution by an execution engine, wherein execution of the instruction requires: (i) the set of ANN data elements from the first data tile; (ii) retrieving the first data tile from the memory; (iii) the metadata as stored in the memory; and (iv) a set of arithmetic logic operations; and Temam, ¶ 0055, “Arithmetic operations performed by the MAC operators of the MAC array 214 generally include multiplying an input activation provided by narrow memory 210 with a parameter accessed from wide memory 212 to produce a single output activation value.” Also see ¶ 0031 and 0038-0039 as cited above, e.g. “¶ 0031, “In one example, a header (i.e., a bitmap) of the instruction indicates, to a receiving tile, that the receiving tile needs to consume a particular instruction based on a bitmap associated with the instruction.”
conditionally executing the arithmetic logic operations from the set of arithmetic logic operations based on the metadata. Temam, ¶ 0031, “In one example, a header (i.e., a bitmap) of the instruction indicates, to a receiving tile, that the receiving tile needs to consume a particular instruction based on a bitmap associated with the instruction.”
In regard to claim 7 Temam also discloses:
7. The computer-implemented method of claim 1, further comprising: storing metadata from the metadata for the first data tile in a register of a control unit of an arithmetic logic unit; Temam ¶ 0031 “… a header (i.e., a bitmap) of the instruction … ” Also ¶ 0051, “TTU register 232 includes instruction buffers for storing one or more instructions comprising operations to be performed by TensorOp TTU 226 upon execution of the instructions by TensorOp control 206.”
wherein conditionally executing the arithmetic logic operations based on the metadata includes: (i) the control unit evaluating the metadata in the register; and (ii) the control unit suppressing transmission of the operation to the arithmetic logic unit based on the metadata in the register. Temam ¶ 0039, “Hence, when the header bitmap has no remaining bit set data indicating a particular tile that is to receive the payload, forwarding of the payload to another tile will stop.”
In regard to claim 8 Temam also discloses:
8. The computer-implemented method of claim 1, wherein: conditionally executing the arithmetic logic operation based on the metadata involves suppressing an arithmetic logic operation from the set of arithmetic logic operations. Temam, ¶ 0029, “If the node ID of the header does not indicate that the destination is the inspecting tile, the inspecting tile will copy the input CSR instruction packet to the CSR bus input connected to the next tile for inspection by the next tile.”
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Temam as applied above, and further in view of U.S. Patent 8736624 to Mahan et al. ("Mahan").
In regard to claim 2, Temam does not expressly disclose:
2. The computer-implemented method of claim 1, further comprising: evaluating the set of ANN data elements; wherein the generating of the metadata for the first data tile is based on the evaluating of the set of ANN data elements. This is taught by Mahan. See Mahan, col. 5, lines 26-30, “The conditional execution controller, in the depicted embodiment, is used to test pixel data packets as they are passed through the graphics pipeline, and determine which "branch" of instructions should be applied to that particular packet.” It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use Mahan’s data evaluation with Temam’s conditional instructions in order to avoid redundant or unnecessary operations as suggested by Mahan (see col. 5 lines 37-39).
Claims 3, 5-6, 9-10 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Temam as applied above, and further in view of U.S. Patent Application Publication 20180218518 by Yan et al. (“Yan”).
In regard to claim 3, Temam also discloses:
3. The computer-implemented method of claim 1, further comprising:
generating a set of output data from the conditional execution of the arithmetic logic operations; … storing, in the memory … a second data tile, wherein the second data tile: (i) holds the … set of output data; (ii) is larger than the single ANN data element; and (iii) is smaller than the layer of the ANN. Temam, ¶ 0046, “Each of tiles 112, 114 will store a subset of the input activations needed to compute a subset of output activations that are assigned to that particular tile.”
Temam does not expressly disclose:
compressing, using a compression engine, the set of output data; and storing … subsequent to the compressing, a second data tile, wherein the second data tile: (i) holds the compressed set of output data. However, this is taught by Yan. See Yan, Fig. 1A, depicting data compression. Also see ¶ 0021, e.g. “Additional benefits can be achieved by a compressed or compact encoding for sparse weights and/or activations that include several zeros, thus allowing more weight and/or activation values to fit in on-chip random access memory (RAM) and reducing the number of energy-costly dynamic random access memory (DRAM) accesses to read activations and weights. Furthermore, transmitting the compact encoding may reduce the number of transitions on buses, further reducing energy consumption. In one embodiment, a compacted data sequence for input to a PE is received by the DLA, where the compacted data sequence comprises at least one single bit signal indicating that at least one multi-bit value equals zero, and the single bit signal is transmitted to the PE in lieu of the multi -bit value.” It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use Temam’s data tiles and storage with Yan’s compacted representation in order to increase data capacity and reduce costly memory accesses as suggested by Yan.
In regard to claim 5, Temam does not expressly disclose:
5. The computer-implemented method of claim 3, wherein: a set of non-sparse data values of the output data are zeroes; and This is taught by Yan. See Yan, ¶ 0034, “Sparsity in a layer of a CNN is defined as the fraction of zeros in the layer's weight and input activation matrices.” Note that non-sparse is broadly interpreted as an inverse of sparsity. Thus, non-sparsity would also be a fraction of zeros. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use Bruestle’s data with Yan’s sparsity/non-sparsity in order to utilize a smaller network while maintaining accuracy as suggested by Yan (see ¶ 0034).
Temam also discloses:
conditionally executing the arithmetic logic operations based on the metadata involves suppressing an arithmetic logic operation from the set of arithmetic logic operations. Temam, ¶ 0029, “If the node ID of the header does not indicate that the destination is the inspecting tile, the inspecting tile will copy the input CSR instruction packet to the CSR bus input connected to the next tile for inspection by the next tile.”
In regard to claim 6, Temam also discloses:
6. The computer-implemented method of claim 1, wherein conditionally executing the arithmetic logic operations from the set of arithmetic logic operations based on the metadata comprises: suppressing an arithmetic logic operation from the set of arithmetic logic operations; and Temam, ¶ 0029, “If the node ID of the header does not indicate that the destination is the inspecting tile, the inspecting tile will copy the input CSR instruction packet to the CSR bus input connected to the next tile for inspection by the next tile.”
Temam does not expressly disclose:
providing a zero value in place of the arithmetic logic operation. However, Yan teaches this. See Yan, Fig. 1A, along with ¶ 0020, e.g. “a single bit signal is transmitted by the compaction engine to the PE in lieu of the multi-bit data, where the single bit signal indicates that the multi-bit data equals zero. … In one embodiment, the single bit is used by the PE to generate a zero as a product of a multiplication operation without performing the multiplication” Here, the single bit signal functions as a metadata flag. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use Temam’s logic operation with Yan’s zero output in order to reduce energy consumption as suggested by Yan.
In regard to claim 9 Temam does not expressly disclose:
9. The computer-implemented method of claim 1, wherein: the metadata includes at least two flags associated with at least two portions of the first data tile in a one-to-one correspondence. However, Yan teaches a flag to indicate that a section of data is zero. See Yan, Fig. 1A, along with ¶ 0020, e.g. “a single bit signal is transmitted by the compaction engine to the PE in lieu of the multi-bit data, where the single bit signal indicates that the multi-bit data equals zero. … In one embodiment, the single bit is used by the PE to generate a zero as a product of a multiplication operation without performing the multiplication” Here, the single bit signal functions as a metadata flag. Also see Fig. 3A, depicting a one-to-one correspondence of data 305 and flags 310. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use Temam’s header with Yan’s compacted representation in order to increase data capacity and reduce costly memory accesses as suggested by Yan.
In regard to claim 10 Temam does not expressly disclose:
10. The computer-implemented method of claim 1, wherein: the first data tile stores the set of ANN data elements in a compressed format; and the metadata includes at least two zero flags. However, this is taught by Yan. See Yan, Fig. 3A, element 300. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use Temam’s header with Yan’s compacted representation in order to increase data capacity and reduce costly memory accesses as suggested by Yan.
In regard to claim 13 Temam discloses:
13. The computer-implemented method of claim 1, wherein: the set of arithmetic logic operations are multiplications between the set of ANN data elements and a second set of ANN data elements; and See Temam, ¶ 0056, “The first portion is complete when multiply operations produce an output activation, for example, by completing a multiplication of an input activation and a parameter to generate the output activation.”
All further limitations of claim 13 have been addressed in the above rejection of claim 6.
Claims 4 and 55 are rejected under 35 U.S.C. 103 as being unpatentable over Temam as applied above, and further in view of Mahan and Yan.
In regard to claim 4, Temam does not expressly disclose: discloses:
4. The computer-implemented method of claim 3, further comprising: evaluating a set of data values in the set of output data during the compressing; This is taught by Yan. See Yan, Fig. 1A, element 110 along with ¶ 0020, e.g. “At step 110, the multi-bit data is determined to equal zero.”
Temam and Yan does not expressly teach:
generating second metadata based on the evaluating of the set of data values in the set of output data; and This is taught by Mahan. See Mahan, col. 5, lines 26-30, “The conditional execution controller, in the depicted embodiment, is used to test pixel data packets as they are passed through the graphics pipeline, and determine which "branch" of instructions should be applied to that particular packet.” It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use Mahan’s data evaluation with Temam’s conditional instructions in order to avoid redundant or unnecessary operations as suggested by Mahan (see col. 5 lines 37-39).
Temam and Mahan teach:
storing the second metadata in association with the second data tile; See Temam, ¶ 0031 and 0038-0039 as cited above, in view of Mahan above.
Temam does not expressly disclose:
wherein the second data tile holds a set of sparse values of the output data. However, this is taught by Yan. See Yan, Fig. 1A, depicting data compression. Also see ¶ 0021, e.g. “In one embodiment, a compacted data sequence for input to a PE is received by the DLA, where the compacted data sequence comprises at least one single bit signal indicating that at least one multi-bit value equals zero, and the single bit signal is transmitted to the PE in lieu of the multi -bit value.” It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use Temam’s data tiles and storage with Yan’s sparse data in order to assist in CNN evaluation and network pruning as suggested by Yan (see ¶ 0034).
In regard to claim 55, Temam does not expressly disclose:
55. The computer-implemented method of claim 2, wherein: evaluating the set of ANN data elements includes forming a sequence of sparse data values; and a sequence of indexes into the sequence of sparse data values; and conditionally executing the set of arithmetic logic operations from the set of arithmetic logic operations requires the sequence of sparse data values and the sequence of indexes into the sequence of sparse data values. However, Yan teaches execution using the compacted data format. See Yan, Fig. 3B, depicting formation of a sequence of values and intrinsic indexes. Also see Yan, ¶ 0043, "A dense encoding of sparse weights and activations is used to reduce the bandwidth needed to transmit the weight and activation values from the memory to the DLA 200, between different levels of the memory hierarchy, and between the different logic circuits within the DLA 200. Input data, such as weights and activations with zeros can be represented in a compact form." It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use Temam’s data tiles and storage with Yan's compacted representation in order to increase data capacity and reduce costly memory accesses as suggested by Yan.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Temam as applied above, and further in view of U.S. Patent Application Publication 20150301826 by Sideris et al. ("Sideris").
In regard to claim 11, Temam also discloses:
11. The computer-implemented method of claim 1, wherein: the instruction is part of an instruction sequence for a standard execution of the ANN; and Temam, ¶ 0023, “Controller 102 is configured to execute one or more instructions relating to tensor computations within system 100.”
Temam discloses conditional execution using a bitmap header as cited above. Temam does not expressly disclose: the conditional execution is less computationally intensive than the standard execution. This is taught by Sideris. See ¶ 0040, “In some embodiments, power consumption can be reduced by reusing the result of the second micro-operation for the first micro-operation. The control circuitry may prevent the first processing lane processing the first micro-operation by placing at least part of the first processing lane in a power saving state during a processing cycle when it would otherwise be processing the first micro-operation. Hence, even if the first micro-operation gives the same result as the second micro-operation, it can still be passed to the processing circuitry of the first processing lane, but parts of the first processing lane may be placed in a power saving state so that they do not actually process the micro-operation in order to save power. This avoids duplication of processing which might occur if both the first and second processing lanes process the same micro-operation.” It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the reduction of computation taught by Sideris along with Temam’s conditional execution in order to save power as suggested by Sideris.
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Temam as applied above, and further in view of U.S. Patent 6215507 to Nally et al. ("Nally").
In regard to claim 12, Temam discloses:
12. The computer-implemented method of claim 1, wherein: the first data tile includes the set of ANN data elements in a block of the memory. Temam, ¶ 0005, “A tensor accessible from narrow memory and wide memory units, in a single compute tile, is traversed based on memory address values retrieved from registers.” Also ¶ 0073, “Activation values are stored in a narrow memory 210.”
Temam does not expressly disclose: contiguous. This is taught by Nally. See col. 6, lines 35-38, “By following the arrows of FIG. 7 it will be seen that the first 144 tiles of data for screen 2 are stored in contiguous locations of memory 26.” It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use Nally’s contiguous memory with Temam’s tensors in order to optimize storage as suggested by Nally (col. 2 lines 30-32).
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Temam as applied above, and further in view of Mahan and U.S. Patent 5164938 to Jurkevich et al. (“Jurkevich”).
In regard to claim 14, Temam does not expressly disclose:
14. The computer-implemented method of claim 1, wherein: a data structure that holds the metadata is smaller than the first data tile by a factor of four. However, Mahan teaches the use of a data structure to hold metadata. See Mahan, Fig. 3 and col. 5, lines 54-59, e.g. "For data packets being passed through a graphics pipeline, the header contains information relevant to the processing of the data contained in the payload. For example, the header is shown in FIG. 3 as including sequence number 311 and conditional execution (CX) flag 313." It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use Temam’s data and metadata with Mahan’s relational metadata header storage in order to provide faster and more efficient calculation in a processing pipeline as suggested by Mahan.
Also, Jurkevich teaches data/metadata size ratios. See Jurkevich, col. 13, lines 8-45, e.g. “For example, if the frame is small, with a ratio of payload size to header size approaching unity, it has a good packetization delay … If the frame is too large, in the sense that the ratio of payload size to header size is substantial, packetization delay is poor in low speed applications. … The current ATM standard is a cell having a payload size of 48 bytes and a header size of five bytes.” Jurkevich teaches that header/metadata to payload ratio in the range of 1:1 (i.e. “unity”) through 5:48 (i.e. 1:9.6) and beyond, which is inclusive of a ratio of 1:4. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the metadata data structure of Temam and Mahan with Jurkevich’s ratio teaching in order to select a ratio that meets the needs of the system/user as essentially suggested by Jurkevich.
Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Temam as applied above, and further in view of Yan and Nally.
In regard to claim 15 Temam does not expressly disclose:
15. The computer-implemented method of claim 1, wherein: the first data tile holds … a compressed format, consisting of sparse data values and non-sparse data values, and … the non-sparse data values of the compressed format are zeroes; However, this is taught by Yan. See Yan, Fig. 1A, depicting data compression. Also see ¶ 0021, e.g. “In one embodiment, a compacted data sequence for input to a PE is received by the DLA, where the compacted data sequence comprises at least one single bit signal indicating that at least one multi-bit value equals zero, and the single bit signal is transmitted to the PE in lieu of the multi -bit value.” Also see ¶ 0034, “Sparsity in a layer of a CNN is defined as the fraction of zeros in the layer's weight and input activation matrices.” Note that non-sparse is broadly interpreted as an inverse of sparsity. Thus, non-sparsity would also be a fraction of zeros. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use Temam’s data with Yan’s sparsity/non-sparsity to assist in CNN evaluation and network pruning in order to utilize a smaller network while maintaining accuracy as suggested by Yan (see ¶ 0034).
All further limitations of claim 15 have been addressed in the above rejections of claims 6, 9-10, and 12-13.
Claim 54 is rejected under 35 U.S.C. 103 as being unpatentable over Temam as applied above, and further in view of U.S. Patent Application Publication 2017/0200094 by Bruestle et al. (“Bruestle”).
In regard to claim 54 Temam does not expressly disclose:
54. The computer-implemented method of claim 1, further comprising: conducting a simplified execution of the ANN using the set of ANN data elements; wherein the simplified execution of the ANN uses a down-sampled version of the ANN; and This is taught by Bruestle. See ¶ 0154, “reduce precision.”
wherein the generating of the metadata is conducted during the simplified execution of the ANN. This is taught by Bruestle. See ¶ 0113, “During each clock cycle … Each operation unit 116 then updates its registers.” Also see ¶ 0119, “CMP:S[D]:=(B<=A<=C)//Compare A to range, update S[D].”
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use Bruestle’s simplified execution with Temam’s data in order to provide efficient implementation as suggested by Bruestle (see ¶ 0032).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to James D Rutten whose telephone number is (571)272-3703. The examiner can normally be reached M-F 9:00-5:30 ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Li B Zhen can be reached on (571)272-3768. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/James D. Rutten/Primary Examiner, Art Unit 2121
Prosecution Insights