DETAILED ACTION
1. The present application is being examined under the pre-AIA first to invent provisions.
Information Disclosure Statement
2. The information disclosure statements (IDS) submitted on 06/09/2023, 12/11/2024, and 02/02/2026 have been received, entered into the record, and considered. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.
Claim Rejections - 35 USC § 102
3. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
4. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
5. Claims 1, 3-6, 8-13, and 15-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Chinya et al. (U.S. PGPUB 2020/0228137).
6. Regarding claim 1, Chinya teaches a system comprising:
A) a hardware accelerator that includes one or more circuitry components configured for: receiving a first tensor (Paragraphs 23, 25, and 53);
B) the first tensor being a condensed representation of a second tensor (Paragraph 23);
C) the second tensor including at least one zero value and the first tensor not including any zero values (Paragraph 23);
D) deriving shape data of the second tensor (Paragraphs 30 and 41);
E) deriving a binary map (Paragraph 23);
F) each zero bit in the binary map indicating a corresponding zero value in the second tensor (Paragraph 23);
G) each one bit in the binary map indicating a corresponding non-zero value in the second tensor (Paragraph 23); and
H) based at least in part on the shape data and the binary map, performing a neural network operation on the first tensor (Paragraphs 25, 30, 41, and 50).
The examiner notes that Chinya teaches “a hardware accelerator that includes one or more circuitry components configured for: receiving a first tensor” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]” (Paragraph 23), “Machine learning accelerators (e.g., those utilizing DNN engines, CNN engines, etc.) handle a large amount of tensor data (e.g., data stored in multi-dimensional data structures) for performing inference tasks” (Paragraph 25), and “the first input buffer 112 includes the header 202. In the example of FIG. 2, the header 202 indicates whether the data following the header is uncompressed or whether the data following the header includes a sparsity bitmap and a ZVC data vector” (Paragraph 53). The examiner further notes that a received ZVC vector (i.e. the claimed first tensor) teaches the claimed receiving of a first tensor. The examiner further notes that Chinya teaches “the first tensor being a condensed representation of a second tensor” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]” (Paragraph 23). The examiner further notes that a ZVC vector (i.e. the claimed first tensor) is a condensed version of a sparse vector (i.e. the claimed second tensor). The examiner further notes that Chinya teaches “the second tensor including at least one zero value and the first tensor not including any zero values” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]” (Paragraph 23). The examiner further notes that a ZVC vector (i.e. the claimed first tensor) does not contain any zero-elements whereas a sparse vector (i.e. the claimed second tensor) does include zero-elements. The examiner further notes that Chinya teaches “deriving shape data of the second tensor” as “Examples disclosed herein advantageously reconstruct the sparsity bitmap at PEs on the fly according to the flexible tensor shapes” (Paragraph 30) and “The byte select signals (e.g., Byte_Sel[0] through Byte_Sel[N]) determine the shape of the tensor (e.g., two by two by three, etc.) to be processed and the volume processed by each PE according to a schedule” (Paragraph 41). The examiner further notes that determining (i.e. deriving) the shape of a tensor includes determining the shape of a sparse vector (i.e. the claimed second tensor). The examiner further notes that Chinya teaches “deriving a binary map” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero)” (Paragraph 23). The examiner further notes that the example determined sparsity bitmap teaches the claimed binary map. The examiner further notes that Chinya teaches “each zero bit in the binary map indicating a corresponding zero value in the second tensor” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero)” (Paragraph 23). The examiner further notes that the zero-elements in the sparsity bitmap (i.e. the claimed binary map) correspond to the zero-elements of the sparse vector (i.e. the claimed second tensor). The examiner further notes that Chinya teaches “each one bit in the binary map indicating a corresponding non-zero value in the second tensor” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero)” (Paragraph 23). The examiner further notes that the one-elements in the sparsity bitmap (i.e. the claimed binary map) correspond to the non-zero elements of the sparse vector (i.e. the claimed second tensor). The examiner further notes that Chinya teaches “based at least in part on the shape data and the binary map, performing a neural network operation on the first tensor” as “Machine learning accelerators (e.g., those utilizing DNN engines, CNN engines, etc.) handle a large amount of tensor data (e.g., data stored in multi-dimensional data structures) for performing inference tasks. Processing large amounts of tensor data requires data movement across multiple levels of a memory hierarchy (e.g., hard drives, flash storage, RAM, cache, registers, etc.) to a processing element (PE) array. Reducing data transfer and increasing (e.g., maximizing) data reuse and resource utilization can improve energy efficiency. Due to the nature of DNN and/or other AI engines, both inputs to the DNN (sometimes referred to as input activations and/or input feature maps) and weights (sometimes referred to as trained DNN model parameters) include sparse vectors. For example, input activation vectors and/or weight vectors can include a significant amount of zero elements due to rectifying operations in DNN layers. As illustrated above, utilizing ZVC data vectors and sparsity bitmaps can be an effective technique to accelerate the inference and training of a DNN as well as to reduce the storage requirement for parameters (e.g. compression) for energy efficiency” (Paragraph 25), “Examples disclosed herein advantageously reconstruct the sparsity bitmap at PEs on the fly according to the flexible tensor shapes” (Paragraph 30), “The byte select signals (e.g., Byte_Sel[0] through Byte_Sel[N]) determine the shape of the tensor (e.g., two by two by three, etc.) to be processed and the volume processed by each PE according to a schedule” (Paragraph 41), and “The example InSAD system 100 illustrated in FIG. 1 can be implemented with machine learning accelerators to reduce data movement. The example InSAD system 100 combines both flexible tensor distribution and sparse data compression by (1) decoding ZVC data vectors with software programed byte select signals (e.g., Byte_Sel[0]-Byte_Sel[N]) to distribute non-zero data to respective PE arrays, (2) reconstructing the sparsity bitmap at each PE on the fly for different tensor shapes, (3) eliminating one or more storage requirements for uncompressed data across on-chip memory hierarchy, and (4) serving different tensor shapes (e.g., one or more multi-dimension array dimensions) for each PE. The examples disclosed herein are applicable to various dataflow-based accelerators” (Paragraph 50). The examiner further notes that the execution of machine learning (which uses neural networks) on the ZVC (i.e. first tensor) is based off of shape data and the sparsity bitmap (i.e. the claimed binary map).
Regarding claim 3, Chinya further teaches a system comprising:
A) wherein the first tensor is a vector of contiguous non-zero values (Paragraph 23);
B) the vector being a 1×N array (Paragraph 23); and
C) wherein the first tensor is generated by transferring all non-zero values in the second tensor to the first tensor (Paragraph 23).
The examiner notes that Chinya teaches “wherein the first tensor is a vector of contiguous non-zero values” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero)” (Paragraph 23). The examiner further notes that the ZVC vector (i.e. the claimed first tensor) is an array vector of contiguous non-zero values. The examiner further notes that Chinya teaches “the vector being a 1×N array” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero)” (Paragraph 23). The examiner further notes that the example ZVC vector (i.e. the claimed first tensor) is an array vector of dimensions 1 x X, where X is 3. The examiner further notes that Chinya teaches “wherein the first tensor is generated by transferring all non-zero values in the second tensor to the first tensor” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero)” (Paragraph 23). The examiner further notes that the ZVC vector (i.e. the claimed first tensor) is resultant from “transferring” all non-zero elements of the sparse vector (i.e. the claimed second tensor).
Regarding claim 4, Chinya further teaches a system comprising:
A) wherein the shape data is calculated based on a quantity of values in each axis of the second tensor (Paragraph 41).
The examiner notes that Chinya teaches “wherein the shape data is calculated based on a quantity of values in each axis of the second tensor” as “The byte select signals (e.g., Byte_Sel[0] through Byte_Sel[N]) determine the shape of the tensor (e.g., two by two by three, etc.) to be processed and the volume processed by each PE according to a schedule” (Paragraph 41). The examiner further notes that the determined shape of a tensor is based on a quantity of values in each axis of that tensor.
Regarding claim 5, Chinya further teaches a system comprising:
A) wherein the one or more circuitry components are further configured for: generating a second binary map of one bits and zero bits (Paragraphs 23 and 26);
B) the zero bits indicate the zero values in a second output tensor and the one bits indicate non-zero values in the second output tensor (Paragraphs 23 and 26);
C) based on the generating of the second binary map, generating a first output tensor that is a condensed representation of the second output tensor associated with the second tensor (Paragraphs 23 and 26); and
D) wherein the second output tensor includes zero values and the first output tensor does not include any zero values (Paragraphs 23 and 26); and
E) wherein the generating of the first output tensor is an output for the performing of the neural network operation on the first tensor (Paragraphs 23 and 26).
The examiner notes that Chinya teaches “wherein the one or more circuitry components are further configured for: generating a second binary map of one bits and zero bits” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero)” (Paragraph 23) and “A non-linear function (e.g. rectified linear unit (ReLu)), is applied to the output activations which become the input activations for the next layer. In some DNNs, a significant fraction of each DNN layer's activations and weights are zero-valued due to ReLu operations, hence this data can be compressed via various techniques to save the on-chip storage requirements and bandwidth demands” (Paragraph 26). The examiner further notes that a second sparsity bitmap (i.e. the claimed binary map) includes only one and zero bits and can be generated. The examiner further notes that Chinya teaches “the zero bits indicate the zero values in a second output tensor and the one bits indicate non-zero values in the second output tensor” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero)” (Paragraph 23) and “A non-linear function (e.g. rectified linear unit (ReLu)), is applied to the output activations which become the input activations for the next layer. In some DNNs, a significant fraction of each DNN layer's activations and weights are zero-valued due to ReLu operations, hence this data can be compressed via various techniques to save the on-chip storage requirements and bandwidth demands” (Paragraph 26). The examiner further notes that a second sparsity bitmap (i.e. the claimed binary map) includes only one and zero bits that correspond to zero values and non-zero values of a corresponding sparse vector (i.e. second output tensor). The examiner further notes that Chinya teaches “based on the generating of the second binary map, generating a first output tensor that is a condensed representation of the second output tensor associated with the second tensor” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero)” (Paragraph 23) and “A non-linear function (e.g. rectified linear unit (ReLu)), is applied to the output activations which become the input activations for the next layer. In some DNNs, a significant fraction of each DNN layer's activations and weights are zero-valued due to ReLu operations, hence this data can be compressed via various techniques to save the on-chip storage requirements and bandwidth demands” (Paragraph 26). The examiner further notes that a generated ZVC vector (i.e. first output tensor) is a condensed representation of a sparse vector (i.e. second output tensor). The examiner further notes that Chinya teaches “wherein the second output tensor includes zero values and the first output tensor does not include any zero values” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero)” (Paragraph 23) and “A non-linear function (e.g. rectified linear unit (ReLu)), is applied to the output activations which become the input activations for the next layer. In some DNNs, a significant fraction of each DNN layer's activations and weights are zero-valued due to ReLu operations, hence this data can be compressed via various techniques to save the on-chip storage requirements and bandwidth demands” (Paragraph 26). The examiner further notes that a generated ZVC vector (i.e. first output tensor) does not include zero values whereas a sparse vector (i.e. second output tensor) does include zero values. The examiner further notes that Chinya teaches “wherein the generating of the first output tensor is an output for the performing of the neural network operation on the first tensor” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero)” (Paragraph 23) and “A non-linear function (e.g. rectified linear unit (ReLu)), is applied to the output activations which become the input activations for the next layer. In some DNNs, a significant fraction of each DNN layer's activations and weights are zero-valued due to ReLu operations, hence this data can be compressed via various techniques to save the on-chip storage requirements and bandwidth demands” (Paragraph 26). The examiner further notes that the application of ReLu on output (which then becomes input) teaches the aforementioned.
Regarding claim 6, Chinya teaches a system comprising:
A) wherein the second tensor is a sparse tensor among a plurality of non-sparse tensors (Paragraph 23); and
B) wherein the one or more circuitry components are further configured for: selecting the second tensor for the condensing to the first tensor (Paragraph 23); and
C) refraining from selecting any of the non-sparse tensors to condense into another tensor (Paragraph 23).
The examiner notes that Chinya teaches “wherein the second tensor is a sparse tensor among a plurality of non-sparse tensors” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero). The combination of the ZVC data vector and the sparsity bitmap represents the sparse vector (e.g., the sparse vector could be generated/reconstructed based on the corresponding ZVC data vector and sparsity bitmap)” (Paragraph 23). The examiner further notes that the sparse vector (i.e. the claimed second tensor) is amongst multiple non-sparse tensors (See ZVC vector, sparsity bitmap, etc). The examiner further notes that Chinya teaches “wherein the one or more circuitry components are further configured for: selecting the second tensor for the condensing to the first tensor” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero). The combination of the ZVC data vector and the sparsity bitmap represents the sparse vector (e.g., the sparse vector could be generated/reconstructed based on the corresponding ZVC data vector and sparsity bitmap)” (Paragraph 23). The examiner further notes that the sparse vector (i.e. the claimed second tensor) is “condensed” into the ZVC vector (i.e. the claimed first tensor). The examiner further notes that Chinya teaches “refraining from selecting any of the non-sparse tensors to condense into another tensor” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero). The combination of the ZVC data vector and the sparsity bitmap represents the sparse vector (e.g., the sparse vector could be generated/reconstructed based on the corresponding ZVC data vector and sparsity bitmap)” (Paragraph 23). The examiner further notes that the sparse vector (i.e. the claimed second tensor) is “condensed” into the ZVC vector (i.e. the claimed first tensor) whereas the sparsity bitmap, ZVC vector (i.e. non-sparse tensors) are not condensed.
Regarding claim 8, Chinya teaches a computer-implemented method comprising:
A) receiving a first tensor (Paragraphs 23, 25, and 53);
B) the first tensor including a plurality sparse values (Paragraph 23);
C) generating a second tensor that represents a condensed version of the first tensor without the plurality of sparse values (Paragraph 23);
D) computing a shape of the first tensor (Paragraphs 30 and 41);
E) based at least in part on the first tensor, generating a binary map (Paragraph 23);
F) each zero bit in the binary map indicating a corresponding position of a sparse value in the first tensor (Paragraph 23);
G) each one bit in the binary map indicating a corresponding position of a non-sparse value in the first tensor (Paragraph 23); and
H) based at least in part on the second tensor, the shape of the first tensor, and the binary map, performing a machine learning model operation (Paragraphs 25, 30, 41, and 50).
The examiner notes that Chinya teaches “receiving a first tensor” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]” (Paragraph 23), “Machine learning accelerators (e.g., those utilizing DNN engines, CNN engines, etc.) handle a large amount of tensor data (e.g., data stored in multi-dimensional data structures) for performing inference tasks” (Paragraph 25), and “the first input buffer 112 includes the header 202. In the example of FIG. 2, the header 202 indicates whether the data following the header is uncompressed or whether the data following the header includes a sparsity bitmap and a ZVC data vector” (Paragraph 53). The examiner further notes that a received sparse vector (i.e. the claimed first tensor) teaches the claimed receiving of a first tensor. The examiner further notes that Chinya teaches “the first tensor including a plurality sparse values” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]” (Paragraph 23). The examiner further notes that a sparse vector (i.e. the claimed first tensor) includes a plurality of sparse values. The examiner further notes that Chinya teaches “generating a second tensor that represents a condensed version of the first tensor without the plurality of sparse values” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]” (Paragraph 23). The examiner further notes that a ZVC vector (i.e. the claimed second tensor) does not contain any zero-elements (i.e. sparse values) and is a condensed version of a sparse vector (i.e. the claimed first tensor). The examiner further notes that Chinya teaches “computing a shape of the first tensor” as “Examples disclosed herein advantageously reconstruct the sparsity bitmap at PEs on the fly according to the flexible tensor shapes” (Paragraph 30) and “The byte select signals (e.g., Byte_Sel[0] through Byte_Sel[N]) determine the shape of the tensor (e.g., two by two by three, etc.) to be processed and the volume processed by each PE according to a schedule” (Paragraph 41). The examiner further notes that determining (i.e. computing) the shape of a tensor includes determining the shape of a sparse vector (i.e. the claimed first tensor). The examiner further notes that Chinya teaches “based at least in part on the first tensor, generating a binary map” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero)” (Paragraph 23). The examiner further notes that the example determined sparsity bitmap teaches the claimed binary map. The examiner further notes that Chinya teaches “each zero bit in the binary map indicating a corresponding position of a sparse value in the first tensor” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero)” (Paragraph 23). The examiner further notes that the zero-elements in the sparsity bitmap (i.e. the claimed binary map) correspond to the zero-elements of the sparse vector (i.e. the claimed first tensor). The examiner further notes that Chinya teaches “each one bit in the binary map indicating a corresponding position of a non-sparse value in the first tensor” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero)” (Paragraph 23). The examiner further notes that the one-elements in the sparsity bitmap (i.e. the claimed binary map) correspond to the non-zero elements of the sparse vector (i.e. the claimed first tensor). The examiner further notes that Chinya teaches “based at least in part on the second tensor, the shape of the first tensor, and the binary map, performing a machine learning model operation” as “Machine learning accelerators (e.g., those utilizing DNN engines, CNN engines, etc.) handle a large amount of tensor data (e.g., data stored in multi-dimensional data structures) for performing inference tasks. Processing large amounts of tensor data requires data movement across multiple levels of a memory hierarchy (e.g., hard drives, flash storage, RAM, cache, registers, etc.) to a processing element (PE) array. Reducing data transfer and increasing (e.g., maximizing) data reuse and resource utilization can improve energy efficiency. Due to the nature of DNN and/or other AI engines, both inputs to the DNN (sometimes referred to as input activations and/or input feature maps) and weights (sometimes referred to as trained DNN model parameters) include sparse vectors. For example, input activation vectors and/or weight vectors can include a significant amount of zero elements due to rectifying operations in DNN layers. As illustrated above, utilizing ZVC data vectors and sparsity bitmaps can be an effective technique to accelerate the inference and training of a DNN as well as to reduce the storage requirement for parameters (e.g. compression) for energy efficiency” (Paragraph 25), “Examples disclosed herein advantageously reconstruct the sparsity bitmap at PEs on the fly according to the flexible tensor shapes” (Paragraph 30), “The byte select signals (e.g., Byte_Sel[0] through Byte_Sel[N]) determine the shape of the tensor (e.g., two by two by three, etc.) to be processed and the volume processed by each PE according to a schedule” (Paragraph 41), and “The example InSAD system 100 illustrated in FIG. 1 can be implemented with machine learning accelerators to reduce data movement. The example InSAD system 100 combines both flexible tensor distribution and sparse data compression by (1) decoding ZVC data vectors with software programed byte select signals (e.g., Byte_Sel[0]-Byte_Sel[N]) to distribute non-zero data to respective PE arrays, (2) reconstructing the sparsity bitmap at each PE on the fly for different tensor shapes, (3) eliminating one or more storage requirements for uncompressed data across on-chip memory hierarchy, and (4) serving different tensor shapes (e.g., one or more multi-dimension array dimensions) for each PE. The examples disclosed herein are applicable to various dataflow-based accelerators” (Paragraph 50). The examiner further notes that the execution of machine learning (which uses neural networks) on the ZVC (i.e. second tensor) is based off of shape data and the sparsity bitmap (i.e. the claimed binary map).
Regarding claim 9, Chinya further teaches a computer-method comprising:
A) wherein the method is performed by one of: a Central Processing Unit (CPU) or a hardware accelerator (Paragraph 123).
The examiner notes that Chinya teaches “wherein the method is performed by one of: a Central Processing Unit (CPU) or a hardware accelerator” as “although individually listed, a plurality of means, elements or method actions may be implemented by, e.g., a single unit or processor” (Paragraph 123). The examiner further notes that the processor teaches the claimed CPU.
Regarding claim 10, Chinya further teaches a computer-implemented method comprising:
A) wherein the second tensor is a vector of contiguous non-zero values (Paragraph 23);
B) the vector being a 1×N array (Paragraph 23); and
C) wherein the second tensor is generated by transferring all non-zero values in the first tensor to the second tensor (Paragraph 23).
The examiner notes that Chinya teaches “wherein the second tensor is a vector of contiguous non-zero values” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero)” (Paragraph 23). The examiner further notes that the ZVC vector (i.e. the claimed second tensor) is an array vector of contiguous non-zero values. The examiner further notes that Chinya teaches “the vector being a 1×N array” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero)” (Paragraph 23). The examiner further notes that the example ZVC vector (i.e. the claimed first tensor) is an array vector of dimensions 1 x X, where X is 3. The examiner further notes that Chinya teaches “wherein the second tensor is generated by transferring all non-zero values in the first tensor to the second tensor” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero)” (Paragraph 23). The examiner further notes that the ZVC vector (i.e. the claimed second tensor) is resultant from “transferring” all non-zero elements of the sparse vector (i.e. the claimed first tensor).
Regarding claim 11, Chinya further teaches a computer-implemented method comprising:
A) wherein the shape data is computed based on a quantity of values in each axis of the first tensor (Paragraph 41).
The examiner notes that Chinya teaches “wherein the shape data is computed based on a quantity of values in each axis of the first tensor” as “The byte select signals (e.g., Byte_Sel[0] through Byte_Sel[N]) determine the shape of the tensor (e.g., two by two by three, etc.) to be processed and the volume processed by each PE according to a schedule” (Paragraph 41). The examiner further notes that the determined shape of a tensor is based on a quantity of values in each axis of that tensor.
Regarding claim 12, Chinya further teaches a computer-implemented method comprising:
A) generating a second binary map of one bits and zero bits (Paragraphs 23 and 26);
B) the zero bits indicate the sparse values in a second output tensor and the one bits indicate non-sparse values in the second output tensor (Paragraphs 23 and 26);
C) generating final shape data (Paragraphs 26, 30, and 41); and
D) based on the generating of the second binary map and the final shape data, generating a first output tensor that is a condensed representation of the second output tensor associated with the first tensor (Paragraphs 23, 26, 30, and 41); and
E) wherein the second output tensor includes sparse values and the first output tensor does not include any sparse values (Paragraphs 23 and 26); and
F) wherein the generating of the first output tensor is an output for the performing of the machine learning operation (Paragraphs 23 and 26).
The examiner notes that Chinya teaches “generating a second binary map of one bits and zero bits” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero)” (Paragraph 23) and “A non-linear function (e.g. rectified linear unit (ReLu)), is applied to the output activations which become the input activations for the next layer. In some DNNs, a significant fraction of each DNN layer's activations and weights are zero-valued due to ReLu operations, hence this data can be compressed via various techniques to save the on-chip storage requirements and bandwidth demands” (Paragraph 26). The examiner further notes that a second sparsity bitmap (i.e. the claimed binary map) includes only one and zero bits and can be generated. The examiner further notes that Chinya teaches “the zero bits indicate the sparse values in a second output tensor and the one bits indicate non-sparse values in the second output tensor” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero)” (Paragraph 23) and “A non-linear function (e.g. rectified linear unit (ReLu)), is applied to the output activations which become the input activations for the next layer. In some DNNs, a significant fraction of each DNN layer's activations and weights are zero-valued due to ReLu operations, hence this data can be compressed via various techniques to save the on-chip storage requirements and bandwidth demands” (Paragraph 26). The examiner further notes that a second sparsity bitmap (i.e. the claimed binary map) includes only one and zero bits that correspond to zero values and non-zero values of a corresponding sparse vector (i.e. second output tensor). The examiner further notes that Chinya teaches “generating final shape data” as “A non-linear function (e.g. rectified linear unit (ReLu)), is applied to the output activations which become the input activations for the next layer. In some DNNs, a significant fraction of each DNN layer's activations and weights are zero-valued due to ReLu operations, hence this data can be compressed via various techniques to save the on-chip storage requirements and bandwidth demands” (Paragraph 26), “Examples disclosed herein advantageously reconstruct the sparsity bitmap at PEs on the fly according to the flexible tensor shapes” (Paragraph 30), and “The byte select signals (e.g., Byte_Sel[0] through Byte_Sel[N]) determine the shape of the tensor (e.g., two by two by three, etc.) to be processed and the volume processed by each PE according to a schedule” (Paragraph 41). The examiner further notes that determining (i.e. computing) the shape in the iterative computation includes a final shape determination. The examiner further notes that Chinya teaches “based on the generating of the second binary map and the final shape data, generating a first output tensor that is a condensed representation of the second output tensor associated with the first tensor” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero)” (Paragraph 23), “A non-linear function (e.g. rectified linear unit (ReLu)), is applied to the output activations which become the input activations for the next layer. In some DNNs, a significant fraction of each DNN layer's activations and weights are zero-valued due to ReLu operations, hence this data can be compressed via various techniques to save the on-chip storage requirements and bandwidth demands” (Paragraph 26), “Examples disclosed herein advantageously reconstruct the sparsity bitmap at PEs on the fly according to the flexible tensor shapes” (Paragraph 30), and “The byte select signals (e.g., Byte_Sel[0] through Byte_Sel[N]) determine the shape of the tensor (e.g., two by two by three, etc.) to be processed and the volume processed by each PE according to a schedule” (Paragraph 41). The examiner further notes that a generated ZVC vector (i.e. first output tensor) is a condensed representation of a sparse vector (i.e. second output tensor). The examiner further notes that Chinya teaches “wherein the second output tensor includes sparse values and the first output tensor does not include any sparse values” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero)” (Paragraph 23) and “A non-linear function (e.g. rectified linear unit (ReLu)), is applied to the output activations which become the input activations for the next layer. In some DNNs, a significant fraction of each DNN layer's activations and weights are zero-valued due to ReLu operations, hence this data can be compressed via various techniques to save the on-chip storage requirements and bandwidth demands” (Paragraph 26). The examiner further notes that a generated ZVC vector (i.e. first output tensor) does not include sparse values whereas a sparse vector (i.e. second output tensor) does include sparse values. The examiner further notes that Chinya teaches “wherein the generating of the first output tensor is an output for the performing of the machine learning operation” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero)” (Paragraph 23) and “A non-linear function (e.g. rectified linear unit (ReLu)), is applied to the output activations which become the input activations for the next layer. In some DNNs, a significant fraction of each DNN layer's activations and weights are zero-valued due to ReLu operations, hence this data can be compressed via various techniques to save the on-chip storage requirements and bandwidth demands” (Paragraph 26). The examiner further notes that the application of ReLu on output (which then becomes input) teaches the aforementioned.
Regarding claim 13, Chinya teaches a computer-implemented method comprising:
A) wherein the first tensor is a sparse tensor among a plurality of non-sparse tensors (Paragraph 23); and
B) wherein method further comprises selecting the first tensor for the condensing to the second tensor (Paragraph 23); and
C) refraining from selecting any of the non-sparse tensors to condense into another tensor (Paragraph 23).
The examiner notes that Chinya teaches “wherein the first tensor is a sparse tensor among a plurality of non-sparse tensors” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero). The combination of the ZVC data vector and the sparsity bitmap represents the sparse vector (e.g., the sparse vector could be generated/reconstructed based on the corresponding ZVC data vector and sparsity bitmap)” (Paragraph 23). The examiner further notes that the sparse vector (i.e. the claimed first tensor) is amongst multiple non-sparse tensors (See ZVC vector, sparsity bitmap, etc). The examiner further notes that Chinya teaches “wherein method further comprises selecting the first tensor for the condensing to the second tensor” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero). The combination of the ZVC data vector and the sparsity bitmap represents the sparse vector (e.g., the sparse vector could be generated/reconstructed based on the corresponding ZVC data vector and sparsity bitmap)” (Paragraph 23). The examiner further notes that the sparse vector (i.e. the claimed first tensor) is “condensed” into the ZVC vector (i.e. the claimed second tensor). The examiner further notes that Chinya teaches “refraining from selecting any of the non-sparse tensors to condense into another tensor” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero). The combination of the ZVC data vector and the sparsity bitmap represents the sparse vector (e.g., the sparse vector could be generated/reconstructed based on the corresponding ZVC data vector and sparsity bitmap)” (Paragraph 23). The examiner further notes that the sparse vector (i.e. the claimed first tensor) is “condensed” into the ZVC vector (i.e. the claimed second tensor) whereas the sparsity bitmap, ZVC vector (i.e. non-sparse tensors) are not condensed.
Regarding claim 15, Chinya teaches a method comprising:
A) receiving a first data structure (Paragraphs 23, 25, and 53);
B) the first data structure being a condensed representation of a second data structure (Paragraph 23);
C) the second data structure including at least one sparse value and the first data structure not including any sparse values (Paragraph 23);
D) deriving shape data of the second data structure (Paragraphs 30 and 41);
E) based at least in part on using the first data structure and the shape data as input, performing a machine learning model operation on the first data structure but not the second data structure (Paragraphs 25, 30, 41, and 50).
The examiner notes that Chinya teaches “receiving a first data structure” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]” (Paragraph 23), “Machine learning accelerators (e.g., those utilizing DNN engines, CNN engines, etc.) handle a large amount of tensor data (e.g., data stored in multi-dimensional data structures) for performing inference tasks” (Paragraph 25), and “the first input buffer 112 includes the header 202. In the example of FIG. 2, the header 202 indicates whether the data following the header is uncompressed or whether the data following the header includes a sparsity bitmap and a ZVC data vector” (Paragraph 53). The examiner further notes that a received ZVC vector (i.e. the claimed first data structure) teaches the claimed receiving of a first data structure. The examiner further notes that Chinya teaches “the first data structure being a condensed representation of a second data structure” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]” (Paragraph 23). The examiner further notes that a ZVC vector (i.e. the claimed first data structure) is a condensed version of a sparse vector (i.e. the claimed second data structure). The examiner further notes that Chinya teaches “the second data structure including at least one sparse value and the first data structure not including any sparse values” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]” (Paragraph 23). The examiner further notes that a ZVC vector (i.e. the claimed first data structure) does not contain any sparse values whereas a sparse vector (i.e. the claimed second data structure) does include sparse values. The examiner further notes that Chinya teaches “deriving shape data of the second data structure” as “Examples disclosed herein advantageously reconstruct the sparsity bitmap at PEs on the fly according to the flexible tensor shapes” (Paragraph 30) and “The byte select signals (e.g., Byte_Sel[0] through Byte_Sel[N]) determine the shape of the tensor (e.g., two by two by three, etc.) to be processed and the volume processed by each PE according to a schedule” (Paragraph 41). The examiner further notes that determining (i.e. deriving) the shape of a tensor includes determining the shape of a sparse vector (i.e. the claimed second data structure). The examiner further notes that Chinya teaches “deriving a binary map” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero)” (Paragraph 23). The examiner further notes that the example determined sparsity bitmap teaches the claimed binary map. The examiner further notes that Chinya teaches “each zero bit in the binary map indicating a corresponding zero value in the second tensor” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero)” (Paragraph 23). The examiner further notes that the zero-elements in the sparsity bitmap (i.e. the claimed binary map) correspond to the zero-elements of the sparse vector (i.e. the claimed second tensor). The examiner further notes that Chinya teaches “each one bit in the binary map indicating a corresponding non-zero value in the second tensor” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero)” (Paragraph 23). The examiner further notes that the one-elements in the sparsity bitmap (i.e. the claimed binary map) correspond to the non-zero elements of the sparse vector (i.e. the claimed second tensor). The examiner further notes that Chinya teaches “based at least in part on using the first data structure and the shape data as input, performing a machine learning model operation on the first data structure but not the second data structure” as “Machine learning accelerators (e.g., those utilizing DNN engines, CNN engines, etc.) handle a large amount of tensor data (e.g., data stored in multi-dimensional data structures) for performing inference tasks. Processing large amounts of tensor data requires data movement across multiple levels of a memory hierarchy (e.g., hard drives, flash storage, RAM, cache, registers, etc.) to a processing element (PE) array. Reducing data transfer and increasing (e.g., maximizing) data reuse and resource utilization can improve energy efficiency. Due to the nature of DNN and/or other AI engines, both inputs to the DNN (sometimes referred to as input activations and/or input feature maps) and weights (sometimes referred to as trained DNN model parameters) include sparse vectors. For example, input activation vectors and/or weight vectors can include a significant amount of zero elements due to rectifying operations in DNN layers. As illustrated above, utilizing ZVC data vectors and sparsity bitmaps can be an effective technique to accelerate the inference and training of a DNN as well as to reduce the storage requirement for parameters (e.g. compression) for energy efficiency” (Paragraph 25), “Examples disclosed herein advantageously reconstruct the sparsity bitmap at PEs on the fly according to the flexible tensor shapes” (Paragraph 30), “The byte select signals (e.g., Byte_Sel[0] through Byte_Sel[N]) determine the shape of the tensor (e.g., two by two by three, etc.) to be processed and the volume processed by each PE according to a schedule” (Paragraph 41), and “The example InSAD system 100 illustrated in FIG. 1 can be implemented with machine learning accelerators to reduce data movement. The example InSAD system 100 combines both flexible tensor distribution and sparse data compression by (1) decoding ZVC data vectors with software programed byte select signals (e.g., Byte_Sel[0]-Byte_Sel[N]) to distribute non-zero data to respective PE arrays, (2) reconstructing the sparsity bitmap at each PE on the fly for different tensor shapes, (3) eliminating one or more storage requirements for uncompressed data across on-chip memory hierarchy, and (4) serving different tensor shapes (e.g., one or more multi-dimension array dimensions) for each PE. The examples disclosed herein are applicable to various dataflow-based accelerators” (Paragraph 50). The examiner further notes that the execution of machine learning (which uses neural networks) on the ZVC (i.e. first data structure) and not the sparse vector (i.e. second data structure) is based off of shape data and the sparsity bitmap (i.e. the claimed binary map).
Regarding claim 16, Chinya further teaches a method comprising:
A) wherein the method is performed by one of: a Central Processing Unit (CPU) or a hardware accelerator (Paragraph 123).
The examiner notes that Chinya teaches “wherein the method is performed by one of: a Central Processing Unit (CPU) or a hardware accelerator” as “although individually listed, a plurality of means, elements or method actions may be implemented by, e.g., a single unit or processor” (Paragraph 123). The examiner further notes that the processor teaches the claimed CPU.
Regarding claim 17, Chinya further teaches a method comprising:
A) wherein the first data structure is a vector of contiguous non-zero values (Paragraph 23);
B) the vector being a 1×N array (Paragraph 23); and
C) wherein the first data structure is generated by transferring all non-zero values in the second data structure to the first data structure (Paragraph 23).
The examiner notes that Chinya teaches “wherein the first data structure is a vector of contiguous non-zero values” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero)” (Paragraph 23). The examiner further notes that the ZVC vector (i.e. the claimed first data structure) is an array vector of contiguous non-zero values. The examiner further notes that Chinya teaches “the vector being a 1×N array” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero)” (Paragraph 23). The examiner further notes that the example ZVC vector (i.e. the claimed first tensor) is an array vector of dimensions 1 x X, where X is 3. The examiner further notes that Chinya teaches “wherein the first data structure is generated by transferring all non-zero values in the second data structure to the first data structure” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero)” (Paragraph 23). The examiner further notes that the ZVC vector (i.e. the claimed first data structure) is resultant from “transferring” all non-zero elements of the sparse vector (i.e. the claimed second data structure).
Regarding claim 18, Chinya further teaches a method comprising:
A) wherein the shape data is computed based on a quantity of values in each axis of the second data structure (Paragraph 41).
The examiner notes that Chinya teaches “wherein the shape data is computed based on a quantity of values in each axis of the second data structure” as “The byte select signals (e.g., Byte_Sel[0] through Byte_Sel[N]) determine the shape of the tensor (e.g., two by two by three, etc.) to be processed and the volume processed by each PE according to a schedule” (Paragraph 41). The examiner further notes that the determined shape of a tensor is based on a quantity of values in each axis of that tensor.
Regarding claim 19, Chinya further teaches a method comprising:
A) generating a binary map (Paragraph 23);
B) each zero bit in the binary map indicating a corresponding position of a sparse value in the second data structure (Paragraph 23);
C) each one bit in the binary map indicating a corresponding position of a non-sparse value in the second data structure (Paragraph 23);
D) wherein the performing of the machine learning operation is further based on the generating of the binary map (Paragraphs 25, 30, 41, and 50).
The examiner notes that Chinya teaches “a binary map” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero)” (Paragraph 23). The examiner further notes that the example determined sparsity bitmap teaches the claimed binary map. The examiner further notes that Chinya teaches “each zero bit in the binary map indicating a corresponding position of a sparse value in the second data structure” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero)” (Paragraph 23). The examiner further notes that the zero-elements in the sparsity bitmap (i.e. the claimed binary map) correspond to the zero-elements of the sparse vector (i.e. the claimed second data structure). The examiner further notes that Chinya teaches “each one bit in the binary map indicating a corresponding position of a non-sparse value in the second data structure” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero)” (Paragraph 23). The examiner further notes that the one-elements in the sparsity bitmap (i.e. the claimed binary map) correspond to the non-zero elements of the sparse vector (i.e. the claimed second data structure). The examiner further notes that Chinya teaches “wherein the performing of the machine learning operation is further based on the generating of the binary map” as “Machine learning accelerators (e.g., those utilizing DNN engines, CNN engines, etc.) handle a large amount of tensor data (e.g., data stored in multi-dimensional data structures) for performing inference tasks. Processing large amounts of tensor data requires data movement across multiple levels of a memory hierarchy (e.g., hard drives, flash storage, RAM, cache, registers, etc.) to a processing element (PE) array. Reducing data transfer and increasing (e.g., maximizing) data reuse and resource utilization can improve energy efficiency. Due to the nature of DNN and/or other AI engines, both inputs to the DNN (sometimes referred to as input activations and/or input feature maps) and weights (sometimes referred to as trained DNN model parameters) include sparse vectors. For example, input activation vectors and/or weight vectors can include a significant amount of zero elements due to rectifying operations in DNN layers. As illustrated above, utilizing ZVC data vectors and sparsity bitmaps can be an effective technique to accelerate the inference and training of a DNN as well as to reduce the storage requirement for parameters (e.g. compression) for energy efficiency” (Paragraph 25). The examiner further notes that the execution of machine learning (which uses neural networks) on the ZVC is based off of the sparsity bitmap (i.e. the claimed binary map).
Regarding claim 20, Chinya further teaches a method comprising:
A) generating a second binary map of one bits and zero bits (Paragraphs 23 and 26);
B) the zero bits indicate the sparse values in a second output data structure and the one bits indicate non-sparse values in the second output data structure (Paragraphs 23 and 26);
C) based on the generating of the second binary map, generating a first output data structure that is a condensed representation of the second output data structure associated with the second data structure (Paragraphs 23 and 26); and
D) wherein the second output data structure includes sparse values and the first output data structure does not include any sparse values (Paragraphs 23 and 26); and
E) wherein the generating of the first output data structure is an output for the performing of the machine learning operation on the first data structure (Paragraphs 23 and 26).
The examiner notes that Chinya teaches “generating a second binary map of one bits and zero bits” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero)” (Paragraph 23) and “A non-linear function (e.g. rectified linear unit (ReLu)), is applied to the output activations which become the input activations for the next layer. In some DNNs, a significant fraction of each DNN layer's activations and weights are zero-valued due to ReLu operations, hence this data can be compressed via various techniques to save the on-chip storage requirements and bandwidth demands” (Paragraph 26). The examiner further notes that a second sparsity bitmap (i.e. the claimed binary map) includes only one and zero bits and can be generated. The examiner further notes that Chinya teaches “the zero bits indicate the sparse values in a second output data structure and the one bits indicate non-sparse values in the second output data structure” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero)” (Paragraph 23) and “A non-linear function (e.g. rectified linear unit (ReLu)), is applied to the output activations which become the input activations for the next layer. In some DNNs, a significant fraction of each DNN layer's activations and weights are zero-valued due to ReLu operations, hence this data can be compressed via various techniques to save the on-chip storage requirements and bandwidth demands” (Paragraph 26). The examiner further notes that a second sparsity bitmap (i.e. the claimed binary map) includes only one and zero bits that correspond to zero values and non-zero values of a corresponding sparse vector (i.e. second output data structure). The examiner further notes that Chinya teaches “based on the generating of the second binary map, generating a first output data structure that is a condensed representation of the second output data structure associated with the second data structure” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero)” (Paragraph 23) and “A non-linear function (e.g. rectified linear unit (ReLu)), is applied to the output activations which become the input activations for the next layer. In some DNNs, a significant fraction of each DNN layer's activations and weights are zero-valued due to ReLu operations, hence this data can be compressed via various techniques to save the on-chip storage requirements and bandwidth demands” (Paragraph 26). The examiner further notes that a generated ZVC vector (i.e. first output data structure) is a condensed representation of a sparse vector (i.e. second output data structure). The examiner further notes that Chinya teaches “wherein the second output data structure includes sparse values and the first output data structure does not include any sparse values” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero)” (Paragraph 23) and “A non-linear function (e.g. rectified linear unit (ReLu)), is applied to the output activations which become the input activations for the next layer. In some DNNs, a significant fraction of each DNN layer's activations and weights are zero-valued due to ReLu operations, hence this data can be compressed via various techniques to save the on-chip storage requirements and bandwidth demands” (Paragraph 26). The examiner further notes that a generated ZVC vector (i.e. first output data structure) does not include zero values whereas a sparse vector (i.e. second output data structure) does include zero values. The examiner further notes that Chinya teaches “wherein the generating of the first output data structure is an output for the performing of the machine learning operation on the first data structure” as “a zero-value-compression (ZVC) data vector is a vector that includes all non-zero elements of a vector in the same order as a sparse vector, but excludes all zero elements. As defined herein, a sparse vector is an input vector including both non-zero elements and zero elements. As defined herein, a dense vector is an input vector including all non-zero elements. As such, an example sparse vector [0, 0, 5, 0, 18, 0, 4, 0] corresponds to an example ZVC data vector [5, 18, 4]. As defined herein, a sparsity bitmap is a vector that includes one-bit elements identifying whether respective elements of the sparse vector are zero or non-zero. Thus, a sparsity bitmap may map non-zero values of a sparse vector to ‘1’ and may map zero values of the sparse vector to ‘0’. For the above example sparse vector of [0, 0, 5, 0, 18, 0, 4, 0], an example sparsity bitmap may be [0, 0, 1, 0, 1, 0, 1, 0] (e.g., because the third, fifth, seventh, and eight elements of the sparse vector are non-zero)” (Paragraph 23) and “A non-linear function (e.g. rectified linear unit (ReLu)), is applied to the output activations which become the input activations for the next layer. In some DNNs, a significant fraction of each DNN layer's activations and weights are zero-valued due to ReLu operations, hence this data can be compressed via various techniques to save the on-chip storage requirements and bandwidth demands” (Paragraph 26). The examiner further notes that the application of ReLu on output (which then becomes input) teaches the aforementioned.
Claim Rejections - 35 USC § 103
7. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
8. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
9. Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Chinya et al. (U.S. PGPUB 2020/0228137) as applied to claims 1, 3-6, 8-13, and 15-20 above, and further in view of Choe et al. (U.S. PGPUB 2022/0100669).
10. Regarding claim 2, Chinya further teaches a system comprising:
A) wherein the one or more circuitry components are included in one of: a Graphics Processing Unit (GPU), a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), or a Tensor Processing Unit (TPU) (Paragraph 37).
The examiner notes that Chinya teaches “wherein the one or more circuitry components are included in one of: a Graphics Processing Unit (GPU), a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), or a Tensor Processing Unit (TPU)” as “In the example of FIG. 1, the software compiler 108 is implemented as a program executing on a processor. In additional or alternative examples, the software compiler 108 can be implemented by one or more analog or digital circuit(s), logic circuits, programmable processor(s), programmable controller(s), graphics processing unit(s) (GPU(s)), digital signal processor(s) (DSP(s)), application specific integrated circuit(s) (ASIC(s)), programmable logic device(s) (PLD(s)) and/or field programmable logic device(s) (FPLD(s))” (Paragraph 37). The examiner further notes that the graphics processing unit(s) teaches the claimed GPU.
Chinya does not explicitly teach:
B) wherein the hardware accelerator excludes a Central Processing Unit (CPU).
Choe, however, teaches “wherein the hardware accelerator excludes a Central Processing Unit (CPU)” as “the accelerator circuit 200 may be a neural network processing unit, an AI accelerator, a CPU, a graphical processing unit (GPU), a digital signal processing unit (DSP), a neural processing unit (NPU), a coprocessor, or another suitable processor” (Paragraph 28).
The examiner further notes that although Chinya teaches an accelerator, there is no explicit teaching of the exclusion of a CPU. Nevertheless, Choe teaches that an accelerator circuit can be varied, including as an AI accelerator ((which entails that it is not CPU) (See also the diction of “or” in the citation)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of instant invention to combine the teachings of the cited references because teaching Choe’s would have allowed Chinya’s to provide a method for expanding the accelerating of computations, as noted by Choe (Abstract, Paragraph 28).
11. Claims 7 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Chinya et al. (U.S. PGPUB 2020/0228137) as applied to claims 1, 3-6, 8-13, and 15-20 above, and further in view of Xiao (Article entitled “Optimizing the Serving System for Large Language Model Inference”, dated 2022).
12. Regarding claim 7, Chinya does not explicitly teach a system comprising:
A) wherein the neural network operation is performed via a Large Language Model (LLM).
Xiao, however, teaches “wherein the neural network operation is performed via a Large Language Model (LLM)” as “The transformer is the core of most LLMs nowadays” (Section 3, Page 11) and “A tensor of shape [L,H] is fed into the transformer layer, where L is some sequence length and H is the hidden size of the model. It is passed through the QKVLinear sublayer, which is a matrix multiplication with a weight tensor [H,3H], and split into three tensors Q, K, and V , each of shape [L,H], known as the query, key, and value” (Section 3, Pages 11-12).
The examiner further notes that although Chinya teaches a neural network operation on tensors, there is no explicit teaching of an LLM. Nevertheless, Xiao teaches the concept of using an LLM to perform an operation on a tensor. The combination would result in using an LLM to perform operations on the tensors of Chinya.
It would have been obvious to one of ordinary skill in the art before the effective filing date of instant invention to combine the teachings of the cited references because teaching Xiao’s would have allowed Chinya’s to provide a method for revolutionizing tasks while avoiding redundant computations, as noted by Xiao (Abstract, Section 3, Page 13).
Regarding claim 14, Chinya does not explicitly teach a computer-implemented method comprising:
A) wherein the performing of the machine learning operation is performed via a Large Language Model (LLM).
Xiao, however, teaches “wherein the performing of the machine learning operation is performed via a Large Language Model (LLM)” as “The transformer is the core of most LLMs nowadays” (Section 3, Page 11) and “A tensor of shape [L,H] is fed into the transformer layer, where L is some sequence length and H is the hidden size of the model. It is passed through the QKVLinear sublayer, which is a matrix multiplication with a weight tensor [H,3H], and split into three tensors Q, K, and V , each of shape [L,H], known as the query, key, and value” (Section 3, Pages 11-12).
The examiner further notes that although Chinya teaches a neural network operation on tensors, there is no explicit teaching of an LLM. Nevertheless, Xiao teaches the concept of using an LLM to perform an operation on a tensor. The combination would result in using an LLM to perform operations on the tensors of Chinya.
It would have been obvious to one of ordinary skill in the art before the effective filing date of instant invention to combine the teachings of the cited references because teaching Xiao’s would have allowed Chinya’s to provide a method for revolutionizing tasks while avoiding redundant computations, as noted by Xiao (Abstract, Section 3, Page 13).
Choe et al. (U.S. PGPUB 2022/0100669).
Xiao (Article entitled “Optimizing the Serving System for Large Language Model Inference”, by Xiao, dated 2022).
Conclusion
13. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
U.S. PGPUB 2022/0207345 issued to Khaitan et al. on 30 June 2022. The subject matter disclosed therein is pertinent to that of claims 1-20 (e.g., methods to implement accelerators).
U.S. Patent 11,392,829 issued to Pool et al. on 19 July 2022. The subject matter disclosed therein is pertinent to that of claims 21-40 (e.g., methods to interact with visualizations).
Contact Information
14. Any inquiry concerning this communication or earlier communications from the examiner should be directed to Mahesh Dwivedi whose telephone number is (571) 272-2731. The examiner can normally be reached on Monday to Friday 8:20 am – 4:40 pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Charles Rones can be reached (571) 272-4085. The fax number for the organization where this application or proceeding is assigned is (571) 273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Mahesh Dwivedi
Primary Examiner
Art Unit 2168
August 03, 2026
/MAHESH H DWIVEDI/Primary Examiner, Art Unit 2168
Prosecution Insights