DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 02/26/2026 has been entered.
This action is in response to arguments filed on 02/26/2026. Claims 1, 3-7, 9-12, 14, 16-20 and 23-30 are pending in the application and have been considered below.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 3-4, 14, 17, 20, 22-23, 27 and 29-30 are rejected under 35 U.S.C. 103 as being unpatentable over Shin et al. (US 2021/0357748 A1, hereinafter referred to as Shin), in view of Xiao et al. (US 11,113,601 B1, hereinafter referred to as Xiao).
As to claim 1, Shin teaches a computing system for executing a neural network comprising:
(paragraphs [0063], general purpose or special purpose central processing units (CPUs); [0014]-[0015] In some embodiments, the method further includes: multiplying, in a first processing element circuit, the first row by a first vector of activations, to form a first dot product, multiplying, in a second processing element circuit, a second row of weights, of the first pre-processed weight tensor, by a second vector of activations, to form a second dot product, and adding the first product and the second product; [0023]- [0024], multiply a third row of weights, of the first pre-processed weight tensor, by a second vector of activations, to form a second dot product, and add the first dot product and the second dot product ; [0048] In computations performed for a neural network, e.g., a convolutional neural network, various operations may be performed on tensors of weights and tensors of activations (the latter of which may be referred to as the input feature map (IFM))); and
a memory storing executable program instructions, which when executed by the processor, cause the computing system to (paragraphs [0064]-[0065], the term "array" refers to an ordered set of numbers regardless of how stored (e.g., whether stored in consecutive memory locations, or in a linked list):
identify an assignment of portions of the weight tensor to respective ones of the plurality of processing elements (paragraph [0014]-[0015] In some embodiments, the method further includes: multiplying, in a first processing element circuit, the first row by a first vector of activations, to form a first dot product, multiplying, in a second processing element circuit, a second row of weights, of the first pre-processed weight tensor, by a second vector of activations, to form a second dot product, and adding the first product and the second product; [0049]-[0050], wherein Examiner interprets “array of weights” as assignment of portions of the weight tensor);
generate sparsity representations for the portions of the weight tensor, a sparsity representation of a portion of the weight tensor having a same number of elements as the portion of the weight tensor, each element of the sparsity representation corresponding to a weight of the portion of the weight tensor and indicating whether the corresponding weight is a zero value or a non-zero value (paragraphs [0010] creating a tile sparsity map corresponding to the first pre-processed weight tensor, the tile sparsity map having: a column for each weight tile of the first preprocessed weight tensor, and a row for each row of the weight tiles, the tile sparsity map indicating positions of empty rows of the weight tiles of the first pre-processed weight tensor; and [0022], creating a tile sparsity map corresponding to the first pre-processed weight tensor, the tile sparsity map having: a column for each weight tile of the first preprocessed weight tensor, and a row for each row of the weight tiles, the tile sparsity map indicating positions of empty rows of the weight tiles of the first pre-processed weight tensor; [0048]-[0049] For example, referring to FIG. 2A, each array of the arrays of weights for four tiles may include some elements that are zero, or "empty" (illustrated as unshaded squares) and some elements that are nonzero (illustrated as shaded squares). The array of weights corresponding to a tile may be referred to as a "weight tile". In each of the first, second, and third weight tiles, an element labeled "lookahead" will, if processed one computation...; [0054] To perform inter-tile preprocessing, a tile sparsity map (having one fewer dimension than the weight tensor) may first be generated; [0054] ( ) A tile sparsity map for the first pre-processed weight tensor of FIG. 2B is shown in FIG. 2C. The tile sparsity map may have one element for each row of each weight tile of the first pre-processed weight tensor, the element being a zero if the row is empty, and the element being a one if the row is not empty).
However, Shin fails to explicitly teach:
store the sparsity representations and the portions of the weight tensor into partitions of a data storage structure based on the assignment of the portions of the weight tensor, wherein the partitions of the data storage structure each include a fixed number of bytes for a corresponding sparsity representation and another fixed number of bytes for a corresponding portion of the weight tensor.
Xiao, in combination with Shin, teaches:
store the sparsity representations and the portions of the weight tensor into partitions of a data storage structure based on the assignment of the portions of the weight tensor, wherein the partitions of the data storage structure each include a fixed number of bytes for a corresponding sparsity representation and another fixed number of bytes for a corresponding portion of the weight tensor (col. 5, lines 25-49 In some embodiments, each of the sparse filters may be segmented into a plurality of sub-filters that may be stored in a memory-efficient layout. For example, after being pruned and segmented, each of the sparse filters may be segmented in a way where each of the sub-filters comprises a same number of non-zero weights. Since the non-zero weights are evenly distributed into the plurality of sub filters, the convolution processing using these sub-filters may be referred to as a balanced-weight sparse convolution. In some embodiments, the non-zero weights within each sub-filter may be stored as index-value pairs, which may then be combined and stored as a fixed number of bits or bytes within the memory to represent the sub-filter…).
It would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify the system Shin to add a data storage with fixed number of bytes to the system of Shin, as taught by Xiao, above. The modification would have been obvious because one of ordinary skill would be motivated to assure the pairs of sub-tensor and sub-filter assigned to different PEs are independent from
each other so that the plurality of PEs may execute the local operations in parallel to boost performance, as suggested by Xiao (col.5, lines 43-49).
As to claim 3, which incorporates the rejection of claim 1, Shin teaches:
remove one or more zero weights from weights assigned to a processing element of the plurality of processing elements to generate a compressed group of weights (paragraph [0049], skip performing multiplications when some of the elements of the weight tensor are zeros. For example, referring to FIG. 2A, each array of the arrays of weights for four tiles may include some elements that are zero, or "empty" (illustrated as unshaded squares) and some elements that are nonzero (illustrated as shaded squares). The array of weights corresponding to a tile may be referred to as a "weight tile". In each of the first, second, and third weight tiles, an element labeled "lookahead" will, if processed one computation cycle earlier, cause the row it currently occupies to become entirely empty (i.e., all of the elements of the row will be zero), which means that the processing could skip the row and proceed directly to the next row).
However, Shin fails to explicitly teach:
determine that a total number of weights in the compressed group of the weights is less than another fixed number;
insert one or more zero values into the compressed group of weights to generate a new group of weights; and
store the new group of weights in a partition of the data storage structure corresponding to the processing element.
Xiao, in combination with Shin, teaches wherein the instructions, when executed by the processor, further cause the computing system to:
determine that a total number of weights in the compressed group of the weights is less than the another fixed number (col. 5, lines 25-37, the non-zero weights within each sub-filter may be stored as index-value pairs, which may then be combined and stored as a fixed number of bits or bytes within the memory to represent the sub-filter; wherein Examiner interprets; wherein using the broadest reasonable interpretation, Examiner interprets “the non-zero weights within each sub-filter to be less than the combined and stored as a fixed number of bits or bytes);
insert one or more zero values into the compressed group of weights to generate a new group of weights (col.5, lines 1-24, the filters may be pruned or sparsified by introducing a large number of zeros to reduce the computation cost and improve inferencing speed. For example, more than 20% or 50% of the values in the filters
may be set to zeros…After each iteration of convolution process, the sub-tensor assigned to the PE as an input may be updated and evolved, output, which may be used as the input for the next iteration of convolution process); and
store the new group of weights in a partition of the data storage structure corresponding to the processing element (col. 5, lines 25-49 In some embodiments, each of the sparse filters may be segmented into a plurality of sub-filters that may be stored in a memory-efficient layout. For example, after being pruned and segmented, each of the sparse filters may be segmented in a way where each of the sub-filters comprises a same number of non-zero weights. Since the non-zero weights are evenly distributed into the plurality of sub filters, the convolution processing using these sub-filters may be referred to as a balanced-weight sparse convolution. In some embodiments, the non-zero weights within each sub-filter may be stored as index-value pairs, which may then be combined and stored as a fixed number of bits or bytes within the memory to represent the sub-filter).
It would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify the system Shin to add a data storage with fixed number of bytes to the system Shin, as taught by Xiao, above. The modification would have been obvious because one of ordinary skill would be motivated to assure the pairs of sub-tensor and sub-filter assigned to different PEs are independent from each other so that the plurality of PEs may execute the local operations in parallel to boost performance, as suggested by Xiao (col.5, lines 43-49).
As to claim 4, which incorporates the rejection of claim 1, Shin teaches:
wherein the operation further has an activation tensor, wherein the instructions, when executed by the processor, further cause the computing system to:
in response to determining that the activation corresponding to the next element is zero, bypass a next load process associated with the next element (paragraph [0049], system to skip performing multiplications when some of the elements of the weight tensor are zeros).
However, Shin fails to explicitly teach:
obtain a sparsity representation of the activation tensor, the sparsity representation comprising sequence of elements, each of which corresponds to a respective activation in the activation tensor and is associated with a separate load process; and
during a load process associated with an element in the sparsity representation,
determine whether an activation corresponding to the element is zero and whether an activation corresponding to a next element in the sparsity representation is zero; and
in response to determining that the activation corresponding to the next element is zero, bypass a next load process associated with the next element.
Xiao, in combination with Shin, teaches:
wherein the operation further has an activation tensor, wherein the instructions, when executed by the processor, further cause the computing system to:
obtain a sparsity representation of the activation tensor, the sparsity representation comprising sequence of elements, each of which corresponds to a respective activation in the activation tensor and is associated with a separate load process (col. 6, lines 56-67, in some cases, if there are multiple input tensors for a convolutional layer (e.g., when there are multiple images input into the convolution layer, or there are multiple input activation tensors received from a previous layer), each input tensor may be represented in an NHWC format, where N refers to an index of the input tensor within the batch of input tensors. In the following description, N may be omitted for simplicity (e.g., assuming there is only one input tensor) unless explicitly stated otherwise. It may be obvious for a person in the art to expand the embodiments to cover the cases with N>l); and
during a load process associated with an element in the sparsity representation,
determine whether an activation corresponding to the element is zero; and whether an activation corresponding to a next element in the sparsity representation is zero (col. 6, lines 56-67, in some cases, if there are multiple input tensors for a convolutional layer (e.g., when there are multiple images input into the convolution layer, or there are multiple input activation tensors received from a previous layer), each input tensor may be represented in an NHWC format, where N refers to an index of the input tensor within the batch of input tensors. In the following description, N may be omitted for simplicity (e.g., assuming there is only one input tensor) unless explicitly stated otherwise. It may be obvious for a person in the art to expand the embodiments to cover the cases with N>l).
It would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify the system Shin to add a data storage with fixed number of bytes to the system of Shin, as taught by Xiao, above. The modification would have been obvious because one of ordinary skill would be motivated to assure the pairs of sub-tensor and sub-filter assigned to different PEs are independent from
each other so that the plurality of PEs may execute the local operations in parallel to boost performance, as suggested by Xiao (col.5, lines 43-49).
As to claim 14, Shin teaches at least one computer readable storage medium comprising a set of instructions, which when executed by a computing device, cause the computing device to:
identify an assignment of portions of a weight tensor to respective ones of a plurality of processing elements, wherein the weight tensor is associated with an operation in a neural network (paragraph [0014]-[0015] In some embodiments, the method further includes: multiplying, in a first processing element circuit, the first row by a first vector of activations, to form a first dot product, multiplying, in a second processing element circuit, a second row of weights, of the first pre-processed weight tensor, by a second vector of activations, to form a second dot product, and adding the first product and the second product; [0049]-[0050], wherein Examiner interprets “array of weights” as assignment of portions of the weight tensor);
generate sparsity representations for the portions of the weight tensor, a sparsity representation of a portion of the weight tensor having a same number of elements as the portion of the weight tensor, each element of the sparsity representation corresponding to a weight of the portion of the weight tensor and indicating whether the corresponding weight is a zero value or a non-zero value (paragraphs [0010] creating a tile sparsity map corresponding to the first pre-processed weight tensor, the tile sparsity map having: a column for each weight tile of the first preprocessed weight tensor, and a row for each row of the weight tiles, the tile sparsity map indicating positions of empty rows of the weight tiles of the first pre-processed weight tensor; and [0022], the inter-tile preprocessing further includes creating a tile sparsity map corresponding to the first pre-processed weight tensor, the tile sparsity map having: a column for each weight tile of the first preprocessed weight tensor, and a row for each row of the weight tiles, the tile sparsity map indicating positions of empty rows of the weight tiles of the first pre-processed weight tensor; [0049] For example, referring to FIG. 2A, each array of the arrays of weights for four tiles may include some elements that are zero, or "empty" (illustrated as unshaded squares) and some elements that are nonzero (illustrated as shaded squares). The array of weights corresponding to a tile may be referred to as a "weight tile". In each of the first, second, and third weight tiles, an element labeled "lookahead" will, if processed one computation...; [0054] To perform inter-tile preprocessing, a tile sparsity
map (having one fewer dimension than the weight tensor) may first be generated.).
However, Shin fails to explicitly teach:
store the sparsity representations and the portions of the weight tensor into partitions of a data storage structure based on the assignment of the portions of the weight tensor, wherein the partitions of the data storage structure each include a fixed number of bytes for a corresponding sparsity representation and another fixed number of bytes for a corresponding portion of the weight tensor.
Xiao, in combination with Shin, teaches:
store the sparsity representations and the portions of the weight tensor into partitions of a data storage structure based on the assignment of the portions of the weight tensor, wherein the partitions of the data storage structure each include a fixed number of bytes for a corresponding sparsity representation and another fixed number of bytes for a corresponding portion of the weight tensor (col. 5, lines 25-49 In some embodiments, each of the sparse filters may be segmented into a plurality of sub-filters that may be stored in a memory-efficient layout. For example, after being pruned and segmented, each of the sparse filters may be segmented in a way where each of the sub-filters comprises a same number of non-zero weights. Since the non-zero weights are evenly distributed into the plurality of sub filters, the convolution processing using these sub-filters may be referred to as a balanced-weight sparse convolution. In some embodiments, the non-zero weights within each sub-filter may be stored as index-value pairs, which may then be combined and stored as a fixed number of bits or bytes within the memory to represent the sub-filter).
It would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify the system Shin to add a data storage with fixed number of bytes to the system of Shin, as taught by Xiao, above. The modification would have been obvious because one of ordinary skill would be motivated to assure the pairs of sub-tensor and sub-filter assigned to different PEs are independent from
each other so that the plurality of PEs may execute the local operations in parallel to boost performance, as suggested by Xiao (col.5, lines 43-49).
As to claim 16, which incorporates the rejection of claim 14, Shin teaches:
teaches:
remove one or more zero weights from weights assigned to a processing element of the plurality of processing elements to generate a compressed group of weights (paragraph [0049], skip performing multiplications when some of the elements of the weight tensor are zeros. For example, referring to FIG. 2A, each array of the arrays of weights for four tiles may include some elements that are zero, or "empty" (illustrated as unshaded squares) and some elements that are nonzero (illustrated as shaded squares). The array of weights corresponding to a tile may be referred to as a "weight tile". In each of the first, second, and third weight tiles, an element labeled "lookahead" will, if processed one computation cycle earlier, cause the row it currently occupies to become entirely empty (i.e., all of the elements of the row will be zero), which means that the processing could skip the row and proceed directly to the next row).
However, Shin fails to explicitly teach:
determine that a total number of weights in the compressed group of the weights is less than another fixed number;
insert one or more zero values into the compressed group of weights to generate a new group of weights; and
store the new group of weights in a partition of the data storage structure corresponding to the processing element.
Xiao, in combination with Shin, teaches wherein the instructions, when executed by the processor, further cause the computing system to:
determine that a total number of weights in the compressed group of the weights is less than the another fixed number (col. 5, lines 25-37, the non-zero weights within each sub-filter may be stored as index-value pairs, which may then be combined and stored as a fixed number of bits or bytes within the memory to represent the sub-filter; ; wherein using the broadest reasonable interpretation, Examiner interprets “the non-zero weights within each sub-filter to be less than the combined and stored as a fixed number of bits or bytes);
insert one or more zero values into the compressed group of weights to generate a new group of weights (col.5, lines 1-24, the filters may be pruned or sparsified by introducing a large number of zeros to reduce the computation cost and improve inferencing speed. For example, more than 20% or 50% of the values in the filters
may be set to zeros…After each iteration of convolution process, the sub-tensor assigned to the PE as an input may be updated and evolved, output, which may be used as the input for the next iteration of convolution process); and
store the new group of weights in a partition of the data storage structure corresponding to the processing element (col. 5, lines 25-49 In some embodiments, each of the sparse filters may be segmented into a plurality of sub-filters that may be stored in a memory-efficient layout. For example, after being pruned and segmented, each of the sparse filters may be segmented in a way where each of the sub-filters comprises a same number of non-zero weights. Since the non-zero weights are evenly distributed into the plurality of sub filters, the convolution processing using these sub-filters may be referred to as a balanced-weight sparse convolution. In some embodiments, the non-zero weights within each sub-filter may be stored as index-value pairs, which may then be combined and stored as a fixed number of bits or bytes within the memory to represent the sub-filter).
It would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify the system Shin to add a data storage with fixed number of bytes to the system Shin, as taught by Xiao, above. The modification would have been obvious because one of ordinary skill would be motivated to assure the pairs of sub-tensor and sub-filter assigned to different PEs are independent from each other so that the plurality of PEs may execute the local operations in parallel to boost performance, as suggested by Xiao (col.5, lines 43-49).
As to claim 17, which incorporates the rejection of claim 14, Shin teaches:
wherein the operation further has an activation tensor, wherein the instructions, when executed by the processor, further cause the computing system to:
in response to determining that the activation corresponding to the next element is zero, bypass a next load process associated with the next element (paragraph [0049], system to skip performing multiplications when some of the elements of the weight tensor are zeros).
However, Shin fails to explicitly teach:
obtain a sparsity representation of the activation tensor, the sparsity representation comprising sequence of elements, each of which corresponds to a respective activation in the activation tensor and is associated with a separate load process; and
during a load process associated with an element in the sparsity representation,
determine whether an activation corresponding to the element is zero and whether an activation corresponding to a next element in the sparsity representation is zero; and
in response to determining that the activation corresponding to the next element is zero, bypass a next load process associated with the next element.
Xiao, in combination with Shin, teaches:
wherein the operation further has an activation tensor, wherein the instructions, when executed by the processor, further cause the computing system to:
obtain a sparsity representation of the activation tensor, the sparsity representation comprising sequence of elements, each of which corresponds to a respective activation in the activation tensor and is associated with a separate load process (col. 6, lines 56-67, in some cases, if there are multiple input tensors for a convolutional layer (e.g., when there are multiple images input into the convolution layer, or there are multiple input activation tensors received from a previous layer), each input tensor may be represented in an NHWC format, where N refers to an index of the input tensor within the batch of input tensors. In the following description, N may be omitted for simplicity (e.g., assuming there is only one input tensor) unless explicitly stated otherwise. It may be obvious for a person in the art to expand the embodiments to cover the cases with N>l); and
during a load process associated with an element in the sparsity representation,
determine whether an activation corresponding to the element is zero; and whether an activation corresponding to a next element in the sparsity representation is zero (col. 6, lines 56-67, in some cases, if there are multiple input tensors for a convolutional layer (e.g., when there are multiple images input into the convolution layer, or there are multiple input activation tensors received from a previous layer), each input tensor may be represented in an NHWC format, where N refers to an index of the input tensor within the batch of input tensors. In the following description, N may be omitted for simplicity (e.g., assuming there is only one input tensor) unless explicitly stated otherwise. It may be obvious for a person in the art to expand the embodiments to cover the cases with N>l).
It would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify the system Shin to add a data storage with fixed number of bytes to the system of Shin, as taught by Xiao, above. The modification would have been obvious because one of ordinary skill would be motivated to assure the pairs of sub-tensor and sub-filter assigned to different PEs are independent from
each other so that the plurality of PEs may execute the local operations in parallel to boost performance, as suggested by Xiao (col.5, lines 43-49).
As to claim 20, Shin teaches a method comprising:
identifying an assignment of portions of a weight tensor to respective ones of a plurality of processing elements, wherein the weight tensor is associated with an operation in a neural network (paragraph [0014]-[0015] In some embodiments, the method further includes: multiplying, in a first processing element circuit, the first row by a first vector of activations, to form a first dot product, multiplying, in a second processing element circuit, a second row of weights, of the first pre-processed weight tensor, by a second vector of activations, to form a second dot product, and adding the first product and the second product; [0049]-[0050], wherein Examiner interprets “array of weights” as assignment of portions of the weight tensor);
generating sparsity representations for the portions of the weight tensor, a sparsity representation of a portion of the weight tensor having a same number of elements as the portion of the weight tensor, each element of the sparsity representation corresponding to a weight of the portion of the weight tensor and indicating whether the corresponding weight is a zero value or a non-zero value (paragraphs [0010] creating a tile sparsity map corresponding to the first pre-processed weight tensor, the tile sparsity map having: a column for each weight tile of the first preprocessed weight tensor, and a row for each row of the weight tiles, the tile sparsity map indicating positions of empty rows of the weight tiles of the first pre-processed weight tensor; and [0022], the inter-tile preprocessing further includes creating a tile sparsity map corresponding to the first pre-processed weight tensor, the tile sparsity map having: a column for each weight tile of the first preprocessed weight tensor, and a row for each row of the weight tiles, the tile sparsity map indicating positions of empty rows of the weight tiles of the first pre-processed weight tensor; [0049] For example, referring to FIG. 2A, each array of the arrays of weights for four tiles may include some elements that are zero, or "empty" (illustrated as unshaded squares) and some elements that are nonzero (illustrated as shaded squares). The array of weights corresponding to a tile may be referred to as a "weight tile". In each of the first, second, and third weight tiles, an element labeled "lookahead" will, if processed one computation...; [0054] To perform inter-tile preprocessing, a tile sparsity map (having one fewer dimension than the weight tensor) may first be generated).
However, Shin fails to explicitly teach:
storing the sparsity representations and the portions of the weight tensor into partitions of a data storage structure based on the assignment of the portions of the weight tensor, wherein the partitions of the data storage structure each
include a fixed number of bytes for a corresponding sparsity representation and another fixed number of bytes for a corresponding portion of the weight tensor.
Xiao, in combination with Shin, teaches:
storing the sparsity representations and the portions of the weight tensor into partitions of a data storage structure based on the assignment of the portions of the weight tensor, wherein the partitions of the data storage structure each
include a fixed number of bytes for a corresponding sparsity representation and another fixed number of bytes for a corresponding portion of the weight tensor (col. 5, lines 25-49 In some embodiments, each of the sparse filters may be segmented into a plurality of sub-filters that may be stored in a memory-efficient layout. For example, after being pruned and segmented, each of the sparse filters may be segmented in a way where each of the sub-filters comprises a same number of non-zero weights. Since the non-zero weights are evenly distributed into the plurality of sub filters, the convolution processing using these sub-filters may be referred to as a balanced-weight sparse convolution. In some embodiments, the non-zero weights within each sub-filter may be stored as index-value pairs, which may then be combined and stored as a fixed number of bits or bytes within the memory to represent the sub-filter).
It would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify the combination system Shin and Appu to add a data storage with fixed number of bytes to the combination system Shin and Appu, as taught by Xiao, above. The modification would have been obvious because one of ordinary skill would be motivated to assure the pairs of sub-tensor and sub-filter assigned to different PEs are independent from each other so that the plurality of PEs may execute the local operations in parallel to boost performance, as suggested by Xiao (col.5, lines 43-49).
As to claim 22, which incorporates the rejection of claim 20, Shin teaches:
teaches:
removing one or more zero weights from weights assigned to a processing element of the plurality of processing elements to generate a compressed group of weights (paragraph [0049], skip performing multiplications when some of the elements of the weight tensor are zeros. For example, referring to FIG. 2A, each array of the arrays of weights for four tiles may include some elements that are zero, or "empty" (illustrated as unshaded squares) and some elements that are nonzero (illustrated as shaded squares). The array of weights corresponding to a tile may be referred to as a "weight tile". In each of the first, second, and third weight tiles, an element labeled "lookahead" will, if processed one computation cycle earlier, cause the row it currently occupies to become entirely empty (i.e., all of the elements of the row will be zero), which means that the processing could skip the row and proceed directly to the next row).
However, Shin fails to explicitly teach:
determining that a total number of weights in the compressed group of the weights is less than another fixed number;
inserting one or more zero values into the compressed group of weights to generate a new group of weights; and
storing the new group of weights in a partition of the data storage structure corresponding to the processing element.
Xiao, in combination with Shin, teaches wherein the instructions, when executed by the processor, further cause the computing system to:
determining that a total number of weights in the compressed group of the weights is less than the another fixed number (col. 5, lines 25-37, the non-zero weights within each sub-filter may be stored as index-value pairs, which may then be combined and stored as a fixed number of bits or bytes within the memory to represent the sub-filter; wherein Examiner interprets; wherein using the broadest reasonable interpretation, Examiner interprets “the non-zero weights within each sub-filter to be less than the combined and stored as a fixed number of bits or bytes);
inserting one or more zero values into the compressed group of weights to generate a new group of weights (col.5, lines 1-24, the filters may be pruned or sparsified by introducing a large number of zeros to reduce the computation cost and improve inferencing speed. For example, more than 20% or 50% of the values in the filters may be set to zeros…After each iteration of convolution process, the sub-tensor assigned to the PE as an input may be updated and evolved, output, which may be used as the input for the next iteration of convolution process); and
storing the new group of weights in a partition of the data storage structure corresponding to the processing element (col. 5, lines 25-49 In some embodiments, each of the sparse filters may be segmented into a plurality of sub-filters that may be stored in a memory-efficient layout. For example, after being pruned and segmented, each of the sparse filters may be segmented in a way where each of the sub-filters comprises a same number of non-zero weights. Since the non-zero weights are evenly distributed into the plurality of sub filters, the convolution processing using these sub-filters may be referred to as a balanced-weight sparse convolution. In some embodiments, the non-zero weights within each sub-filter may be stored as index-value pairs, which may then be combined and stored as a fixed number of bits or bytes within the memory to represent the sub-filter).
It would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify the system Shin to add a data storage with fixed number of bytes to the system Shin, as taught by Xiao, above. The modification would have been obvious because one of ordinary skill would be motivated to assure the pairs of sub-tensor and sub-filter assigned to different PEs are independent from each other so that the plurality of PEs may execute the local operations in parallel to boost performance, as suggested by Xiao (col.5, lines 43-49).
As to claim 26, which incorporates the rejection of claim 4, Shin teaches wherein the instructions, when executed by the processor, further cause the computing system to:
in response to determining that the activation corresponding to the next element is not zero, perform a next load process associated with the next element
(paragraph [0049], system to skip performing multiplications when some of the elements of the weight tensor are zeros).
As to claim 27, which incorporates the rejection of claim 4, Shin fails to explicitly teach wherein the sequence of elements is a sequence of bits.
Xiao teaches wherein the sequence of elements is a sequence of bits (col. 11, lines 11-14, depending on the implementation, each index-value pair may be represented by a number of bits, such as 8, 16, or 32 bits. These index-value pairs may then be aggregated as a series of bits or a bit array 440 to represent the sub-filter 430; col. 14, lines 37-44).
As to claim 29, which incorporates the rejection of claim 17, Shin teaches wherein the instructions, when executed, further cause the computing device to:
in response to determining that the activation corresponding to the next element is not zero, perform a next load process associated with the next element (paragraph [0049], system to skip performing multiplications when some of the elements of the weight tensor are zeros).
As to claim 30, which incorporates the rejection of claim 23, Shin teaches:
in response to determining that the activation corresponding to the next element is not zero, perform a next load process associated with the next element (paragraph [0049], system to skip performing multiplications when some of the elements of the weight tensor are zeros).
Claims 5, 18 and 24 are rejected under 35 U.S.C. 103 as being unpatentable over Shin et al. (US 2021/0357748 A1, hereinafter referred to as Shin), in view of Xiao et al. (US 11,113,601B1, hereinafter referred to as Xiao), and further in view of XIAO et al. (US 2021/0240684 A1, hereinafter referred to as XIAO).
As to claim 5, which incorporates the rejection of claim 1, Shin and Xiao fail to explicitly teach wherein the sparsity representation is a bitmap and each element of the sparsity representation is a bit.
However, XIAO, in combination with Shin and Xiao, teaches wherein the sparsity representation is a bitmap, and each element of the sparsity representation is a bit. (paragraphs [0153] The operation unit of any of clauses 13-15, wherein the representation comprises a first sub-level bitmap and a second sub-level bitmap, and the sparse engine includes circuitry to: [0154] decompress the first sub-level bitmap to determine whether a sub-block of a part of the second level bitmap comprises a non-zero element; [0126] first sub-level bit map; [0173]-[0174]).
It would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify the combination system Shin and Xiao to add a bitmap to the combination system of Shin and Xiao, as taught by XIAO, above. The modification would have been obvious because one of ordinary skill would be motivated to determine whether a sub-block of a part of the level bitmap comprises a non-zero element, as suggested by XIAO ([0154]).
As to claim 18, which incorporates the rejection of claim 14, Shin and Xiao fail to explicitly teach wherein the sparsity representation is a bitmap and each element of the sparsity representation is a bit.
However, XIAO, in combination with Shin and Xiao, teaches wherein the sparsity representation is a bitmap and each element of the sparsity representation is a bit. (paragraphs [0153] The operation unit of any of clauses 13-15, wherein the representation comprises a first sub-level bitmap and a second sub-level bitmap, and the sparse engine includes circuitry to: [0154] decompress the first sub-level bitmap to determine whether a sub-block of a part of the second level bitmap comprises a non-zero element; [0126] first sub-level bit map; [0173]-[0174]).
It would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify the combination system Shin and Xiao to add a bitmap to the combination system of Shin and Xiao, as taught by XIAO, above. The modification would have been obvious because one of ordinary skill would be motivated to determine whether a sub-block of a part of the level bitmap comprises a non-zero element, as suggested by XIAO ([0154]).
As to claim 24, which incorporates the rejection of claim 20, Shin and Xiao fail to explicitly teach wherein the sparsity representation is a bitmap and each element of the sparsity representation is a bit.
However, XIAO, in combination with Shin and Xiao, teaches wherein the sparsity representation is a bitmap and each element of the sparsity representation is a bit. (paragraphs [0153] The operation unit of any of clauses 13-15, wherein the representation comprises a first sub-level bitmap and a second sub-level bitmap, and the sparse engine includes circuitry to: [0154] decompress the first sub-level bitmap to determine whether a sub-block of a part of the second level bitmap comprises a non-zero element; [0126] first sub-level bit map; [0173]-[0174]).
It would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify the combination system Shin and Xiao to add a bitmap to the combination system of Shin and Xiao, as taught by XIAO, above. The modification would have been obvious because one of ordinary skill would be motivated to determine whether a sub-block of a part of the level bitmap comprises a non-zero element, as suggested by XIAO ([0154]).
Claims 6, 19 and 25 are rejected under 35 U.S.C. 103 as being unpatentable over Shin et al. (US 2021/0357748 A1, hereinafter referred to as Shin), in view of Xiao et al. (US 11,113,601B1, hereinafter referred to as Xiao), and further in view of XIAO et al. (US 2021/0240684 A1, hereinafter referred to as XIAO), and Appu et al. (US 2018/0308272 A1, hereinafter referred to as Appu).
As to claim 6, which incorporates the rejection of claim 5, Shin, Xiao and XIAO fail to explicitly teach wherein:
a first partition of the partitions is associated with a first lane for a first processing element of the plurality of processing elements, wherein the first partition stores one or more weights assigned to the first processing element; and a second partition of the partitions is associated with a second
Appu, in combination with Shin, Xiao and XIAO, teaches wherein:
a first partition of the partitions is associated with a first lane for a first processing element of the plurality of processing elements, wherein the first partition stores one or more weights assigned to a-the first processing element (paragraphs [0049] The number of partition units 220A-220N generally equals the number of memory units, such that a first partition unit 220A has a corresponding first memory unit 224A; [0095] Application effective address space 482 within system memory 411 stores process elements 483. In one embodiment, the process elements 483 are stored in response to GPU invocations 481 from applications 480 executed on the processor 407; [0102], process element list 499 is stored; [0197]); and
a second partition of the partitions is associated with a second (paragraphs [0049], a second partition unit 220B has a corresponding memory unit 224B, and an Nth partition unit 220N has a corresponding Nth memory unit 224N; [0095] Application effective address space 482 within system memory 411 stores process elements 483. In one embodiment, the process elements 483 are stored in response to GPU invocations 481 from applications 480 executed on the processor 407; [0102], process element list 499 is stored; [0197]);
It would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify the combination system Shin, Xiao and XIAO to add a bitmap to the combination system of Shin, Xiao and XIAO, as taught by Appu, above. The modification would have been obvious because one of ordinary skill would be motivated to allow for all or a subset of processes from all or a subset of partitions in the system to use a graphics acceleration module, as suggested by Appu ([0103]).
As to claim 19, which incorporates the rejection of claim 18, Shin and Xiao fail to explicitly teach wherein:
a first partition of the partitions is associated with a first lane for a first processing element of the plurality of processing elements, wherein the first partition stores one or more weights assigned to a-the first processing element; and a second partition of the partitions is associated with a second
Appu, in combination with Shin and Xiao, teaches wherein:
a first partition of the partitions is associated with a first lane for a first processing element of the plurality of processing elements, wherein the first partition stores one or more weights assigned to a-the first processing element ( (paragraphs [0049] The number of partition units 220A-220N generally equals the number of memory units, such that a first partition unit 220A has a corresponding first memory unit 224A; [0095] Application effective address space 482 within system memory 411 stores process elements 483. In one embodiment, the process elements 483 are stored in response to GPU invocations 481 from applications 480 executed on the processor 407; [0102], process element list 499 is stored [0197]); and
a second partition of the partitions is associated with a second (paragraphs [0049], a second partition unit 220B has a corresponding memory unit 224B, and an Nth partition unit 220N has a corresponding Nth memory unit 224N….; [0095] Application effective address space 482 within system memory 411 stores process elements 483. In one embodiment, the process elements 483 are stored in response to GPU invocations 481 from applications 480 executed on the processor 407; [0102], process element list 499 is stored; [0197]);
It would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify the combination system Shin and Xiao and Woo to add a bitmap to the combination system of Shin and Xiao, as taught by Appu, above. The modification would have been obvious because one of ordinary skill would be motivated to allow for all or a subset of processes from all or a subset of partitions in the system to use a graphics acceleration module, as suggested by Appu ([0103]).
As to claim 25, which incorporates the rejection of claim 4, Shin and Xiao fail to explicitly teach wherein:
a first partition of the partitions is associated with a first lane for a first processing element of the plurality of processing elements, wherein the first partition stores one or more weights assigned to a-the first processing element; and a second partition of the partitions is associated with a second
Appu, in combination with Shin and Xiao, teaches wherein:
a first partition of the partitions is associated with a first lane for a first processing element of the plurality of processing elements, wherein the first partition stores one or more weights assigned to a-the first processing element ( (paragraphs [0049] The number of partition units 220A-220N generally equals the number of memory units, such that a first partition unit 220A has a corresponding first memory unit 224A; [0095] Application effective address space 482 within system memory 411 stores process elements 483. In one embodiment, the process elements 483 are stored in response to GPU invocations 481 from applications 480 executed on the processor 407; [0102], process element list 499 is stored [0197]); and
a second partition of the partitions is associated with a second (paragraphs [0049], a second partition unit 220B has a corresponding memory unit 224B, and an Nth partition unit 220N has a corresponding Nth memory unit 224N….; [0095] Application effective address space 482 within system memory 411 stores process elements 483. In one embodiment, the process elements 483 are stored in response to GPU invocations 481 from applications 480 executed on the processor 407; [0102], process element list 499 is stored; [0197]);
It would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify the combination system of Shin and Xiao and Woo to add a bitmap to the combination system of Shin and Xiao, as taught by Appu, above. The modification would have been obvious because one of ordinary skill would be motivated to allow for all or a subset of processes from all or a subset of partitions in the system to use a graphics acceleration module, as suggested by Appu ([0103]).
Claims 7, 9-10 and 28 are rejected under 35 U.S.C. 103 as being unpatentable over Shin et al. (US 2021/0357748 A1, hereinafter referred to as Shin), in view of Appu et al. (US 2018/0308272 A1, hereinafter referred to as Appu), and further in view of Xiao et al. (US 11,113,601B1, hereinafter referred to as Xiao).
As to claim 7, Shin teaches an apparatus comprising:
[one or more substrates;
logic coupled to the one or more substrates, wherein the logic is implemented at least partly in one or more of configurable logic or fixed-functionality logic hardware, the logic coupled to the one or more substrates to]:
identify an assignment of portions of a weight tensor to respective ones of a plurality of processing elements, wherein the weight tensor is with an operation in a neural network (paragraph [0014]-[0015] In some embodiments, the method further includes: multiplying, in a first processing element circuit, the first row by a first vector of activations, to form a first dot product, multiplying, in a second processing element circuit, a second row of weights, of the first pre-processed weight tensor, by a second vector of activations, to form a second dot product, and adding the first product and the second product);
generate sparsity representations for the portions of the weight tensor, a sparsity representation of a portion of the weight tensor representing whether each of the weights in the portion of the weight tensor is a zero value or a non-zero value (paragraphs [0010] and [0022], the inter-tile preprocessing further includes creating a tile sparsity map corresponding to the first pre-processed weight tensor, the tile sparsity map having: a column for each weight tile of the first preprocessed weight tensor, and a row for each row of the weight tiles, the tile sparsity map indicating positions of empty rows of the weight tiles of the first pre-processed weight tensor; [0049] For example, referring to FIG. 2A, each array of the arrays of weights for four tiles may include some elements that are zero, or "empty" (illustrated as unshaded squares) and some elements that are nonzero (illustrated as shaded squares). The array of weights corresponding to a tile may be referred to as a "weight tile". In each of the first, second, and third weight tiles, an element labeled "lookahead" will, if processed one computation...; [0054] To perform inter-tile preprocessing, a tile sparsity map (having one fewer dimension than the weight tensor) may first be generated.).
However, Shin fails to explicitly teach:
one or more substrates;
logic coupled to the one or more substrates, wherein the logic is implemented at least partly in one or more of configurable logic or fixed-functionality logic hardware, the logic coupled to the one or more substrates.
Appu, in combination with Shin, teaches:
one or more substrates;
logic coupled to the one or more substrates, wherein the logic is implemented at least partly in one or more of configurable logic or fixed-functionality logic hardware, the logic coupled to the one or more substrates (paragraphs [0003]
As integrated circuit fabrication technology improves, manufacturers are able to integrate additional functionality onto a single silicon substrate; [0177] L3 cache, upstream control logic, and/or downstream control logic) may be provided in substrate layer(s) (e.g., between semiconductor packages), on an integrated circuit die).
It would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify the system Shin to add substrates to the system of Shin, as taught by Appu above. The modification would have been obvious because one of ordinary skill would be motivated to have an efficient power management, as suggested by Appu ([0003]).
However, Shin and Appu fail to explicitly teach:
store the sparsity representations and the portions of the weight tensor into partitions of a data storage structure based on the assignment of the portions of the weight tensor, wherein the partitions of the data storage structure each include a fixed number of bytes for a corresponding sparsity representation and another fixed number of bytes for a corresponding portion of the weight tensor.
Xiao, in combination with Shin and Appu, teaches:
store the sparsity representations and the portions of the weight tensor into partitions of a data storage structure based on the assignment of the portions of the weight tensor, wherein the partitions of the data storage structure (col. 5, lines 25-49 In some embodiments, each of the sparse filters may be segmented into a plurality of sub-filters that may be stored in a memory-efficient layout. For example, after being pruned and segmented, each of the sparse filters may be segmented in a way where each of the sub-filters comprises a same number of non-zero weights. Since the non-zero weights are evenly distributed into the plurality of sub filters, the convolution processing using these sub-filters may be referred to as a balanced-weight sparse convolution. In some embodiments, the non-zero weights within each sub-filter may be stored as index-value pairs, which may then be combined and stored as a fixed number of bits or bytes within the memory to represent the sub-filter).
It would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify the combination system of Shin and Appu to add a data storage with fixed number of bytes to the combination system of Shin and Appu, as taught by Xiao, above. The modification would have been obvious because one of ordinary skill would be motivated to assure the pairs of sub-tensor and sub-filter assigned to different PEs are independent from each other so that the plurality of PEs may execute the local operations in parallel to boost performance, as suggested by Xiao (col.5, lines 43-49).
As to claim 9, which incorporates the rejection of claim 7, Shin teaches:
remove one or more zero weights from weights assigned to a processing element of the plurality of processing elements to generate a compressed group of weights (paragraph [0049], skip performing multiplications when some of the elements of the weight tensor are zeros. For example, referring to FIG. 2A, each array of the arrays of weights for four tiles may include some elements that are zero, or "empty" (illustrated as unshaded squares) and some elements that are nonzero (illustrated as shaded squares). The array of weights corresponding to a tile may be referred to as a "weight tile". In each of the first, second, and third weight tiles, an element labeled "lookahead" will, if processed one computation cycle earlier, cause the row it currently occupies to become entirely empty (i.e., all of the elements of the row will be zero), which means that the processing could skip the row and proceed directly to the next row).
However, Shin fails to explicitly teach:
determine that a total number of weights in the compressed group of the weights is less than another fixed number;
insert one or more zero values into the compressed group of weights to generate a new group of weights; and
store the new group of weights in a partition of the data storage structure corresponding to the processing element.
Xiao, in combination with Shin, teaches wherein the instructions, when executed by the processor, further cause the computing system to:
determine that a total number of weights in the compressed group of the weights is less than the another fixed number (col. 5, lines 25-37, the non-zero weights within each sub-filter may be stored as index-value pairs, which may then be combined and stored as a fixed number of bits or bytes within the memory to represent the sub-filter; wherein Examiner interprets; wherein using the broadest reasonable interpretation, Examiner interprets “the non-zero weights within each sub-filter to be less than the combined and stored as a fixed number of bits or bytes);
insert one or more zero values into the compressed group of weights to generate a new group of weights (col.5, lines 1-24, the filters may be pruned or sparsified by introducing a large number of zeros to reduce the computation cost and improve inferencing speed. For example, more than 20% or 50% of the values in the filters
may be set to zeros…After each iteration of convolution process, the sub-tensor assigned to the PE as an input may be updated and evolved, output, which may be used as the input for the next iteration of convolution process); and
store the new group of weights in a partition of the data storage structure corresponding to the processing element (col. 5, lines 25-49 In some embodiments, each of the sparse filters may be segmented into a plurality of sub-filters that may be stored in a memory-efficient layout. For example, after being pruned and segmented, each of the sparse filters may be segmented in a way where each of the sub-filters comprises a same number of non-zero weights. Since the non-zero weights are evenly distributed into the plurality of sub filters, the convolution processing using these sub-filters may be referred to as a balanced-weight sparse convolution. In some embodiments, the non-zero weights within each sub-filter may be stored as index-value pairs, which may then be combined and stored as a fixed number of bits or bytes within the memory to represent the sub-filter).
It would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify the system Shin to add a data storage with fixed number of bytes to the system Shin, as taught by Xiao, above. The modification would have been obvious because one of ordinary skill would be motivated to assure the pairs of sub-tensor and sub-filter assigned to different PEs are independent from each other so that the plurality of PEs may execute the local operations in parallel to boost performance, as suggested by Xiao (col.5, lines 43-49).
As to claim 10, which incorporates the rejection of claim 7, Shin teaches:
wherein the operation further has an activation tensor, wherein the instructions, when executed by the processor, further cause the computing system to:
in response to determining that the activation corresponding to the next element is zero, bypass a next load process associated with the next element (paragraph [0049], system to skip performing multiplications when some of the elements of the weight tensor are zeros).
However, Shin fails to explicitly teach:
obtain a sparsity representation of the activation tensor, the sparsity representation comprising sequence of elements, each of which corresponds to a respective activation in the activation tensor and is associated with a separate load process; and
during a load process associated with an element in the sparsity representation,
determine whether an activation corresponding to the element is zero and whether an activation corresponding to a next element in the sparsity representation is zero; and
in response to determining that the activation corresponding to the next element is zero, bypass a next load process associated with the next element.
Xiao, in combination with Shin, teaches:
wherein the operation further has an activation tensor, wherein the instructions, when executed by the processor, further cause the computing system to:
obtain a sparsity representation of the activation tensor, the sparsity representation comprising sequence of elements, each of which corresponds to a respective activation in the activation tensor and is associated with a separate load process (col. 6, lines 56-67, in some cases, if there are multiple input tensors for a convolutional layer (e.g., when there are multiple images input into the convolution layer, or there are multiple input activation tensors received from a previous layer), each input tensor may be represented in an NHWC format, where N refers to an index of the input tensor within the batch of input tensors. In the following description, N may be omitted for simplicity (e.g., assuming there is only one input tensor) unless explicitly stated otherwise. It may be obvious for a person in the art to expand the embodiments to cover the cases with N>l); and
during a load process associated with an element in the sparsity representation,
determine whether an activation corresponding to the element is zero; and whether an activation corresponding to a next element in the sparsity representation is zero (col. 6, lines 56-67, in some cases, if there are multiple input tensors for a convolutional layer (e.g., when there are multiple images input into the convolution layer, or there are multiple input activation tensors received from a previous layer), each input tensor may be represented in an NHWC format, where N refers to an index of the input tensor within the batch of input tensors. In the following description, N may be omitted for simplicity (e.g., assuming there is only one input tensor) unless explicitly stated otherwise. It may be obvious for a person in the art to expand the embodiments to cover the cases with N>l).
It would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify the system Shin to add a data storage with fixed number of bytes to the system of Shin, as taught by Xiao, above. The modification would have been obvious because one of ordinary skill would be motivated to assure the pairs of sub-tensor and sub-filter assigned to different PEs are independent from
each other so that the plurality of PEs may execute the local operations in parallel to boost performance, as suggested by Xiao (col.5, lines 43-49).
As to claim 28, which incorporates the rejection of claim 10, Shin, teaches wherein the logic is further to:
in response to determining that the activation corresponding to the next element is not zero, perform a next load process associated with the next element (paragraph [0049], system to skip performing multiplications when some of the elements of the weight tensor are zeros).
Claims 11-12 are rejected under 35 U.S.C. 103 as being unpatentable over Shin et al. (US 2021/0357748 A1, hereinafter referred to as Shin), in view of Appu et al. (US 2018/0308272 A1, hereinafter referred to as Appu), and further in view of Xiao et al. (US 11,113,601B1, hereinafter referred to as Xiao), and XIAO et al. (US 2021/0240684 A1, hereinafter referred to as XIAO).
As to claim 11, which incorporates the rejection of claim 7, Shin, Appu and Xiao fail to explicitly teach wherein the sparsity representation is a bitmap.
However, XIAO, in combination with Shin, Appu and Xiao, teaches wherein the sparsity representation is a bitmap and each element of the sparsity representation is a bit. (paragraphs [0153] The operation unit of any of clauses 13-15, wherein the representation comprises a first sub-level bitmap and a second sub-level bitmap, and the sparse engine includes circuitry to: [0154] decompress the first sub-level bitmap to determine whether a sub-block of a part of the second level bitmap comprises a non-zero element; [0126] first sub-level bit map; [0173]-[0174]).
It would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify the combination system Shin, Appu and Xiao to add a bitmap to the combination system of Shin, Appu and Xiao, as taught by XIAO, above. The modification would have been obvious because one of ordinary skill would be motivated to determine whether a sub-block of a part of the level bitmap comprises a non-zero element, as suggested by XIAO ([0154]).
As to claim 12, which incorporates the rejection of claim 11, Shin fails to explicitly teach wherein:
a first partition of the partitions is associated with a first lane for a first processing element of the plurality of processing elements, wherein the first partition stores one or more weights assigned to a-the first processing element; and a second partition of the partitions is associated with a second second partition stores one or more weights assigned to the second processing element.
Appu, in combination with Shin, teaches wherein:
a first partition of the partitions is associated with a first lane for a first processing element of the plurality of processing elements, wherein the first partition stores one or more weights assigned to a-the first processing element ( (paragraphs [0049] The number of partition units 220A-220N generally equals the number of memory units, such that a first partition unit 220A has a corresponding first memory unit 224A; [0095] Application effective address space 482 within system memory 411 stores process elements 483. In one embodiment, the process elements 483 are stored in response to GPU invocations 481 from applications 480 executed on the processor 407; [0102], process element list 499 is stored [0197]); and
a second partition of the partitions is associated with a second (paragraphs [0049], a second partition unit 220B has a corresponding memory unit 224B, and an Nth partition unit 220N has a corresponding Nth memory unit 224N….; [0095] Application effective address space 482 within system memory 411 stores process elements 483. In one embodiment, the process elements 483 are stored in response to GPU invocations 481 from applications 480 executed on the processor 407; [0102], process element list 499 is stored; [0197]);
It would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to modify the system of Shin to add a bitmap to the system of Shin, as taught by Appu, above. The modification would have been obvious because one of ordinary skill would be motivated to allow for all or a subset of processes from all or a subset of partitions in the system to use a graphics acceleration module, as suggested by Appu ([0103]).
Response to Applicant’s arguments
Applicant's arguments on file on 02/26/2026 with respect to claims have been considered and are not persuasive.
Argument (page 12):
Applicant appears to assert that the cited references are silent regarding the sparsity representation in amended claim 1. The Office action and the Advisory Action refer to Xiao as teaching the sparsity representation.
Examiner’s response:
Examiner respectfully disagrees. In this Office action, Shin teaches a tile sparsity map that maintains a direct, fixed mapping to the underlying dense structure. Therefore, Shin teaches the sparsity representation in amended claim 1. Examiner response is based on paragraph [0018] of the original disclosure (specification).
Argument (page 12):
Applicant appears to assert that Xiao is silent regarding any "sparsity representation of a portion of the weight tensor having a same number of elements as the portion of the weight tensor, each element of the sparsity representation corresponding to a weight of the portion of the weight tensor and indicating whether the corresponding weight is a zero value or a non-zero value," as recited in amended claim 1.
The other references cannot remedy the deficiency of Xiao. Therefore, amended claim 1 is patentably distinguishable from the cited references. The other independent claims (i.e., claims 7, 14, and 20) are amended to recite similar limitations as amended claim 1 and therefore, are also patentable. The dependent claims are patentable at least due to their dependency on amended claims 1, 7, 14, and 20. Accordingly, withdrawal of the§ 103 rejections is respectfully requested.
Examiner’s response:
Examiner respectfully disagrees. In this non-final rejection above, Xiao teaches a balanced-weight sparse convolution process in neural networks. Xiao further teaches processing elements, input and output tensors, and segmenting the input tensor into a plurality of sub-tensors. However, Xiao fails to explicitly teach the amendment as recited in the independent claims 1, 7, 14 and 20.
Shin teaches a tile sparsity map that maintains a direct, fixed mapping to the underlying dense structure. Therefore, Shin, in [0010], [0022] and [0054] above, teaches the sparsity representation in amended claim 1 as well as other independent claims (i.e., claims 7, 14, and 20).
No further arguments were presented for the dependent claims. Therefore, the dependent claims are not patentable at least due to their dependency on amended claims 1, 7, 14, and 20. Accordingly, Examiner respectfully maintains the § 103 rejections.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ABABACAR SECK whose telephone number is (571)270-7146. The examiner can normally be reached Monday-Friday 8:00 A.M.-6:00 P.M..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Viker Lamardo can be reached on 571-270-5871. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ABABACAR SECK/Examiner, Art Unit 2147
/VIKER A LAMARDO/Supervisory Patent Examiner, Art Unit 2147
Prosecution Insights