Detailed Action
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This office action is final and is in response to claims filed on 03/17/2026 via Amendment. Claims 1-2, 4, 7-8, 10-12, 14, 17-18 and 20-21 are pending for examination. Claims 1-2, 7, 11-12, 14, 17-18, are currently amended. Claims 4, 8, 10, 20 are as previously filed. Claim 21 is newly presented.
Response to Arguments
Objection to the Claims
Applicant has amended the claims at issue, and, therefore, the previous objections have been withdrawn.
Rejections under 35 U.S.C. 112
Applicant has amended the claims at issue, and, therefore, the previous rejections have been withdrawn.
Rejections under 35 U.S.C. 101
Applicant’s arguments, see Remarks 6-10, filed 03/17/2026, with respect to the rejections under 35 U.S.C. 101 have been fully considered and are persuasive. The previous rejections have been withdrawn.
Specifically, Applicant’s arguments regarding the improvements to the neural network, see Remarks 8, integrate the judicial exceptions into a practical application.
Rejections under 35 U.S.C. 103
Applicant’s arguments with respect to claims 1-2, 4, 7-8, 10-12, 14, 17-18 and 20-21 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Objections
Claim 21 is objected to because of the following informalities:
Claim 21 recites “further comprising a pre-processing circuit configured to generate a complementary dense process tensor by combining the plurality of sparse process tensor”. This should be changed to “further comprising a pre-processing circuit configured to generate a complementary dense process tensor by combining the plurality of sparse processtensors”.
Appropriate correction is required.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1, 7-8, 10 and 11, 12, 14, 17-18 and 20-21 are provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 3, 4-5, 7, 11, and 15, 17, and 21-22 of U.S. Patent No. 17/856,494, (hereinafter “application ‘494”) in view of Karasawa et al. (Machine Translation of JP 2020009209 A) hereinafter Karasawa. Although the claims at issue are not identical, they are not patentably distinct from each other.
Current Application
Application ‘494
Claim 1
Claims 11 and 15
An accelerator for performing operations on tensors, the accelerator comprising: a memory configured to store a complementary dense process tensor,
Claim 11: A computing device, comprising: memory configured to store a model; and a processor coupled to the memory
the complementary dense process tensor generated from combining a plurality of sparse process tensors that have non-overlapping locations of active values;
Claim 11: the processor configured to: combine a plurality of sparse process tensors of the model to a complementary process tensor by overlaying the plurality of sparse process tensors, the plurality of sparse process tensors corresponding to a plurality of nodes of a sparse neural network and having non- overlapping locations of active values;
each of values in the complementary dense process tensor associated with a tensor identifier that identifies a corresponding one of the plurality of sparse process tensors from which the value originated
and a computation core coupled to the memory,
Claim 11: a processor coupled to the memory
the computation core configured to perform computations between two or more tensors to generate a product tensor including computed values,
Claim 11: perform computations between the complementary process tensor and an activation tensor to generate a plurality of products;
the two or more tensors including the complementary dense process tensor,
Claim 11: perform computations between the complementary process tensor and an activation tensor to generate a plurality of products;
and the computation core comprises: a permutation circuit configured to re-arrange the computed values in the product tensor to group the values corresponding to one of the sparse process tensors together based on tensor identifiers.
Claim 11: and separate the plurality of products into groups, each group corresponding to one of the sparse process tensors.
Claim 15: wherein separating the plurality of products into groups comprises a post-multiplication re-arrangement of the plurality of products.
Claim 7
Claim 11 and Claim 22
The accelerator of claim 1, wherein the active values in the plurality of sparse process tensors are partitioned,
Claim 11: separate the plurality of products into groups, each group corresponding to one of the sparse process tensors
and the permutation circuit comprises multiple permutation networks,
Claim 22: wherein the active values in the plurality of sparse process tensors are partitioned and the permutation circuit comprises a plurality of permutation networks,
each of the premutation networks is configured to re-arrange the computed values corresponding a partition.
Claim 22: each permutation network configured to re-arrange values corresponding to a respective partition.
Claim 8
Claim 21
The accelerator of claim 1, wherein the permutation circuit comprises a network of switches.
Claim 21: wherein separating the plurality of products into groups comprises re-arranging the plurality of products using a permutation circuit comprising a network of switches
Claim 10
Claim 17
The accelerator of claim 1, further comprising: an activation circuit configured to select a subset of outputs of the computation core as values in an output activation tensor.
select a subset of the plurality of accumulated values as winners of an activation selection
Claim 11
Claim 1, Claim 4, and Claim 5
A method comprising: generating a complementary dense process tensors by combining a plurality of sparse process tensors that have non-overlapping locations of active values,
Claim 1: A computer-implemented method for operating on tensors, the computer-implemented method comprising: combining a plurality of sparse process tensors to a complementary process tensor by overlaying the plurality of sparse process tensors, the plurality of sparse process tensors corresponding to a plurality of nodes of a sparse neural network and having non-overlapping locations of active values;
each of values in the complementary dense process tensor associated with a tensor identifier that identifies a corresponding one of the plurality of sparse process tensors from which the value originated;
performing computations between two or more tensors to generate a product tensor including computed values,
Claim 1: performing computations between the complementary process tensor and an activation tensor to generate a plurality of products;
the two or more tensors including the complementary dense process tensor;
Claim 1: performing computations between the complementary process tensor and an activation tensor to generate a plurality of products;
and re-arranging the computed values in the product tensor to group the values corresponding to one of the sparse process tensors together based on tensor identifiers.
Claim 1: and separating the plurality of products into groups, each group corresponding to one of the sparse process tensors.
Claim 5: wherein separating the plurality of products into groups comprises a post-multiplication re-arrangement of the plurality of products.
Claim 12
Claim 3 and Claim 1
The method of claim 11, wherein performing the computations between two or more tensors comprises: performing multiplications between two or more tensors;
Claim 3: wherein performing the computations between the complementary process tensor and the activation tensor comprises: performing elementwise multiplications between values in the complementary process tensor and values in the activation tensor.
and accumulating the computed values corresponding to the one of the sparse process tensors.
Claim 1: and accumulating the groups of products to generate a plurality of accumulated values, each accumulated value corresponding to one of the sparse process tensors
Claim 14
Claims 5
The method of claim 12, wherein re-arranging the values in the product tensor is performed after performing multiplications between two or more tensors.
Claim 5: wherein separating the plurality of products into groups comprises a post-multiplication re-arrangement of the plurality of products.
Claim 17
Claim 1 and Claim 4
The method of claim 11, wherein the active values in the plurality of sparse process tensors are partitioned,
Claim 1: separating the plurality of products into groups, each group corresponding to one of the sparse process tensors
and re-arranging the computed values comprises re-arranging the values corresponding to a partition.
Claim 4: wherein separating the plurality of products into groups comprises a pre-multiplication re-arrangement of the activation tensor.
Claim 18
Claim 21
The method of claim 11, wherein re-arranging the computed values is performed by a permutation circuit that comprises a network of switches.
Claim 21: wherein separating the plurality of products into groups comprises re-arranging the plurality of products using a permutation circuit comprising a network of switches
Claim 20
Claim 7
The method of claim 11, further comprising: selecting a subset of outputs as values in an output activation tensor.
selecting a subset of the plurality of accumulated values as winners of an activation selection; and setting the remaining values of the plurality of accumulated values to zero.
Claim 21
Claim 11
The accelerator of claim 1, further comprising a pre-processing circuit configured to generate a complementary dense process tensor by combining the plurality of sparse process tensor.
the processor configured to: combine a plurality of sparse process tensors of the model to a complementary process tensor
As per claim 1, application ‘494 teaches the language as shown in the table above. However, application ‘494 does not teach However, application ‘494 does not teach each of values in the complementary dense process tensor associated with a tensor identifier that identifies a corresponding one of the plurality of sparse process tensors from which the value originated; and grouping the values corresponding to the one of the sparse process tensors based on the tensor identifier. Karasawa teaches each of values in the complementary dense process tensor associated with a tensor identifier that identifies a corresponding one of the plurality of sparse process tensors from which the value originated; (Karasawa page 14 Paragraph 3: The control unit ST110 acquires the row category and colum category identifier of each DCBblock of the Combine DCBblock, the identifier of each DCBblock, the arrangement data using the identifier, and the data of each DCBblock from the storage unit or the operation unit, and registers it in the storage unit)
grouping the values corresponding to the one of the sparse process tensors based on the tensor identifier(Karasawa page 14 Paragraph 3: The control unit ST110 acquires the row category and colum category identifier of each DCBblock of the Combine DCBblock, the identifier of each DCBblock, the arrangement data using the identifier, and the data of each DCBblock from the storage unit or the operation unit, and registers it in the storage unit).
Therefore, it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to combine the teachings of application ‘494 with the tensor identifier as taught by Karasawa. One of ordinary skill in the art would be motivated to make this combination because it would increase the efficiency of the system as, if a portion of the result corresponding to one of the sparse tensors is needed, they will be close together. Also, The advantage of DCBmatrix [1,1], which is an ExtDCBblock, being an orthogonal structure is that it suppresses the explosion of the combination of design specification factors, effectively reduces the effect of design specification factors, and interacts with factors such as Simpson's paradox. This is to avoid bias and to obtain a highly robust model as taught by Karasawa (Karasawa Page 9 Paragraph 11).
As per claim 11, application ‘494 teaches the language as shown in the table above. However, application ‘494 does not teach However, application ‘494 does not teach each of values in the complementary dense process tensor associated with a tensor identifier that identifies a corresponding one of the plurality of sparse process tensors from which the value originated; and grouping the values corresponding to the one of the sparse process tensors based on the tensor identifier. Karasawa teaches each of values in the complementary dense process tensor associated with a tensor identifier that identifies a corresponding one of the plurality of sparse process tensors from which the value originated; (Karasawa page 14 Paragraph 3: The control unit ST110 acquires the row category and colum category identifier of each DCBblock of the Combine DCBblock, the identifier of each DCBblock, the arrangement data using the identifier, and the data of each DCBblock from the storage unit or the operation unit, and registers it in the storage unit)
grouping the values corresponding to the one of the sparse process tensors based on the tensor identifier(Karasawa page 14 Paragraph 3: The control unit ST110 acquires the row category and colum category identifier of each DCBblock of the Combine DCBblock, the identifier of each DCBblock, the arrangement data using the identifier, and the data of each DCBblock from the storage unit or the operation unit, and registers it in the storage unit).
Therefore, it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to combine the teachings of application ‘494 with the tensor identifier as taught by Karasawa. One of ordinary skill in the art would be motivated to make this combination because it would increase the efficiency of the system as, if a portion of the result corresponding to one of the sparse tensors is needed, they will be close together. Also, The advantage of DCBmatrix [1,1], which is an ExtDCBblock, being an orthogonal structure is that it suppresses the explosion of the combination of design specification factors, effectively reduces the effect of design specification factors, and interacts with factors such as Simpson's paradox. This is to avoid bias and to obtain a highly robust model as taught by Karasawa (Karasawa Page 9 Paragraph 11).
Claim 2 is provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over claims 11 and 13 of application ‘494 in view of Karasawa further in view of Hwang et al. (US 20220108159 A1) hereinafter Hwang.
Claim 2
Claim 11 and Claim 13
The accelerator of claim 1, wherein the computation core comprises: a multiply circuit configured to perform multiplications between two or more tensors;
Claim 11: the processor configured to… the processor configured to: combine a plurality of sparse process tensors of the model to a complementary process tensor by overlaying the plurality of sparse process tensors, the plurality of sparse process tensors corresponding to a plurality of nodes of a sparse neural network and having non-overlapping locations of active values; perform computations between the complementary process tensor and an activation tensor to generate a plurality of products
Claim 13: wherein performing the computations between the complementary process tensor and the activation tensor comprises: perform elementwise multiplications between values in the complementary process tensor and values in the activation tensor.
and an adder tree configured to accumulate the computed values corresponding to the one of the sparse process tensors.
Claim 11: accumulate the groups of products to generate a plurality of accumulated values, each accumulated value corresponding to one of the sparse process tensors
As per claim 2, application ‘494 teaches the language as shown in the table above. However, application ‘494 does not teach an adder tree. Hwang teaches and adder tree (Hwang [0086]: the results of the ADC 120a may be summed up through the adder tree).
Therefore, it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to combine the teachings of application ‘494 with the adder tree as taught by Hwang. One of ordinary skill in the art would be motivated to make this combination because it would increase the efficiency of the system as the values could be accumulated in parallel, speeding up calculations.
Claim 4 is provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over claims 11 and 14-15 of application ‘494 in view of Karasawa further in view of Wang et al. (WO 2020190772 A1) hereinafter Wang.
Claim 4
Claim 11, Claim 14, and Claim 15
The accelerator of claim 2, wherein the permutation circuit is located downstream of the multiply circuit.
Claim 11: separate the plurality of products into groups, each group corresponding to one of the sparse process tensors
Claim 14: wherein separating the plurality of products into groups comprises a pre-multiplication re-arrangement of the activation tensor.
Claim 15: wherein separating the plurality of products into groups comprises a post-multiplication re-arrangement of the plurality of products.
As per claim 4, application ‘494 teaches the language as shown in the table above. However, application ‘494 does not teach that the permutation circuit is located downstream of the multiply circuit. Wang teaches that the permutation circuit is located downstream of the multiply circuit (Wang [0010]: The reordering of the output feature map includes swapping columns of the output feature map; Wang Fig. 13A and 13B: shows that the reordering circuit is downstream of the matrix multiplier).
Therefore, it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to combine the teachings of application ‘494 with the location of the permutation circuit as taught by Wang. One of ordinary skill in the art would be motivated to make this combination because it would increase the efficiency of the system as the inference speed can be improved if the matrix sparsity is structured in the same way as taught by Wang (Wang [0086]).
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 17 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 17 recites “and re-arranging the computed values comprises re-arranging the values corresponding to a partition”. It is unclear if the values in “re-arranging the values corresponding to a partition” are the computed values or the active values. For examination purposes, examiner has interpreted “and re-arranging the computed values comprises re-arranging the values corresponding to a partition” as “and re-arranging the computed values comprises re-arranging the computed values corresponding to a partition”
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 11-12, 14, 17, and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Mei et al. (US 20220309124 A1) hereinafter Mei in view of Wang et al. (WO 2020190772 A1) hereinafter Wang further in view of Karasawa et al. (Machine Translation of JP 2020009209 A) hereinafter Karasawa.
With regards to claim 1, Mei teaches An accelerator for performing operations on tensors, the accelerator comprising: a memory configured to store a complementary dense process tensor, (Mei [0231]: As shown in FIG. 17B, in one embodiment the sparse matrix multiply accelerator 1700 includes or couples with memory 1720 that can store matrix elements)
the complementary dense process tensor generated from combining a [plurality] of sparse process tensors that have non-overlapping locations of active values, (Mei [0231]: An element merge unit 1742 can merge elements in a first set of column vectors into a second set of column vectors; Mei [0234]: To utilize sparsity in Matrix A, two input tiles 1822 are loaded and merged together to remove zeros in the input; Mei Fig. 17C: shows the non-overlapping locations of active values)
[each of values in the complementary dense process tensor associated with a tensor identifier that identifies a corresponding one of the plurality of] sparse process tensors [from which the value originated] (Mei [0234]: To utilize sparsity in Matrix A, two input tiles 1822 are loaded and merged together to remove zeros in the input; Mei Fig. 17C: shows the non-overlapping locations of active values)
and a computation core coupled to the memory, (Mei [0231]: As shown in FIG. 17B, in one embodiment the sparse matrix multiply accelerator 1700 includes or couples with memory 1720 that can store matrix elements)
the computation core configured to perform computations between two or more tensors to generate a product tensor including computed values, (Mei [0234]: A multiply and add operation is performed and a partial sum is stored to storage. To utilize sparsity in Matrix A, two input tiles 1822 are loaded and merged together to remove zeros in the input)
the two or more tensors including the complementary dense process tensor, (Mei [0234]: A multiply and add operation is performed and a partial sum is stored to storage. To utilize sparsity in Matrix A, two input tiles 1822 are loaded and merged together to remove zeros in the input)
and the computation core comprises: a permutation circuit configured [to re-arrange the computed values in the product tensor to group the computed values corresponding to one of the sparse process tensors together based on tensor identifiers] (Mei [0231]: The metadata 1732 can be used by a load unit 1737 to determine how to order vector elements of a Matrix B input vector, which is fed to the functional units via an additional feed unit 1738).
Mei fails to teach [combining a] plurality [of sparse process tensors] and each of values in the complementary dense process tensor associated with a tensor identifier that identifies a corresponding one of the plurality of [sparse process tensors] from which the value originated.
However, Wang teaches [combining a] plurality [of sparse process tensors] (Wang [0046]: The trained CNN includes multiple feature maps)
[each of values in the complementary dense process tensor associated with a tensor identifier that identifies a corresponding one of] the plurality of [sparse process tensors from which the value originated] (Wang [0046]: The trained CNN includes multiple feature maps)
to re-arrange the computed values in the product tensor to group the computed values [corresponding to one of the sparse process tensors together based on tensor identifiers] (Wang [0010]: The reordering of the output feature map includes swapping columns of the output feature map).
Therefore, it would have been obvious before the effective filing date of the claimed invention
for one of ordinary skill in the art to combine the teachings of Mei with the plurality of tensors and re-arranging the output as taught by Wang. One of ordinary skill in the art would be motivated to make this combination because it would increase the efficiency of the system as the inference speed can be improved if the matrix sparsity is structured in the same way as taught by Wang (Wang [0086]).
Mei in view of Wang fails to teach each of values in the complementary dense process tensor associated with a tensor identifier that identifies a corresponding one of [the plurality of sparse process tensors] from which the value originated [and the computation core comprises: a permutation circuit configured] [to re-arrange the computed values in the product tensor to group the computed values] corresponding to one of the sparse process tensors together based on tensor identifiers.
However, Karasawa teaches each of values in the complementary dense process tensor associated with a tensor identifier that identifies a corresponding one of [the plurality of sparse process tensors] from which the value originated (Karasawa page 14 Paragraph 3: The control unit ST110 acquires the row category and colum category identifier of each DCBblock of the Combine DCBblock, the identifier of each DCBblock, the arrangement data using the identifier, and the data of each DCBblock from the storage unit or the operation unit, and registers it in the storage unit)
[and the computation core comprises: a permutation circuit configured] to re-arrange the computed values in the product tensor to group the computed values corresponding to one of the sparse process tensors together based on tensor identifiers (Karasawa page 14 Paragraph 3: The control unit ST110 acquires the row category and colum category identifier of each DCBblock of the Combine DCBblock, the identifier of each DCBblock, the arrangement data using the identifier, and the data of each DCBblock from the storage unit or the operation unit, and registers it in the storage unit).
Therefore, it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to combine the teachings of Mei in view of Wang with the tensor identifier as taught by Karasawa. One of ordinary skill in the art would be motivated to make this combination because it would increase the efficiency of the system as, if a portion of the result corresponding to one of the sparse tensors is needed, they will be close together. Also, The advantage of DCBmatrix [1,1], which is an ExtDCBblock, being an orthogonal structure is that it suppresses the explosion of the combination of design specification factors, effectively reduces the effect of design specification factors, and interacts with factors such as Simpson's paradox. This is to avoid bias and to obtain a highly robust model as taught by Karasawa (Karasawa Page 9 Paragraph 11).
With regards to claim 11, Mei teaches A method comprising: generating a complementary dense process tensor, by combining a [plurality] of sparse process tensors that have non-overlapping locations of active values; (Mei [0231]: An element merge unit 1742 can merge elements in a first set of column vectors into a second set of column vectors; Mei [0234]: To utilize sparsity in Matrix A, two input tiles 1822 are loaded and merged together to remove zeros in the input; Mei Fig. 17C: shows the non-overlapping locations of active values)
performing computations between two or more tensors to generate a product tensor including computed values, (Mei [0234]: A multiply and add operation is performed and a partial sum is stored to storage. To utilize sparsity in Matrix A, two input tiles 1822 are loaded and merged together to remove zeros in the input)
the two or more tensors including the complementary dense process tensor; (Mei [0234]: A multiply and add operation is performed and a partial sum is stored to storage. To utilize sparsity in Matrix A, two input tiles 1822 are loaded and merged together to remove zeros in the input)
Mei fails to teach [combining a] plurality [of sparse process tensors], each of values in the complementary dense process tensor associated with a tensor identifier that identifies a corresponding one of the plurality of [sparse process tensors] from which the value originated and to re-arrange the computed values in the product tensor to group the computed values [corresponding to one of the sparse process tensors together based on tensor identifiers] (Wang [0010]: The reordering of the output feature map includes swapping columns of the output feature map)..
However, Wang teaches [combining a] plurality [of sparse process tensors] (Wang [0046]: The trained CNN includes multiple feature maps)
[each of values in the complementary dense process tensor associated with a tensor identifier that identifies a corresponding one of] the plurality of [sparse process tensors from which the value originated] (Wang [0046]: The trained CNN includes multiple feature maps)
and re-arranging the computed values in the product tensor [to group the values corresponding to one of the sparse process tensors together based on tensor identifiers] (Wang [0010]: The reordering of the output feature map includes swapping columns of the output feature map).
Therefore, it would have been obvious before the effective filing date of the claimed invention
for one of ordinary skill in the art to combine the teachings of Mei with the plurality of tensors and re-arranging the output as taught by Wang. One of ordinary skill in the art would be motivated to make this combination because it would increase the efficiency of the system as the inference speed can be improved if the matrix sparsity is structured in the same way as taught by Wang (Wang [0086]).
Mei in view of Wang fails to teach each of values in the complementary dense process tensor associated with a tensor identifier that identifies a corresponding one of [the plurality of sparse process tensors] from which the value originated [and re-arranging the computed values in the product tensor] to group the values corresponding to one of the sparse process tensors together based on tensor identifiers.
However, Karasawa teaches each of values in the complementary dense process tensor associated with a tensor identifier that identifies a corresponding one of [the plurality of sparse process tensors] from which the value originated (Karasawa page 14 Paragraph 3: The control unit ST110 acquires the row category and colum category identifier of each DCBblock of the Combine DCBblock, the identifier of each DCBblock, the arrangement data using the identifier, and the data of each DCBblock from the storage unit or the operation unit, and registers it in the storage unit)
[and re-arranging the computed values in the product tensor] to group the values corresponding to one of the sparse process tensors together based on tensor identifiers (Karasawa page 14 Paragraph 3: The control unit ST110 acquires the row category and colum category identifier of each DCBblock of the Combine DCBblock, the identifier of each DCBblock, the arrangement data using the identifier, and the data of each DCBblock from the storage unit or the operation unit, and registers it in the storage unit).
Therefore, it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to combine the teachings of Mei in view of Wang with the tensor identifier as taught by Karasawa. One of ordinary skill in the art would be motivated to make this combination because it would increase the efficiency of the system as, if a portion of the result corresponding to one of the sparse tensors is needed, they will be close together. Also, The advantage of DCBmatrix [1,1], which is an ExtDCBblock, being an orthogonal structure is that it suppresses the explosion of the combination of design specification factors, effectively reduces the effect of design specification factors, and interacts with factors such as Simpson's paradox. This is to avoid bias and to obtain a highly robust model as taught by Karasawa (Karasawa Page 9 Paragraph 11).
With regards to claim 12, Mei in view of Wang further in view of Karasawa teaches all of the limitations of claim 11 above. Mei further teaches wherein performing the computations between two or more tensors comprises: performing multiplications between two or more tensors; (Mei [0234]: FIG. 18 illustrates operations 1800 for a matrix multiply in which random sparsity is handled via element merges. A matrix multiply operation M×K×N generally accumulates results at K dimension through multiple iterations on inner-product systolic array)
and accumulating the computed values corresponding to the one of the sparse process tensors (Mei [0234]: FIG. 18 illustrates operations 1800 for a matrix multiply in which random sparsity is handled via element merges. A matrix multiply operation M×K×N generally accumulates results at K dimension through multiple iterations on inner-product systolic array).
With regards to claim 14, Mei in view of Wang further in view of Karasawa teaches all of the limitations of claim 12 above. Mei further teaches wherein re-arranging the values in the product tensor is performed [after performing multiplications between two or more tensors] (Mei [0234]: A multiply and add operation is performed and a partial sum is stored to storage. To utilize sparsity in Matrix A, two input tiles 1822 are loaded and merged together to remove zeros in the input).
Mei fails to teach [wherein re-arranging the values in one of the two or more tensors or in the product tensor is performed] after performing multiplications between two or more tensors.
However, Wang teaches [wherein re-arranging the values in one of the two or more tensors or in the product tensor is performed] after performing multiplications between two or more tensors (Wang [0010]: The reordering of the output feature map includes swapping columns of the output feature map).
Therefore, it would have been obvious before the effective filing date of the claimed invention
for one of ordinary skill in the art to combine the teachings of Mei in view of Wang further in view of Karasawa with re-arranging after the multiplication as taught by Wang. One of ordinary skill in the art would be motivated to make this combination because it would increase the efficiency of the system as the inference speed can be improved if the matrix sparsity is structured in the same way as taught by Wang (Wang [0086]).
With regards to claim 17, Mei in view of Wang further in view of Karasawa teaches all of the limitations of claim 11 above. Mei further teaches wherein the active values in the plurality of sparse process tensors are partitioned, (Mei [0232]: As shown in FIG. 17C, a first group of vectors from a sparse input matrix can be a first element group 1746 and a second group of vectors from the spare input matrix can be a second element group 1747)
and re-arranging the [computed] values comprises re-arranging the [computed] values correspond a partition (Mei [0232]: As shown in FIG. 17C, a first group of vectors from a sparse input matrix can be a first element group 1746 and a second group of vectors from the spare input matrix can be a second element group 1747. The element merge unit 1742 can perform an operation 1751 to merge non-zero elements from the first element group 1746 into the second element group 1747).
Mei fails to teach rearranging the computed values.
However, Wang teaches rearranging the computed values (Wang [0010]: The reordering of the output feature map includes swapping columns of the output feature map).
Therefore, it would have been obvious before the effective filing date of the claimed invention
for one of ordinary skill in the art to combine the teachings of Mei in view of Wang further in view of Karasawa with re-arranging after the multiplication as taught by Wang. One of ordinary skill in the art would be motivated to make this combination because it would increase the efficiency of the system as the inference speed can be improved if the matrix sparsity is structured in the same way as taught by Wang (Wang [0086]).
With regards to claim 21, Mei in view of Wang further in view of Karasawa teaches all of the limitations of claim 1 above. Mei further teaches further comprising a pre-processing circuit configured to generate a complementary dense process tensor by combining the [plurality] of sparse process tensor (Mei [0231]: An element merge unit 1742 can merge elements in a first set of column vectors into a second set of column vectors; Mei [0234]: To utilize sparsity in Matrix A, two input tiles 1822 are loaded and merged together to remove zeros in the input; Mei Fig. 17C: shows the non-overlapping locations of active values).
Mei fails to teach the plurality of sparse process tensor.
However, Wang teaches the plurality of sparse process tensor (Wang [0046]: The trained CNN includes multiple feature maps).
Therefore, it would have been obvious before the effective filing date of the claimed invention
for one of ordinary skill in the art to combine the teachings of Mei in view of Wang further in view of Karasawa with the plurality of tensors as taught by Wang. One of ordinary skill in the art would be motivated to make this combination because it would increase the efficiency of the system as the inference speed can be improved if the matrix sparsity is structured in the same way as taught by Wang (Wang [0086]).
Claims 2 and 4 are rejected under 35 U.S.C. 103 as being unpatentable over Mei in view of Wang further in view of Karasawa further in view of Hwang et al. (US 20220108159 A1) hereinafter Hwang.
With regards to claim 2, Mei in view of Wang further in view of Karasawa teaches all of the limitations of claim 1 above. Mei further teaches wherein the computation core comprises: a multiply circuit configured to perform multiplications between two or more tensors; (Mei [0223]: As shown in FIG. 17A, the sparse matrix multiply accelerator 1700 includes processing elements 1702A-1702D that include arrays of multipliers)
and an [adder tree] configured to accumulate the computed values corresponding to the one of the sparse process tensors (Mei [0223]: As shown in FIG. 17A, the sparse matrix multiply accelerator 1700 includes processing elements 1702A-1702D that include arrays of multipliers and adders; Mei [0234]: FIG. 18 illustrates operations 1800 for a matrix multiply in which random sparsity is handled via element merges. A matrix multiply operation M×K×N generally accumulates results at K dimension through multiple iterations on inner-product systolic array).
Mei fails to teach of the adder tree.
However, Hwang teaches of the adder tree (Hwang [0086]: the results of the ADC 120a may be summed up through the adder tree)
Therefore, it would have been obvious before the effective filing date of the claimed invention
for one of ordinary skill in the art to combine the teachings of Mei in view of Wang further in view of Karasawa with the adder tree as taught by Hwang. One of ordinary skill in the art would be motivated to make this combination because it would increase the efficiency of the system as the values could be accumulated in parallel speeding up calculations.
With regards to claim 4, Mei in view of Wang further in view of Karasawa further in view of Hwang teaches all of the limitations of claim 2 above. Mei further teaches wherein the permutation circuit is located [downstream] of the multiply circuit (Mei [0231]: The metadata 1732 can be used by a load unit 1737 to determine how to order vector elements of a Matrix B input vector, which is fed to the functional units via an additional feed unit 1738).
Mei fails to teach [wherein the permutation circuit is located] downstream [of the multiply circuit].
However, Wang teaches [wherein the permutation circuit is located] downstream [of the multiply circuit] (Wang [0010]: The reordering of the output feature map includes swapping columns of the output feature map; Wang Fig. 13A and 13B: shows that the reordering circuit is downstream of the matrix multiplier).
Therefore, it would have been obvious before the effective filing date of the claimed invention
for one of ordinary skill in the art to combine the teachings of Mei in view of Wang further in view of Karasawa further in view of Hwang with the permutation circuit being downstream from the multiply circuit as taught by Wang. One of ordinary skill in the art would be motivated to make this combination because it would increase the efficiency of the system as the inference speed can be improved if the matrix sparsity is structured in the same way as taught by Wang (Wang [0086]).
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Mei in view of Wang further in view of Karasawa further in view of Bondarenko et al. (US 12412081 B2) hereinafter Bondarenko.
With regards to claim 7, Mei in view of Wang further in view of Karasawa teaches all of the limitations of claim 1 above. Mei further teaches wherein the active values in the plurality of sparse process tensors are partitioned, (Mei [0232]: As shown in FIG. 17C, a first group of vectors from a sparse input matrix can be a first element group 1746 and a second group of vectors from the spare input matrix can be a second element group 1747)
and the permutation circuit comprises [multiple permutation networks,] (Mei [0231]: The metadata 1732 can be used by a load unit 1737 to determine how to order vector elements of a Matrix B input vector, which is fed to the functional units via an additional feed unit 1738).
Mei fails to teach [and the permutation circuit comprises] multiple permutation networks, each of the premutation networks is configured to re-arrange the computed values correspond a partition.
However, Bondarenko teaches [and the permutation circuit comprises] multiple permutation networks, (Bondarenko Column 4 Lines 45-49: The one or more programmable engines may be a plurality of programmable engines, wherein the method comprises two or more of the programmable engines permuting the pair of dimensions of the multi-dimensional tensor in parallel)
each of the premutation network is configured to re-arrange the computed values correspond a partition (Bondarenko Column 4 Lines 45-49: The one or more programmable engines may be a plurality of programmable engines, wherein the method comprises two or more of the programmable engines permuting the pair of dimensions of the multi-dimensional tensor in parallel).
Therefore, it would have been obvious before the effective filing date of the claimed invention
for one of ordinary skill in the art to combine the teachings of Mei in view of Wang further in view of Karasawa with the plurality of permutation networks as taught by Bondarenko. One of ordinary skill in the art would be motivated to make this combination because would increase the efficiency of the system as permuting the dimensions of a data set may be a pre-processing step for efficient matrix multiplication algorithms because the permutation may provide improved cache access patterns as taught by Bondarenko (Bondarenko Column 1 Lines 35-38).
Claims 8 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Mei in view of Wang further in view of Tu et al. (US 20170024346 A1) hereinafter Tu.
With regards to claim 8, Mei in view of Wang further in view of Karasawa teaches all of the limitations of claim 1 above. Mei further teaches wherein the permutation circuit comprises [a network of switches] (Mei [0231]: The metadata 1732 can be used by a load unit 1737 to determine how to order vector elements of a Matrix B input vector, which is fed to the functional units via an additional feed unit 1738).
Mei fails to teach [wherein the permutation circuit comprises] a network of switches.
However, Tu teaches [wherein the permutation circuit comprises] a network of switches (Tu [0003]: Benes network is a rearrangeable nonblocking network, which can realize any arbitrary permutation between N input ports and N port groups of the on-chip crossbar; Tu Fig. 2: shows a Benes Network with many switches).
Therefore, it would have been obvious before the effective filing date of the claimed invention
for one of ordinary skill in the art to combine the teachings of Mei in view of Wang further in view of Karasawa with the network of switches as taught by Tu. One of ordinary skill in the art would be motivated to make this combination because it would increase the efficiency of the system because the Benes network allows for high-speed parallel packet routing implementation with support of partial permutations between the input ports and the port groups as taught by Tu (Tu [0003]).
With regards to claim 18, Mei in view of Wang further in view of Karasawa teaches all of the limitations of claim 11 above. Mei further teaches Wherein re-arranging the [computed] values is performed by a permutation circuit that comprises [a network of switches] (Mei [0231: The metadata 1732 can be used by a load unit 1737 to determine how to order vector elements of a Matrix B input vector, which is fed to the functional units via an additional feed unit 1738).
Mei fails to teach re-arranging the computed values.
However, Wang teach re-arranging the computed values (Wang [0010]: The reordering of the output feature map includes swapping columns of the output feature map).
Therefore, it would have been obvious before the effective filing date of the claimed invention
for one of ordinary skill in the art to combine the teachings of Mei in view of Wang further in view of Karasawa with re-arranging the output as taught by Wang. One of ordinary skill in the art would be motivated to make this combination because it would increase the efficiency of the system as the inference speed can be improved if the matrix sparsity is structured in the same way as taught by Wang (Wang [0086]).
Mei in view of Wang fails to teach [Wherein re-arranging the computed values is performed by a permutation circuit that comprises] a network of switches.
However, Tu teaches [Wherein re-arranging the computed values is performed by a permutation circuit that comprises] a network of switches (Tu [0003]: Benes network is a rearrangeable nonblocking network, which can realize any arbitrary permutation between N input ports and N port groups of the on-chip crossbar; Tu Fig. 2: shows a Benes Network with many switches).
Therefore, it would have been obvious before the effective filing date of the claimed invention
for one of ordinary skill in the art to combine the teachings of Mei in view of Wang further in view of Karasawa with the network of switches as taught by Tu. One of ordinary skill in the art would be motivated to make this combination because it would increase the efficiency of the system because the Benes network allows for high-speed parallel packet routing implementation with support of partial permutations between the input ports and the port groups as taught by Tu (Tu [0003]).
Claims 10 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Mei in view of Wang further in view of Karasawa further in view of Ahmad et al (US 20210158168 A1) hereinafter Ahmad.
With regards to claim 10, Mei in view of Wang further in view of Karasawa teaches all of the limitations of claim 1 above. Mei fails to teach further comprising: an activation circuit configured to select a subset of outputs of the computation core as values in an output activation tensor.
However, Ahmad teaches further comprising: an activation circuit configured to select a subset of outputs of the computation core as values in an output activation tensor (Ahmad [0072]: The convolutional layer may generate sparse layer outputs in the form of a sparse tensor by selecting a subset of nodes in the intermediate tensor).
Therefore, it would have been obvious before the effective filing date of the claimed invention
for one of ordinary skill in the art to combine the teachings of Mei in view of Wang further in view of Karasawa with selecting a subset of outputs as taught by Ahmad. One of ordinary skill in the art would be motivated to make this combination because it would improve the flexibility of the system as the system could choose which outputs to include in the output tensor.
With regards to claim 20, Mei in view of Wang further in view of Karasawa teaches all of the limitations of claim 11 above. Mei fails to teach further comprising: selecting a subset of outputs as values in an output activation tensor.
However, Ahmad teaches further comprising: selecting a subset of outputs as values in an output activation tensor (Ahmad [0072]: The convolutional layer may generate sparse layer outputs in the form of a sparse tensor by selecting a subset of nodes in the intermediate tensor).
Therefore, it would have been obvious before the effective filing date of the claimed invention
for one of ordinary skill in the art to combine the teachings of Mei in view of Wang further in view of Karasawa with selecting a subset of outputs as taught by Ahmad. One of ordinary skill in the art would be motivated to make this combination because it would improve the flexibility of the system as the system could choose which outputs to include in the output tensor.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Jakob O Gudas whose telephone number is (571)272-0695. The examiner can normally be reached Monday-Thursday: 7:30AM-5:00PM Friday: 7:30AM-4:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, James Trujillo can be reached at (571) 272-3677. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.O.G./Examiner, Art Unit 2151
/NICHOLAS KLICOS/Primary Examiner, Art Unit 2118
Prosecution Insights