DETAILED ACTION
This action is responsive to Claims filed 11/24/2025.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Claims
Claim 1 is amended. Claims 1-19 are currently pending.
Response to Arguments
The amendment(s) to Claim 1 have rectified the minor informalities.
Applicant’s arguments, see pages 7-10, filed 11/24/2025, illustrate sufficient structure recited in the Specification to overcome the interpretation under 112(f).
Applicant’s arguments, see Pages 10-14, filed 11/24/2025, with respect to the 35 U.S.C. 102(a)(2) rejection of Claims 1-2, 4, 6, 8-10, 11-15, and 18-19 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of 35 U.S.C. 103.
The Examiner contends the “model parameters” and “input or output between layers” continues to read broadly on the claimed “weight” and “activation” tensor, in the absence of further detail or amendments.
Regarding the buffer unit: The Examiner acknowledges the wording of the Rejection can be improved to clarify the intent. The Examiner was not attempting to expressly rely on inherency in mapping the cited section of Pope to the claimed buffer unit. As stated in the Rejection, the Examiner merely interprets the limitation broadly, given the BRI of the limitation is a buffer receiving and/or storing data, which the cited section of Pope broadly teaches, in the absence of further detail or amendments.
Applicant’s arguments, see Page 14, filed 11/24/2025, with respect to the 35 U.S.C. 103 rejection of Claims 3, 5, 7, and 16-17 with respect to claim(s) have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-19 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Jong Hoon Shin, Ali Shafiee, Ardavan Pedram, Hamzah Abdel-Aziz, Ling Li, and Joseph Hassoun (Griffin: Rethinking Sparse Optimization for Deep Learning Architectures, Published 11/01/2021), hereinafter Shin.
Applicant has provided evidence in this file showing that the claimed invention and the subject matter disclosed in the prior art reference were owned by, or subject to an obligation of assignment to, the same entity as Jong Hoon Shin, Ali Shafiee (Provisional Application 63/410216), Ardavan Pedram, and Joseph Hassoun, not later than the effective filing date of the claimed invention, or the subject matter disclosed in the prior art reference was developed and the claimed invention was made by, or on behalf of one or more parties to a joint research agreement in effect not later than the effective filing date of the claimed invention. However, although reference Shin has been excepted as prior art under 35 U.S.C. 102(a)(2), it is still applicable as prior art under 35 U.S.C. 102(a)(1) that cannot be excepted under 35 U.S.C. 102(b)(2)(C).
Applicant may rely on the exception under 35 U.S.C. 102(b)(1)(A) to overcome this rejection under 35 U.S.C. 102(a)(1) by a showing under 37 CFR 1.130(a) that the subject matter disclosed in the reference was obtained directly or indirectly from the inventor or a joint inventor of this application, and is therefore not prior art under 35 U.S.C. 102(a)(1). Alternatively, applicant may rely on the exception under 35 U.S.C. 102(b)(1)(B) by providing evidence of a prior public disclosure via an affidavit or declaration under 37 CFR 1.130(b).
In regards to claim 1: The present invention claims: “A memory system for training a neural network model, comprising: a decompressor unit configured to decompress an activation tensor to a first predetermined sparsity density based on the activation tensor being compressed, and to decompress a weight tensor to a second predetermined sparsity density based on the weight tensor being compressed;” Shin teaches “The python-based simulator receives weight and activation tensor blocks from pytorch and prepreprocess weight tensors,” (Page 6 activation and weight tensor); “For unstructured sparse input matrix A and B the zero operands are not necessarily evenly distributed. This issue still exists after preprocessing B or on-the-fly zero skipping on A. Coarse-grain load balancing is an effective approach to distribute nonzero values and improve performance utilization [18], [32]. In this approach a GEMM operation is decomposed into smaller blocks and each block is assigned to available idle PEs.” (Page 4, decompression); and “The cycle-accurate simulation estimate the inference end-to-end latency for the given benchmarks. The weight and activation (B,A) sparsity ratios, accuracy, and latency (i.e., number of cycles) with dense matrices for these benchmarks are listed in Table IV.” (Page 7, Table IV shows sparsity ratios for the weight and activation tensors).
“a buffer unit configured to receive the activation tensor at the first predetermined sparsity density and the weight tensor at the second predetermined sparsity density;” Shin teaches “At any cycle, the arbiter looks into a window of elements of A that are fetched from SRAM and currently reside in a buffer called ABUF. The elements of B corresponding to elements of A in ABUF are also fetched to a buffer called BBUF.” (Page 4).
“and a neural processing unit configured to receive the activation tensor and the weight tensor from the buffer unit and to compute a result for the activation tensor and the weight tensor based on first predetermined sparsity density of the activation tensor and based on the second predetermined sparsity density of the weight tensor.” Shin Figure 2 (Page 3) highlights the processing/output of the weight and activation tensors’ processing.
In regards to claim 2: The present invention claims: “wherein the first predetermined sparsity density is based on a structured-sparsity arrangement or a random-sparsity arrangement.” Shin teaches “For unstructured sparse input matrix A and B the zero operands are not necessarily evenly distributed. This issue still exists after preprocessing B or on-the-fly zero skipping on A.” (Page 4).
In regards to claim 3: The present invention claims: “wherein the first predetermined sparsity density is based on a 1:4 structured-sparsity arrangement, or a 2:8 structured-sparsity arrangement.” Shin teaches “The cycle-accurate simulation estimate the inference end-to-end latency for the given benchmarks. The weight and activation (B,A) sparsity ratios, accuracy, and latency (i.e., number of cycles) with dense matrices for these benchmarks are listed in Table IV.” (Page 7, Table IV shows sparsity ratios for the weight and activation tensors).
In regards to claim 4: The present invention claims: “wherein the second predetermined sparsity density is based on a structured-sparsity arrangement or a random-sparsity arrangement.” Shin teaches “For unstructured sparse input matrix A and B the zero operands are not necessarily evenly distributed. This issue still exists after preprocessing B or on-the-fly zero skipping on A.” (Page 4).
In regards to claim 5: The present invention claims: “wherein the second predetermined sparsity density is based on a 1:4 structured-sparsity arrangement, or a 2:8 structured-sparsity arrangement.” Shin teaches “The cycle-accurate simulation estimate the inference end-to-end latency for the given benchmarks. The weight and activation (B,A) sparsity ratios, accuracy, and latency (i.e., number of cycles) with dense matrices for these benchmarks are listed in Table IV.” (Page 7, Table IV shows sparsity ratios for the weight and activation tensors).
In regards to claim 6: The present invention claims: “wherein the second predetermined sparsity density is based on a structured-sparsity arrangement or a random-sparsity arrangement.” Shin teaches “The cycle-accurate simulation estimate the inference end-to-end latency for the given benchmarks. The weight and activation (B,A) sparsity ratios, accuracy, and latency (i.e., number of cycles) with dense matrices for these benchmarks are listed in Table IV.” (Page 7, Table IV shows sparsity ratios for the weight and activation tensors).
In regards to claim 7: The present invention claims: “wherein the second predetermined sparsity density is based on a 1:4 structured-sparsity arrangement, or a 2:8 structured-sparsity arrangement.” Shin teaches “The cycle-accurate simulation estimate the inference end-to-end latency for the given benchmarks. The weight and activation (B,A) sparsity ratios, accuracy, and latency (i.e., number of cycles) with dense matrices for these benchmarks are listed in Table IV.” (Page 7, Table IV shows sparsity ratios for the weight and activation tensors).
In regards to claim 8: The present invention claims: “wherein the decompressor unit is further configured to decompress the activation tensor to the first predetermined sparsity density using first metadata associated with the activation tensor and is further configured to decompress the weight tensor to the second predetermined sparsity density using second metadata associated with the weight tensor.” Shin Page 5, right column and Page 6, left column refer to the use of metadata for the A and B matrices (weight and activation matrices).
In regards to claim 9: The present invention claims: “a compressor unit configured to receive and compress the result computed by the neural processing unit, and a memory further stores the result compressed by the compressor unit.” Shin teaches “Preprocessing B: Since B is known before the execution, it is preprocessed to a compressed format with metadata. The preprocessed elements of B in SRAM are fetched in BBUF, which holds a window of current elements every cycle. In our example, b1;0, b3;0, b5;0, b6;0, b7;0, and b10;0 are kept in BBUF.” (Page 5)
In regards to claim 10: The present invention claims: “wherein the compressor unit is further configured to generate metadata associated with the result, and wherein the memory further stores the metadata.” Shin teaches “Preprocessing B: Since B is known before the execution, it is preprocessed to a compressed format with metadata. The preprocessed elements of B in SRAM are fetched in BBUF, which holds a window of current elements every cycle. In our example, b1;0, b3;0, b5;0, b6;0, b7;0, and b10;0 are kept in BBUF.” (Page 5)
In regards to claims 11-19: Claims 11-19 recites similar limitations to claims 1--10 with the exception of “A memory system for training a neural network model, comprising:” of claim 11. Therefore, both sets of claims are similarly rejected.
Claim Rejections - 35 USC § 103
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1-2, 4, 6, 8-15, and 18-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Pope (US 11,728,826 B2), hereinafter Pope and Parashar et al. (SCNN: An Accelerator for Compressed-sparse Convolutional Neural Networks, 2017), hereinafter Parashar.
In regards to claim 1: The present invention claims: “A memory system for training a neural network model, comprising: a decompressor unit configured to decompress an activation tensor…based on the activation tensor being compressed, and to decompress a weight tensor…based on the weight tensor being compressed;” Pope teaches “The data can be stored and sent by the memory device 130 according to a variety of different formats. In examples in which the workload for the system 100 is to execute or train a neural network, data can be stored and sent as tensors. A tensor is a multi-dimensional array. For example, a zero-dimensional tensor is a scalar value, a one-dimensional tensor is a vector, and a two-dimensional tensor is a matrix. Elements of a tensor can be, for example values for different model parameters [weight tensor] for a given layer of a neural network, or input or output between layers [activation tensor] of a neural network or of the neural network itself.” (Column 7, Lines 11-21) See Pope Figure 1 for the memory device sending the compressed data to a decompression unit (Item 110). Figure 1 also illustrates the data sent to the decompressor is already compressed [based on the…tensor being compressed]
“a buffer unit configured to receive the activation tensor…and the weight tensor…” (Examiner’s Note: The Examiner interprets this limitation to broadly point to a buffer or memory unit for storing the decompressed data) Pope teaches the memory and processing device can be any of multiple, commonplace memory and processing technologies (Column 6, Lines 55-63).
“and a neural processing unit configured to receive the activation tensor and the weight tensor from the buffer unit and to compute a result for the activation tensor and the weight tensor …the activation tensor and based on …the weight tensor.” (Examiner’s Note: The Examiner interprets this limitation to broadly point to executing or training the neural processing unit on the decompressed data). Pope teaches “The decompressor device 120 receives and processes the compressed data 112A to generate uncompressed data 114A. The processor 140 receives the uncompressed data 114A and performs one or more operations on the data. For example, the processor 140 can be configured for performing logical or arithmetic operations on the uncompressed data 114A, and generate output uncompressed data 114B.” See above where Pope teaches the system can be for training or executing a neural network.
Pope fails to explicitly teach “a first predetermined sparsity density” and “a second predetermined sparsity density” explicitly; however, Parashar, in a similar field of endeavor, teaches the benefits of exploiting sparsity in weight and activation tensors, particularly in Section 6.1 (Page 36), where Figure 8 demonstrates improved performance as sparsity increases.
Parashar indicates the improved performance of models like SCNN over the state of the art at the time of their writing as sparsity increases (Page 36). It would have been obvious to one of ordinary skill in the art at the time of the Applicant’s filing to incorporate sparsity densities requisite to achieve their performance desires when implemented in a compression/decompression system such as Pope’s.
In regards to claim 2: The present invention claims: “wherein the first predetermined sparsity density is based on a structured-sparsity arrangement or a random-sparsity arrangement.” Pope utilizes entropy encoding for compression and decompression (Brief Summary, Columns 1-2). A cursory search shows entropy encoding is a structure-sparsity arrangement. Parashar also teaches “Figure 7 shows an example of SCNN’s compressed sparse encoding for R = S = 3 and K = 2 with 6 non-zero elements. The encoding includes a data vector consisting of the non-zero values and an index vector that includes the number of non-zero values followed by the number of zeros before each value.” (Pages 33-34).
In regards to claim 4: The present invention claims: “wherein the second predetermined sparsity density is based on a structured-sparsity arrangement or a random-sparsity arrangement.” Pope utilizes entropy encoding for compression and decompression (Brief Summary, Columns 1-2). A cursory search shows entropy encoding is a structure-sparsity arrangement. Parashar also teaches “Figure 7 shows an example of SCNN’s compressed sparse encoding for R = S = 3 and K = 2 with 6 non-zero elements. The encoding includes a data vector consisting of the non-zero values and an index vector that includes the number of non-zero values followed by the number of zeros before each value.” (Pages 33-34).
In regards to claim 6: The present invention claims: “wherein the second predetermined sparsity density is based on a structured-sparsity arrangement or a random-sparsity arrangement.” Pope utilizes entropy encoding for compression and decompression (Brief Summary, Columns 1-2). A cursory search shows entropy encoding is a structure-sparsity arrangement. Parashar also teaches “Figure 7 shows an example of SCNN’s compressed sparse encoding for R = S = 3 and K = 2 with 6 non-zero elements. The encoding includes a data vector consisting of the non-zero values and an index vector that includes the number of non-zero values followed by the number of zeros before each value.” (Pages 33-34).
In regards to claim 8: The present invention claims: “wherein the decompressor unit is further configured to decompress the activation tensor to the first predetermined sparsity density using first metadata associated with the activation tensor and is further configured to decompress the weight tensor to the second predetermined sparsity density using second metadata associated with the weight tensor.” Pope Column 10, Lines 26-43 details how the compression unit may include or concatenate pertinent entropy codewords to the data it receives. The Examiner maps this to the broad recitation of “metadata” in the context of its relevance to the compression or decompression of the data stored in the memory device. Column 3, Lines 10-12 (at least) teach “The decompressor device can include a plurality of 10 entropy coders configured to decompress data using the entropy encoding.” The Examiner maps this to the broad recitation of using the metadata to decompress the data in conjunction with sparsity densities taught by Parashar.
In regards to claim 9: The present invention claims: “a compressor unit configured to receive and compress the result computed by the neural processing unit, and a memory further stores the result compressed by the compressor unit.” See above where Pope Figure 1 shows a compressor which stores compressed data into a memory unit.
In regards to claim 10: The present invention claims: “wherein the compressor unit is further configured to generate metadata associated with the result, and wherein the memory further stores the metadata.” Pope Column 10, Lines 26-43 details how the compression unit may include or concatenate pertinent entropy codewords to the data it receives. The Examiner maps this to the broad recitation of “metadata” in the context of its relevance to the compression or decompression of the data stored in the memory device in conjunction with sparsity densities taught by Parashar.
In regards to claims 11-15 and 18-19: Claims 11-15 and 18-19 recites similar limitations to claims 1-2, 4, 6, and 8-10 with the exception of “A memory system for training a neural network model, comprising:” of claim 11. Therefore, both sets of claims are similarly rejected.
Claim(s) 3, 5, 7, 16-17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Pope and Parashar as applied to claims 1 and 11 above, in further view of Zhou et al. (LEARNING N:M FINE-GRAINED STRUCTURED SPARSE NEURAL NETWORKS FROM SCRATCH, 2021), hereinafter Zhou.
In regards to claims 3, 5, 7, 16-17: The claims similarly recite a first and second predetermined sparsity density “is based on a 1:4 structured-sparsity arrangement, or a 2:8 structured-sparsity arrangement.” While Pope makes reference to compression ratios throughout their disclosure (Background, etc. at least), and Parashar Figure 8 at least references sparsity ratios as well, the combination of Pope and Parashar fails to explicitly teach ratios with the claimed specificity.
However, Zhou teaches “We also compare the performance of neural networks with different granularities of fine-grained structured sparsity (i.e., 1:4, 2:4, 2:8, 4:8) and conduct thorough experiments on several typical deep neural networks with different N:M sparsity levels, covering image classification, detection, segmentation, optical flow estimation, and machine translation.” (Page 2) and “The sparsity of 1:4 and 2:8 are both 75%. In Table 1, we observe that the 4:8 structural sparsity outperforms 2:4 with the same computational cost, and 2:8 also performs better than 1:4. the training curve in Fig. 6(a). It shows that with the same sparsity for N:M structural sparse patterns, a larger M will lead to better performance since it can provide more abundant convolution kernel shape…” (Page 8).
Zhou demonstrates that using fine grain compression ratios such as 1:4 and 2:8 would have been known in the art at the time of the Applicant’s filing, and demonstrates a benefit of using 2:8 over 1:4. It would have been obvious to one of ordinary skill in the art at the time of the Applicant’s filing to combine the systems of Pope and Parashar with the known methods of Zhou.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GRIFFIN T BEAN whose telephone number is (703)756-1473. The examiner can normally be reached M - F 7:30 - 4:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Li Zhen can be reached at (571) 272-3768. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GRIFFIN TANNER BEAN/ Examiner, Art Unit 2121
/Li B. Zhen/ Supervisory Patent Examiner, Art Unit 2121
Prosecution Insights