DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 6/26/2026 has been entered.
Other reference: Takasugi (US 7219200) discloses memory device allowing only memories capable of providing quick serial access in a row direction to obtain quick serial access in a column direction (see Fig. 1),
relates to claimed row slices and column slices.
Frayer (US 20150364215) – Low-Test memory stack for non-volatile storage with parallel memory operations on RAID stripe (0102).
Danilov (US 11023331) – fast recovery of data in a geographically distributed storage environment (col 1:6-10) relates to claimed computing system comprising request to retrieve particular slices.
Herault, Thomas, Generic Matrix Multiplication for Multi-GPU Accelerated Distributed-Memory Platforms over PARSEC, 2019 IEEE/ACM 10th Workshop on Latest Advance in Scalable Algorithms for Large-Scale System (ScalA), relates to claimed computing system comprising plurality of memory banks in series.. plurality of slices.
In order to potentially overcome the rejections, the Office suggest amending the claims (providing showing of support from the original disclosure) to clarify the limitations related to: “the same sub-banks” (Claim 1, line 9) and “a same sub-bank” (Claim 1, line 14) such as to clarify the antecedent basis; and similarly for Claims 9 and 16.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1 –3, 5, 9 – 11,13, 14 and 16-18 are rejected under 35 U.S.C. 103 as being unpatentable over Wai et al. (US 20220058126) in view of Chatterjee (US 20170255552) and further in view of Narayanamoorthy (US 20190042542)
Regarding claim 1, Wai a computing system comprising: a processor; and
a storage subsystem comprising a plurality of memory banks configured in . . . memory banks comprising a plurality of sub-banks, (eg., Figure 5 and a plurality of memory sub-banks in Figures 1, 2 and 3. [0006] the addressing scheme defining a plurality of memory tiles, each memory tile among the plurality of memory tiles designated as belonging to a memory bank among a plurality of memory banks and a memory sub-banks.)
wherein the processor is configured to store whole memory tiles (eg., [0006] memory element as a first memory element for storing a first matrix element among a plurality of matrix elements, 0036 - [0036] A bank “0” in the memory segment 100 includes memory tiles 102-0/0, 102-0/1, 102-0/2, 102-0/3, 102-0/4. A bank “1” includes memory tiles ; [0043] matrix element defined by a row and a column … row and the column of the matrix element to a specific bank and sub-bank)
Wai does not disclose, but Chatterjee discloses
each comprising a plurality of slices in the memory banks, , wherein different slices corresponding to a same memory tile are stored in different sub-banks (eg., see Fig. 1B [0029] Each bank 120 is organized as a two-dimensional (2D) array of storage units such as the storage units 155-0A through 155-3A, 155-4A through 155-7A, 155-0B through 155-3B, and 155-3B through 155-7A. Details of the storage unit 155 are shown in FIG. 1C. In the context of the following description, a sub-channel 180 may include T storage units 155, where T is an integer value greater or equal to 1. As shown in FIG. 1B, the sub-channel 170-0 includes the four storage units 155-0A through 155-3A and the four storage units 155-0B through 155-3B. In one embodiment, the four storage units 155 within each of the storage units 155-0A through 155-3A and the storage units 155-0B through 155-3B are aligned horizontally and share a CA (column address) storage 150-0 and global SAs (sense amplifiers) 145-0. In other embodiments, more or less than four storage units 155 may be included in each of the storage units 155-0A through 155-3A and 155-0B through 155-3B. Similarly, the sub-channel 170-1 includes the storage units 155-4A through 155-7A and storage units 155-4B through 155-7B. In one embodiment, the four storage units 155 within each of the storage units 155-4A through 155-7A and the storage units 155-3B through 155-7B may each include four storage units 155 that are aligned horizontally and share a CA storage 150-1 and global SAs 145-1. In other embodiments, more or less than four storage units 155 may be included in each of the storage units 155-4A through 155-7A and 155-4B through 155-7B.)
corresponding slice positions of the plurality of slices of the memory tiles are stored in the same sub-banks (eg., 0020 Fig. 1B - the subarray 180-A includes a subarray row address logic 115-A, the storage units 155-0A through 155-3A, 155-4A through 155-7A and the subarray 180-B includes a subarray row address logic 115-B, the storage units 155-0B through 155-3B and 155-4B through 155-7B.; 0028-0029 FIG. 1B, includes two sub-channels, a sub-channel 170-0 and a sub-channel 170-1; disclosed sub-channels render claimed sub-banks as obvious and disclosed subarray row address 115-A and 115-B render claimed slices as obvious; disclosed different storage units 155-4A and 155-4B are in the same sub-channels );
and wherein the plurality of slices include one or more row slices and one or more corresponding column slices (eg., 0005 - first sub-array is associated with a first row address and the first sub-channel is associated with a first column address that is stored in the first sub-channel and used to select a first portion of data from the first row-buffer)
wherein a request to retrieve particular slices from two or more particular memory tiles sequentially moves between the plurality of memory banks to retrieve a predetermined amount of data (eg., 0051 - a single burst access, CSLs should be asserted sequentially in a sub-channel 170. To generate the sequence of CSLs,)
corresponding to different memory tiles from a same sub-bank (eg., [0033] Fig. 1B - The I/O buffer 160-0 is included in the sub-channel 170-0 and the I/O buffer 160-1 is included in the sub-channel 170-1. The I/O buffer 160-0 is coupled to the storage units 155-0A through 155-3A and the storage units 155-0B through 155-3B through the global sense amplifiers 145-0. The I/O buffer 160-1 is coupled to the storage units 155-4A through 155-7A and the storage units 155-3B through 155-7B; 0021 - multiple sub-channels can be operated in parallel to return different access requests simultaneously- this disclosure renders obvious that I/O buffer 160-1 is configured to retrieve from storage units 155-4B and 155-4A)
It would have been obvious to one of ordinary skill in the art prior to the filing date of the claimed invention to modify the memory storage system having banks and sub-banks of Wai in view of Chatterjee, providing the benefit of multiple sub-channels can be operated in parallel to return different access requests simultaneously (see Chatterjee, 0021) A DRAM sub-channel architecture with improved energy efficiency (0020) to reduce the minimum row-activation granularity and the DRAM datapath is divided while avoiding significant increases in the die area (0021).
Wai in view of Chatterjee does not disclose, but Narayanamoorthy discloses
the plurality of slices comprising a plurality of numerical data values (eg., [0046] the specified sparse source matrix 212 is logically a M by K matrix, but only its non-zero elements are stored in memory in a compressed sparse row (CSR) or compressed sparse column (CSC) format,)
wherein the request retrieves two or more slices in a single cycle (eg., [0162] core may support multithreading (executing two or more parallel sets of operations or threads), and may do so in a variety of ways including time sliced multithreading, simultaneous multithreading (where a single physical core provides a logical core for each of the threads that physical core is simultaneously multithreading), or a combination thereof (e.g., time sliced fetching and decoding and simultaneous multithreading thereafter such as in the Intel® Hyperthreading technology).).
and wherein the particular slices are selectively retrieved based on a multiplication result between numerical … refraining from retrieving the row slice upon determining that the multiplication result is the zero value (eg., 0058 - the specified sparse source matrix is stored in sparse format in memory 412, such that the execution circuit 411 reads the sparsely-formatted matrix, but only buffers the non-zero values in sparse buffers 414.; 0199 Fig. 2A - Example 1 provides an exemplary processor to execute a sparse-dense matrix multiplication instruction, comprising: fetch circuitry to fetch, from code storage, the sparse-dense matrix multiplication instruction having fields to specify an opcode, a dense output matrix, a dense source matrix, and a sparse source matrix having a sparsity of non-zero elements ; 0162 - multithreading (executing two or more parallel sets of operations or threads), and may do so in a variety of ways including time sliced multithreading, simultaneous multithreading (where a single physical core provides a logical core for each of the threads that physical core is simultaneously multithreading), or a combination thereof (e.g., time sliced fetching and decoding and simultaneous multithreading thereafter such as in the Intel® Hyperthreading technology).
It would have been obvious to one of ordinary skill in the art prior to the filing date of the claimed invention to modify the memory storage system having banks and sub-banks of Wai in view of Chatterjee, with Narayanamoorthy providing the benefit of Sparse-dense matrix multiplication (SDMM) operations are useful in a deep learning context. But traditional CPU and GPU instruction set architectures require symmetric inputs having the same density, which limits the ability to gain a performance advantage by taking advantage of the sparsity of a sparse input matrix (see Narayanamoorthy, 0004) executing a sparse-dense matrix multiplication (SDMM) instruction (0040).
Regarding claim 2, Wai discloses the computing system of claim 1. Wai teaches wherein the predetermined amount of data comprises an amount of data stored in one whole tile, Wai teaches Figure 1 a diagram of memory segments including banks and sub-banks. [0005] Indeed, matrix elements in a data matrix may be mapped to the memory elements within the given memory segment in accordance with the addressing scheme, and [0006] first selected memory element as a first memory element for storing a first matrix element among a plurality of matrix elements. Where a collection of one or more matrix elements is analogous to a tile.
Regarding claim 3, Wai discloses the computing system of claim 1. Wai teaches wherein the request retrieves a plurality of different slices from a plurality of different tile stored in a plurality of different memory banks. Wai teaches retrieval of a plurality of different slices from a plurality of different tile stored in a plurality of different memory banks, where [0004] - [0006] A memory tile, in a local memory, may be considered to be a unit of memory that carries multiple memory elements, wherein each memory element is a one-dimensional memory structure . . . matrix elements in a data matrix may be mapped to the memory elements within the given memory segment. A memory tile and slice are synonymous.
Regarding claim 5, Wai discloses the computing system of claim 1. Wai further teaches wherein the request retrieves a plurality of same slices from a plurality of different tile stored in a same memory bank in a plurality of cycles. [0055] eight memory lines (up to 32E per memory line) to be accessed in a single clock cycle and [0061]- [0063] access to a maximum of 50 memory lines in a single clock cycle ... access 32 memory elements in a single clock cycle.
Regarding claim 9, Wai teaches A method of storing and retrieving data comprising: [0006] method . . . including, . . . accessing . . . for storing matrix element among a plurality of matrix elements, . . . matrix elements in the plurality of matrix elements being equivalent to a quantity of memory elements in the plurality of memory elements each memory line and accessing the second selected memory element as a second memory . . . storing a plurality of whole tile in memory banks of a memory computing system Figures 2 and 3 and [0067] mappings of matrix elements of various matrices to memory locations, as identified by memory entry and by bank . . . comprising a plurality of said memory banks configured . . ., the memory banks comprising a plurality of sub-banks, Figures 13 & 14 and [0035] A memory segment arranged … a two-dimensional memory array of B×SB memory tiles, where “B” is the number of rows of memory tiles, with each row referred to as a “bank,” and where “SB” is the number of columns of memory tiles, with each column referred to as a “sub-bank” and [0050] bank 0 and sub-bank 0 … bank 0, the memory tile 102-0/1 in the next sub-bank
and the tile comprising a plurality of slices, as previously identified, the term “slices” in synonymous with “elements or matrix elements” in [0006] in a single clock cycle, accessing the first selected memory element as a first memory element for storing a first matrix element among a plurality of matrix elements, a quantity of matrix elements in the plurality of matrix elements being equivalent to a quantity of memory elements in the plurality of memory elements each memory line and accessing the second selected memory element as a second memory element for storing a second matrix element among a second plurality of matrix elements.
the slices comprising a plurality of data values, Figures 1 and 2 show memory banks, sub-banks and tiles. Tile or groups of matrix elements can be stored sub-banks. [0062] eight 16-bit matrix elements are stored in the eight 16-bit memory elements in the memory line with index 0 in the memory tile 102-0/0 of memory entry 0, the matrix elements may be indexed as CO-7. The subsequent eight 16-bit matrix elements may be stored in eight 16-bit memory elements in a memory in a memory tile 102 in a bank different from bank 0 and may be indexed as C8-15. Similarly, each subsequent set of eight memory elements may be stored in a memory tile 102 in a bank distinct from the other banks.
Wai does not disclose, but Chatterjee discloses
each comprising a plurality of slices in the memory banks, , wherein different slices corresponding to a same memory tile are stored in different sub-banks (eg., see Fig. 1B [0029] Each bank 120 is organized as a two-dimensional (2D) array of storage units such as the storage units 155-0A through 155-3A, 155-4A through 155-7A, 155-0B through 155-3B, and 155-3B through 155-7A. Details of the storage unit 155 are shown in FIG. 1C. In the context of the following description, a sub-channel 180 may include T storage units 155, where T is an integer value greater or equal to 1. As shown in FIG. 1B, the sub-channel 170-0 includes the four storage units 155-0A through 155-3A and the four storage units 155-0B through 155-3B. In one embodiment, the four storage units 155 within each of the storage units 155-0A through 155-3A and the storage units 155-0B through 155-3B are aligned horizontally and share a CA (column address) storage 150-0 and global SAs (sense amplifiers) 145-0. In other embodiments, more or less than four storage units 155 may be included in each of the storage units 155-0A through 155-3A and 155-0B through 155-3B. Similarly, the sub-channel 170-1 includes the storage units 155-4A through 155-7A and storage units 155-4B through 155-7B. In one embodiment, the four storage units 155 within each of the storage units 155-4A through 155-7A and the storage units 155-3B through 155-7B may each include four storage units 155 that are aligned horizontally and share a CA storage 150-1 and global SAs 145-1. In other embodiments, more or less than four storage units 155 may be included in each of the storage units 155-4A through 155-7A and 155-4B through 155-7B.)
corresponding slice positions of the plurality of slices of the memory tiles are stored in the same sub-banks (eg., 0020 Fig. 1B - the subarray 180-A includes a subarray row address logic 115-A, the storage units 155-0A through 155-3A, 155-4A through 155-7A and the subarray 180-B includes a subarray row address logic 115-B, the storage units 155-0B through 155-3B and 155-4B through 155-7B.; 0028-0029 FIG. 1B, includes two sub-channels, a sub-channel 170-0 and a sub-channel 170-1; disclosed sub-channels render claimed sub-banks as obvious and disclosed subarray row address 115-A and 115-B render claimed slices as obvious; disclosed different storage units 155-4A and 155-4B are in the same sub-channels );
and wherein the plurality of slices include one or more row slices and one or more corresponding column slices (eg., 0005 - first sub-array is associated with a first row address and the first sub-channel is associated with a first column address that is stored in the first sub-channel and used to select a first portion of data from the first row-buffer)
wherein a request to retrieve particular slices from two or more particular memory tiles sequentially moves between the plurality of memory banks to retrieve a predetermined amount of data (eg., 0051 - a single burst access, CSLs should be asserted sequentially in a sub-channel 170. To generate the sequence of CSLs,)
corresponding to different memory tiles from a same sub-bank (eg., [0033] Fig. 1B - The I/O buffer 160-0 is included in the sub-channel 170-0 and the I/O buffer 160-1 is included in the sub-channel 170-1. The I/O buffer 160-0 is coupled to the storage units 155-0A through 155-3A and the storage units 155-0B through 155-3B through the global sense amplifiers 145-0. The I/O buffer 160-1 is coupled to the storage units 155-4A through 155-7A and the storage units 155-3B through 155-7B; 0021 - multiple sub-channels can be operated in parallel to return different access requests simultaneously- this disclosure renders obvious that I/O buffer 160-1 is configured to retrieve from storage units 155-4B and 155-4A)
It would have been obvious to one of ordinary skill in the art prior to the filing date of the claimed invention to modify the memory storage system having banks and sub-banks of Wai in view of Chatterjee, providing the benefit of multiple sub-channels can be operated in parallel to return different access requests simultaneously (see Chatterjee, 0021) A DRAM sub-channel architecture with improved energy efficiency (0020) to reduce the minimum row-activation granularity and the DRAM datapath is divided while avoiding significant increases in the die area (0021).
Wai in view of Chatterjee does not disclose, but Narayanamoorthy discloses
the plurality of slices comprising a plurality of numerical data values (eg., [0046] the specified sparse source matrix 212 is logically a M by K matrix, but only its non-zero elements are stored in memory in a compressed sparse row (CSR) or compressed sparse column (CSC) format,)
wherein the request retrieves two or more slices in a single cycle (eg., [0162] core may support multithreading (executing two or more parallel sets of operations or threads), and may do so in a variety of ways including time sliced multithreading, simultaneous multithreading (where a single physical core provides a logical core for each of the threads that physical core is simultaneously multithreading), or a combination thereof (e.g., time sliced fetching and decoding and simultaneous multithreading thereafter such as in the Intel® Hyperthreading technology).).
and wherein the particular slices are selectively retrieved based on a multiplication result between numerical … refraining from retrieving the row slice upon determining that the multiplication result is the zero value (eg., 0058 - the specified sparse source matrix is stored in sparse format in memory 412, such that the execution circuit 411 reads the sparsely-formatted matrix, but only buffers the non-zero values in sparse buffers 414.; 0199 Fig. 2A - Example 1 provides an exemplary processor to execute a sparse-dense matrix multiplication instruction, comprising: fetch circuitry to fetch, from code storage, the sparse-dense matrix multiplication instruction having fields to specify an opcode, a dense output matrix, a dense source matrix, and a sparse source matrix having a sparsity of non-zero elements ; 0162 - multithreading (executing two or more parallel sets of operations or threads), and may do so in a variety of ways including time sliced multithreading, simultaneous multithreading (where a single physical core provides a logical core for each of the threads that physical core is simultaneously multithreading), or a combination thereof (e.g., time sliced fetching and decoding and simultaneous multithreading thereafter such as in the Intel® Hyperthreading technology).
It would have been obvious to one of ordinary skill in the art prior to the filing date of the claimed invention to modify the memory storage system having banks and sub-banks of Wai in view of Chatterjee, with Narayanamoorthy providing the benefit of Sparse-dense matrix multiplication (SDMM) operations are useful in a deep learning context. But traditional CPU and GPU instruction set architectures require symmetric inputs having the same density, which limits the ability to gain a performance advantage by taking advantage of the sparsity of a sparse input matrix (see Narayanamoorthy, 0004) executing a sparse-dense matrix multiplication (SDMM) instruction (0040).
Regarding claim 10, it has substantially similar limitations as stated in claims 2; therefore, it is rejected under the same subject matter.
Regarding claim 11, it has substantially similar limitations as stated in claims 4; therefore, it is rejected under the same subject matter.
Regarding claim 13, Wai further teaches wherein the request retrieves a plurality of same slices from a plurality of different tile stored in a same memory bank in a plurality of cycles. Wai teaches [0055] eight memory lines (up to 32E per memory line) to be accessed in a single clock cycle and [0061]- [0063] access to a maximum of 50 memory lines in a single clock cycle ... access 32 memory elements in a single clock cycle.
Regarding claim 14. Wai further teaches wherein the request comprises a plurality of addresses for a corresponding plurality of tile and, for each tile, a corresponding slice mask specifying slices of each tile to be retrieved. Wai teaches an address translation computing system in Figures 5,6, 10,11 12 and 14 which are capable of address the memory banks, sub banks and tiles described. These memories can be used to access matrices and matrix elements [0005] – [0006] memory elements carried by the memory tiles in a given memory segment, it may be shown that non-blocking, concurrent write and read accesses to the memory elements may be achieved, for multiple requestors, with relatively high throughput. Indeed, matrix elements in a data matrix may be mapped to the memory elements within the given memory segment in accordance with the addressing scheme. The accesses may be either row-major or column-major when the data matrix is two-dimensional and a first memory element for storing a first matrix element among a plurality of matrix elements, a quantity of matrix elements in the plurality of matrix elements.
Regarding claim 16, Wai teaches computer storage subsystem storing a program executable by a computer for storing and retrieving data, the program comprising sets of instructions for: [0034] non-transitory computer/processor readable storage medium or media for storage of information, such as computer/processor readable instructions . . .
storing a plurality whole tile in memory banks of a memory computing system Figures 2 and 3 and [0067] mappings of matrix elements of various matrices to memory locations, as identified by memory entry and by bank . . . comprising a plurality of said memory banks configured . . ., the memory banks comprising a plurality of sub-banks, Figures 13 & 14 and [0035] A memory segment arranged … a two-dimensional memory array of B×SB memory tiles, where “B” is the number of rows of memory tiles, with each row referred to as a “bank,” and where “SB” is the number of columns of memory tiles, with each column referred to as a “sub-bank” and [0050] bank 0 and sub-bank 0 … bank 0, the memory tile 102-0/1 in the next sub-bank
and the tile comprising a plurality of slices, as previously identified, the term “slices” in synonymous with “elements or matrix elements” in [0006] in a single clock cycle, accessing the first selected memory element as a first memory element for storing a first matrix element among a plurality of matrix elements, a quantity of matrix elements in the plurality of matrix elements being equivalent to a quantity of memory elements in the plurality of memory elements each memory line and accessing the second selected memory element as a second memory element for storing a second matrix element among a second plurality of matrix elements.
the slices comprising a plurality of data values; Figures 1 and 2 show memory banks, sub-banks and tiles. Tile or groups of matrix elements can be stored sub-banks. [0062] eight 16-bit matrix elements are stored in the eight 16-bit memory elements in the memory line with index 0 in the memory tile 102-0/0 of memory entry 0, the matrix elements may be indexed as CO-7. The subsequent eight 16-bit matrix elements may be stored in eight 16-bit memory elements in a memory in a memory tile 102 in a bank different from bank 0 and may be indexed as C8-15. Similarly, each subsequent set of eight memory elements may be stored in a memory tile 102 in a bank distinct from the other banks.
Wai does not disclose, but Chatterjee discloses
each comprising a plurality of slices in the memory banks, , wherein different slices corresponding to a same memory tile are stored in different sub-banks (eg., see Fig. 1B [0029] Each bank 120 is organized as a two-dimensional (2D) array of storage units such as the storage units 155-0A through 155-3A, 155-4A through 155-7A, 155-0B through 155-3B, and 155-3B through 155-7A. Details of the storage unit 155 are shown in FIG. 1C. In the context of the following description, a sub-channel 180 may include T storage units 155, where T is an integer value greater or equal to 1. As shown in FIG. 1B, the sub-channel 170-0 includes the four storage units 155-0A through 155-3A and the four storage units 155-0B through 155-3B. In one embodiment, the four storage units 155 within each of the storage units 155-0A through 155-3A and the storage units 155-0B through 155-3B are aligned horizontally and share a CA (column address) storage 150-0 and global SAs (sense amplifiers) 145-0. In other embodiments, more or less than four storage units 155 may be included in each of the storage units 155-0A through 155-3A and 155-0B through 155-3B. Similarly, the sub-channel 170-1 includes the storage units 155-4A through 155-7A and storage units 155-4B through 155-7B. In one embodiment, the four storage units 155 within each of the storage units 155-4A through 155-7A and the storage units 155-3B through 155-7B may each include four storage units 155 that are aligned horizontally and share a CA storage 150-1 and global SAs 145-1. In other embodiments, more or less than four storage units 155 may be included in each of the storage units 155-4A through 155-7A and 155-4B through 155-7B.)
corresponding slice positions of the plurality of slices of the memory tiles are stored in the same sub-banks (eg., 0020 Fig. 1B - the subarray 180-A includes a subarray row address logic 115-A, the storage units 155-0A through 155-3A, 155-4A through 155-7A and the subarray 180-B includes a subarray row address logic 115-B, the storage units 155-0B through 155-3B and 155-4B through 155-7B.; 0028-0029 FIG. 1B, includes two sub-channels, a sub-channel 170-0 and a sub-channel 170-1; disclosed sub-channels render claimed sub-banks as obvious and disclosed subarray row address 115-A and 115-B render claimed slices as obvious; disclosed different storage units 155-4A and 155-4B are in the same sub-channels );
and wherein the plurality of slices include one or more row slices and one or more corresponding column slices (eg., 0005 - first sub-array is associated with a first row address and the first sub-channel is associated with a first column address that is stored in the first sub-channel and used to select a first portion of data from the first row-buffer)
wherein a request to retrieve particular slices from two or more particular memory tiles sequentially moves between the plurality of memory banks to retrieve a predetermined amount of data (eg., 0051 - a single burst access, CSLs should be asserted sequentially in a sub-channel 170. To generate the sequence of CSLs,)
corresponding to different memory tiles from a same sub-bank (eg., [0033] Fig. 1B - The I/O buffer 160-0 is included in the sub-channel 170-0 and the I/O buffer 160-1 is included in the sub-channel 170-1. The I/O buffer 160-0 is coupled to the storage units 155-0A through 155-3A and the storage units 155-0B through 155-3B through the global sense amplifiers 145-0. The I/O buffer 160-1 is coupled to the storage units 155-4A through 155-7A and the storage units 155-3B through 155-7B; 0021 - multiple sub-channels can be operated in parallel to return different access requests simultaneously- this disclosure renders obvious that I/O buffer 160-1 is configured to retrieve from storage units 155-4B and 155-4A)
It would have been obvious to one of ordinary skill in the art prior to the filing date of the claimed invention to modify the memory storage system having banks and sub-banks of Wai in view of Chatterjee, providing the benefit of multiple sub-channels can be operated in parallel to return different access requests simultaneously (see Chatterjee, 0021) A DRAM sub-channel architecture with improved energy efficiency (0020) to reduce the minimum row-activation granularity and the DRAM datapath is divided while avoiding significant increases in the die area (0021).
Wai in view of Chatterjee does not disclose, but Narayanamoorthy discloses
the plurality of slices comprising a plurality of numerical data values (eg., [0046] the specified sparse source matrix 212 is logically a M by K matrix, but only its non-zero elements are stored in memory in a compressed sparse row (CSR) or compressed sparse column (CSC) format,)
wherein the request retrieves two or more slices in a single cycle (eg., [0162] core may support multithreading (executing two or more parallel sets of operations or threads), and may do so in a variety of ways including time sliced multithreading, simultaneous multithreading (where a single physical core provides a logical core for each of the threads that physical core is simultaneously multithreading), or a combination thereof (e.g., time sliced fetching and decoding and simultaneous multithreading thereafter such as in the Intel® Hyperthreading technology).).
and wherein the particular slices are selectively retrieved based on a multiplication result between numerical … refraining from retrieving the row slice upon determining that the multiplication result is the zero value (eg., 0058 - the specified sparse source matrix is stored in sparse format in memory 412, such that the execution circuit 411 reads the sparsely-formatted matrix, but only buffers the non-zero values in sparse buffers 414.; 0199 Fig. 2A - Example 1 provides an exemplary processor to execute a sparse-dense matrix multiplication instruction, comprising: fetch circuitry to fetch, from code storage, the sparse-dense matrix multiplication instruction having fields to specify an opcode, a dense output matrix, a dense source matrix, and a sparse source matrix having a sparsity of non-zero elements ; 0162 - multithreading (executing two or more parallel sets of operations or threads), and may do so in a variety of ways including time sliced multithreading, simultaneous multithreading (where a single physical core provides a logical core for each of the threads that physical core is simultaneously multithreading), or a combination thereof (e.g., time sliced fetching and decoding and simultaneous multithreading thereafter such as in the Intel® Hyperthreading technology).
It would have been obvious to one of ordinary skill in the art prior to the filing date of the claimed invention to modify the memory storage system having banks and sub-banks of Wai in view of Chatterjee, with Narayanamoorthy providing the benefit of Sparse-dense matrix multiplication (SDMM) operations are useful in a deep learning context. But traditional CPU and GPU instruction set architectures require symmetric inputs having the same density, which limits the ability to gain a performance advantage by taking advantage of the sparsity of a sparse input matrix (see Narayanamoorthy, 0004) executing a sparse-dense matrix multiplication (SDMM) instruction (0040).
Claims 17-18, have substantially similar limitations as stated in claims 2-3, respectively; therefore, they are rejected under the same subject matter.
Claims 6, 7, 8, 15 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Wai et al. (US 20220058126) in view of Chatterjee (US 20170255552) Narayanamoorthy (cited above) and further in view of Sade (US 20190102196).
Regarding claim 6, Wai in view of Chatterjee discloses the computing system of claim 1. However, neither Wai nor Chatterjee teaches wherein the request comprises a plurality of addresses for a corresponding plurality of tile and, for each tile, a corresponding slice mask specifying slices of each tile to be retrieved.
On the other hand, Sade discloses wherein the request comprises a plurality of addresses for a corresponding plurality of tile and, for each tile, a corresponding slice mask specifying slices of each tile to be retrieved. [0165] each J-element sub-column of the specified source matrix in row-major order into a K-wide submatrix of the specified destination matrix, the K-wide submatrix having K columns and enough rows to hold the J elements and ability to mask [0218] zeroing vector masks allow any set of elements in the destination to be zeroed during the execution of any operation (specified by the base operation and the augmentation operation); in one embodiment, an element of the destination is set to 0 when the corresponding mask bit has a 0 value. and [0227] with the elements that are actually transferred is dictated by the contents of the vector mask that is selected as the write mask.
Sade describes matrices and tile, and masks (writemask) which is analogous to “slice mask”. It would have been obvious to combine Wai with Schuetz for reasons set forth in claim 1. In addition, Sade, and Wai are considered to analogous arts, because they all relate to matrix manipulations in memories, therefore, it would have been obvious to combine Sade with the Wai-Schuetz combination for the benefit of using various masking techniques to isolate matrix elements, either row and/or line or non-zero matrix elements. Sade’s use of writemask advantageously select if a storage in memory occurs, which achieve performance gains for not storing when matrix elements are not needed. Sade also teaches operations that will zero desired rows and columns using vector masks.
Regarding claim 7, Wai in view of Chatterjee in further view of Sade teach the computing system of claim 6. Sade further teaches wherein the slice mask comprises a plurality of bits corresponding to the plurality of slices of each tile. Sade teaches tile and sub-columns (slices) [0165] each J-element sub-column of the specified source matrix in row-major order into a K-wide submatrix of the specified destination matrix, the K-wide submatrix having K columns and enough rows to hold the J elements and ability to mask [0218] zeroing vector masks allow any set of elements in the destination to be zeroed during the execution of any operation (specified by the base operation and the augmentation operation); in one embodiment, an element of the destination is set to 0 when the corresponding mask bit has a 0 value. and [0227] with the elements that are actually transferred is dictated by the contents of the vector mask that is selected as the write mask. Sade’s “J-element sub-column” is analogous to a slice in a tile, and Sade’s use of write mask in combination with sub-column yields the same capability as selecting a slice in a tile.
Regarding claim 8, Neither Wai nor Chatterjee appear to specifically teach wherein the memory banks are configured to store one or more whole tile using low address interleaving. On the other hand, Sade teaches wherein the memory banks are configured to store one or more whole tile using low address interleaving. [0163] row-interleaved (RowInt) formatted matrix . . . In particular, a RowInt-formatted matrix can be used when performing fused multiply-add (FMA) operations (or other arithmetic) to perform the FMA and then combine (add together) the results of adjacent operations and [0165] processor is to transform the specified source matrix into the specified destination matrix having a row-interleaved . . . format. In particular, in response to the opcode, the processor is to interleave elements of . . . sub-column of the specified source matrix . . . into a . . . submatrix of the specified destination matrix. It would have been obvious to combine Wai with Schuetz and Hung and Wang for the reasons set forth in claim 1 above. However, Sade, which also relates to the same field of endeavor, does disclose systems and methods for manipulating matrices with interleaved format, and the feature is desirable for fused multiply -add (FMA) operations or other arithmetic operations. There for it would be obvious by a person of ordinary skill in the art to combine Sade’s feature with Wai-Schuetz to the benefit of interleaving matrices and associated operations.
Claim 15, has substantially similar limitations as stated in claims 6 and 7, it is rejected under that same subject matter.
Claim 20 has substantially similar limitations as stated in claims 6, it is rejected under the same subject matter.
Response to Arguments
Applicant's arguments filed 6/26/2026 have been fully considered but they are not persuasive.
For Claims 1, 9, 16, the present OA rejects these amended limitations that Applicant argues.
Specifically, Wai does not disclose, but Chatterjee discloses
each comprising a plurality of slices in the memory banks, , wherein different slices corresponding to a same memory tile are stored in different sub-banks (eg., see Fig. 1B [0029] Each bank 120 is organized as a two-dimensional (2D) array of storage units such as the storage units 155-0A through 155-3A, 155-4A through 155-7A, 155-0B through 155-3B, and 155-3B through 155-7A. Details of the storage unit 155 are shown in FIG. 1C. In the context of the following description, a sub-channel 180 may include T storage units 155, where T is an integer value greater or equal to 1. As shown in FIG. 1B, the sub-channel 170-0 includes the four storage units 155-0A through 155-3A and the four storage units 155-0B through 155-3B. In one embodiment, the four storage units 155 within each of the storage units 155-0A through 155-3A and the storage units 155-0B through 155-3B are aligned horizontally and share a CA (column address) storage 150-0 and global SAs (sense amplifiers) 145-0. In other embodiments, more or less than four storage units 155 may be included in each of the storage units 155-0A through 155-3A and 155-0B through 155-3B. Similarly, the sub-channel 170-1 includes the storage units 155-4A through 155-7A and storage units 155-4B through 155-7B. In one embodiment, the four storage units 155 within each of the storage units 155-4A through 155-7A and the storage units 155-3B through 155-7B may each include four storage units 155 that are aligned horizontally and share a CA storage 150-1 and global SAs 145-1. In other embodiments, more or less than four storage units 155 may be included in each of the storage units 155-4A through 155-7A and 155-4B through 155-7B.)
corresponding slice positions of the plurality of slices of the memory tiles are stored in the same sub-banks (eg., 0020 Fig. 1B - the subarray 180-A includes a subarray row address logic 115-A, the storage units 155-0A through 155-3A, 155-4A through 155-7A and the subarray 180-B includes a subarray row address logic 115-B, the storage units 155-0B through 155-3B and 155-4B through 155-7B.; 0028-0029 FIG. 1B, includes two sub-channels, a sub-channel 170-0 and a sub-channel 170-1; disclosed sub-channels render claimed sub-banks as obvious and disclosed subarray row address 115-A and 115-B render claimed slices as obvious; disclosed different storage units 155-4A and 155-4B are in the same sub-channels );
and wherein the plurality of slices include one or more row slices and one or more corresponding column slices (eg., 0005 - first sub-array is associated with a first row address and the first sub-channel is associated with a first column address that is stored in the first sub-channel and used to select a first portion of data from the first row-buffer)
wherein a request to retrieve particular slices from two or more particular memory tiles sequentially moves between the plurality of memory banks to retrieve a predetermined amount of data (eg., 0051 - a single burst access, CSLs should be asserted sequentially in a sub-channel 170. To generate the sequence of CSLs,)
corresponding to different memory tiles from a same sub-bank (eg., [0033] Fig. 1B - The I/O buffer 160-0 is included in the sub-channel 170-0 and the I/O buffer 160-1 is included in the sub-channel 170-1. The I/O buffer 160-0 is coupled to the storage units 155-0A through 155-3A and the storage units 155-0B through 155-3B through the global sense amplifiers 145-0. The I/O buffer 160-1 is coupled to the storage units 155-4A through 155-7A and the storage units 155-3B through 155-7B; 0021 - multiple sub-channels can be operated in parallel to return different access requests simultaneously- this disclosure renders obvious that I/O buffer 160-1 is configured to retrieve from storage units 155-4B and 155-4A)
It would have been obvious to one of ordinary skill in the art prior to the filing date of the claimed invention to modify the memory storage system having banks and sub-banks of Wai in view of Chatterjee, providing the benefit of multiple sub-channels can be operated in parallel to return different access requests simultaneously (see Chatterjee, 0021) A DRAM sub-channel architecture with improved energy efficiency (0020) to reduce the minimum row-activation granularity and the DRAM datapath is divided while avoiding significant increases in the die area (0021).
Wai in view of Chatterjee does not disclose, but Narayanamoorthy discloses
the plurality of slices comprising a plurality of numerical data values (eg., [0046] the specified sparse source matrix 212 is logically a M by K matrix, but only its non-zero elements are stored in memory in a compressed sparse row (CSR) or compressed sparse column (CSC) format,)
wherein the request retrieves two or more slices in a single cycle (eg., [0162] core may support multithreading (executing two or more parallel sets of operations or threads), and may do so in a variety of ways including time sliced multithreading, simultaneous multithreading (where a single physical core provides a logical core for each of the threads that physical core is simultaneously multithreading), or a combination thereof (e.g., time sliced fetching and decoding and simultaneous multithreading thereafter such as in the Intel® Hyperthreading technology).).
and wherein the particular slices are selectively retrieved based on a multiplication result between numerical … refraining from retrieving the row slice upon determining that the multiplication result is the zero value (eg., 0058 - the specified sparse source matrix is stored in sparse format in memory 412, such that the execution circuit 411 reads the sparsely-formatted matrix, but only buffers the non-zero values in sparse buffers 414.; 0199 Fig. 2A - Example 1 provides an exemplary processor to execute a sparse-dense matrix multiplication instruction, comprising: fetch circuitry to fetch, from code storage, the sparse-dense matrix multiplication instruction having fields to specify an opcode, a dense output matrix, a dense source matrix, and a sparse source matrix having a sparsity of non-zero elements ; 0162 - multithreading (executing two or more parallel sets of operations or threads), and may do so in a variety of ways including time sliced multithreading, simultaneous multithreading (where a single physical core provides a logical core for each of the threads that physical core is simultaneously multithreading), or a combination thereof (e.g., time sliced fetching and decoding and simultaneous multithreading thereafter such as in the Intel® Hyperthreading technology).
It would have been obvious to one of ordinary skill in the art prior to the filing date of the claimed invention to modify the memory storage system having banks and sub-banks of Wai in view of Chatterjee, with Narayanamoorthy providing the benefit of Sparse-dense matrix multiplication (SDMM) operations are useful in a deep learning context. But traditional CPU and GPU instruction set architectures require symmetric inputs having the same density, which limits the ability to gain a performance advantage by taking advantage of the sparsity of a sparse input matrix (see Narayanamoorthy, 0004) executing a sparse-dense matrix multiplication (SDMM) instruction (0040).
Applicant’s arguments for claims 9 and 16 are based on Applicant’s admission of similarity to Claim 1.
Applicant’s arguments for dependent claims2, 3, 5-8, 10, 11, 13-15, 17, 18 and 20 are based on dependency from base claims, addressed above.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GAUTAM SAIN whose telephone number is (571)270-3555. The examiner can normally be reached M-F 9-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jared Rutz can be reached at 571-272-5535. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GAUTAM SAIN/Primary Examiner, Art Unit 2135
