Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The office action is responding to the arguments filed on 06/16/2026. Claims 1-3, 5-10, 12-17 and 19-20 are pending.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-2, 6-9, 12-16 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Korzh et al. (US 20210382821 A1) in view of HOLM et al. (US 20150134933 A1) and further in view of Minkin et al. (US 20230289292 A1) hereinafter Korzh and HOLM and Minkin.
Regarding claim 1, Korzh teaches A system comprising: a memory device; a cache memory; and (See Fig 1, paragraph [0060], illustrates a memory system 14 having memory device 18, a cache memory 24)
a processing device, operatively coupled with the memory device and the cache memory, to perform operations comprising: (See Fig 1, paragraph [0063], illustrates processing system 12 is coupled with memory system 14 via communication bus 20 to perform operations)
receiving, from a host system, a plurality of memory access requests associated with a plurality of processing threads executed by a plurality of processing cores on the host system; (See Fig 1, paragraph [0064], illustrates memory system 14 may receive write and read memory access requests from processing system 12)
identifying the plurality of processing threads with which the plurality of memory access requests are associated; (See Fig 4, paragraph [0087], illustrates metadata 58 may include a process identifier and a thread identifier to indicate processor thread running on computing system 10 (Fig1))
tracking respective numbers of the plurality of memory access requests that are associated with each of the plurality processing threads in a given period of time; (See Fig 6, paragraph [0103], illustrates data access information 34A may have access count column 66 along with last access column 64 indicating number of times data has been previously access over a time period)
Korzh teaches prefetching or preloading read data based on memory access pattern. However, Korzh does not explicitly teach selecting, based on the tracking, a subset of the plurality of processing threads; and
prefetching data associated with the subset of the plurality of processing threads from the memory device and storing the data in the cache memory
On the other hand, HOLM which also relates to prefetching or preloading read data based on memory access pattern teaches selecting, based on the tracking, a subset of the plurality of processing threads; and (See Fig 1, paragraph [0041], illustrates prefetch unit 19 monitors access requests received and generate prefetch transactions based on access patterns)
prefetching data associated with the subset of the plurality of processing threads from the memory device and storing the data in the cache memory. (See Fig 1, paragraph [0041], illustrates data is prefetched or prepopulated in cache line 20 based on access pattern monitored by prefetch unit 19)
Both Korzh and HOLM relate to prefetching or preloading read data based on memory access pattern (see Korzh, abstract, and see HOLM, abstract, regarding prefetching or preloading read data).
Therefore, it would have been obvious to one of ordinary skill at the time the
invention was effectively filed to combine Korzh with HOLM by incorporating prefetching or preloading read data based on memory access pattern, as taught by
MIZUNO, to enable prefetch unit 19 to monitor access requests received and generate prefetch transactions based on access patterns and data to be prefetched or prepopulated in cache line 20 based on access pattern monitored by prefetch unit 19. The combined system of Korzh – HOLM allows prefetching of a given data value which is predicted to be required by the instruction execution unit is initiated further in advance of its actually being required by the instruction execution unit as mentioned in paragraph [0010]. Therefore, the combination of Korzh - HOLM improves data processing performance. See HOLM, paragraph [0011].
Korzh in view of HOLM teaches prefetching or preloading read data based on memory access pattern above. However, Korzh - HOLM combination does not explicitly teach wherein the tracking comprises determining respective submission queue identifiers (SQIDs) for the plurality of memory requests and
incrementing respective counters associated with the respective SQIDs;
On the other hand, Minkin which also relates to prefetching or preloading read data based on memory access pattern teaches wherein the tracking comprises determining respective submission queue identifiers (SQIDs) for the plurality of memory requests and (See Fig 6, paragraph [0086], illustrates request completion tracking circuitry 618 tracks the memory requests from request queue 606 for the sub-blocks and responses received from the memory subsystem)
incrementing respective counters associated with the respective SQIDs; (See Fig 6 and 7B, paragraph [0095], illustrates TMAU have a dedicated counter that keeps track of issued L2 requests and every time request is sent to L2 cache the counter is incremented)
Both Korzh and HOLM and Minkin relate to prefetching or preloading read data based on memory access pattern (see Korzh, abstract, and see HOLM, abstract, and see Minkin, abstract, regarding prefetching or preloading read data).
Therefore, it would have been obvious to one of ordinary skill at the time the
invention was effectively filed to combine Korzh - HOLM combination with Minkin by incorporating prefetching or preloading read data based on memory access pattern, as taught by Minkin, to enable request completion tracking circuitry 618 tracking the memory requests from request queue 606 for the sub-blocks and responses received from the memory subsystem and TMAU to have a dedicated counter that keeps track of issued L2 requests and every time request is sent to L2 cache the counter is incremented. The combined system of Korzh – HOLM - Minkin allows each parallel processor core to be closely coupled to a tensor memory access unit (TMAU) hardware circuitry for moving large data blocks between the shared memory of the parallel processor core and external memory such as, for example, global memory of the parallel processing system as mentioned in paragraph [0038]. Therefore, the combination of Korzh - HOLM - Minkin improves processing efficiency. See Minkin, paragraph [0043].
Regarding claim 2, Korzh in view of HOLM and further in view of Minkin teaches prefetching or preloading read data based on memory access pattern in claim 1. However, Korzh - HOLM - Minkin combination does not explicitly teach The system of claim 1, wherein the plurality of memory access requests comprise requests to read training data from the memory device for at least one of a machine learning (ML) model or an artificial intelligence (Al) framework
On the other hand, Korzh which also relates to prefetching or preloading read data based on memory access pattern teaches The system of claim 1, wherein the plurality of memory access requests comprise requests to read training data from the memory device for at least one of a machine learning (ML) model or an artificial intelligence (Al) framework. (See Fig 1, paragraph [0070], illustrates machine learning block 32 may implement machine learning techniques to facilitate predicting a data access pattern for access requests based at least in part on data access information 34 indicative of a previous data access pattern)
The same motivation that was utilized for combining Korzh - HOLM combination and Minkin as set forth in claim 1 is equally applicable to claim 2.
Regarding claim 6, Korzh in view of HOLM and further in view of Minkin teaches prefetching or preloading read data based on memory access pattern in claim 1. However, Korzh - HOLM - Minkin combination does not explicitly teach The system of claim 1, wherein prefetching the data associated with the subset of the plurality of processing threads from the memory device and storing the data in the cache memory comprises:
subsequent to receiving a request for data at a first memory address in the memory device, retrieving data at a second memory address in the memory device prior to receiving a request for the data at the second memory address
and storing the data at the second memory address in the cache memory, wherein the second memory address is sequential to the first memory address
On the other hand, HOLM which also relates to prefetching or preloading read data based on memory access pattern teaches The system of claim 1, wherein prefetching the data associated with the subset of the plurality of processing threads from the memory device and storing the data in the cache memory comprises:
subsequent to receiving a request for data at a first memory address in the memory device, retrieving data at a second memory address in the memory device prior to receiving a request for the data at the second memory address (See Fig 3, paragraph [0047], illustrates prefetch unit 19 prefetches addresses for a given memory access request for prefetching or retrieving data from memory to cache unit)
and storing the data at the second memory address in the cache memory, wherein the second memory address is sequential to the first memory address. (See Fig 3, paragraph [0047], illustrates prefetches addresses for a given entry in the prefetch table 21 will typically be well within the size of a memory page meaning that the prefetch unit 19 can sequentially issue prefetch transactions for sequential physical addresses)
Both Korzh and HOLM and Minkin relate to prefetching or preloading read data based on memory access pattern (see Korzh, abstract, and see HOLM, abstract, and see Minkin, abstract, regarding prefetching or preloading read data).
Therefore, it would have been obvious to one of ordinary skill at the time the
invention was effectively filed to combine Korzh - HOLM combination with Minkin by incorporating prefetching or preloading read data based on memory access pattern, as taught by HOLM, to enable prefetch unit 19 prefetching addresses for a given memory access request for prefetching or retrieving data from memory to cache unit and prefetching addresses for a given entry in the prefetch table 21 will typically be well within the size of a memory page meaning that the prefetch unit 19 can sequentially issue prefetch transactions for sequential physical addresses. The combined system of Korzh – HOLM - Minkin allows prefetching of a given data value which is predicted to be required by the instruction execution unit is initiated further in advance of its actually being required by the instruction execution unit as mentioned in paragraph [0010]. Therefore, the combination of Korzh - HOLM - Minkin improves data processing performance. See HOLM, paragraph [0011].
Regarding claim 7, Korzh in view of HOLM and further in view of Minkin teaches prefetching or preloading read data based on memory access pattern in claim 6. However, Korzh - HOLM - Minkin combination does not explicitly teach The system of claim 6 wherein the processing device is to perform operations further comprising: receiving, from the host system, a memory access request for the data at the second memory address; and
On the other hand, HOLM which also relates to prefetching or preloading read data based on memory access pattern teaches The system of claim 6 wherein the processing device is to perform operations further comprising: receiving, from the host system, a memory access request for the data at the second memory address; and (See Fig 1, paragraph [0041] and [0042], illustrates prefetch unit 19 receives memory access requests issued by the processor cores 11, 12 and populate cache line 20 before data is transferred)
providing the data at the second memory address to the host system from the cache memory. (See Fig 1, paragraph [0041], illustrates data is transferred from cache 20 once the data is prepopulated and retrieved from memory 18)
The same motivation that was utilized for combining Korzh - HOLM combination and Minkin as set forth in claim 6 is equally applicable to claim 7.
Regarding claim 8, Korzh teaches A method comprising: (See claim 10)
receiving, from a host system, a plurality of memory access requests associated with a plurality of processing threads executed by a plurality of processing cores on the host system; (See Fig 1, paragraph [0064], illustrates memory system 14 may receive write and read memory access requests from processing system 12)
identifying the plurality of processing threads with which the plurality of memory access requests are associated; (See Fig 4, paragraph [0087], illustrates metadata 58 may include a process identifier and a thread identifier to indicate processor thread running on computing system 10 (Fig1))
tracking respective numbers of the plurality of memory access requests that are associated with each of the plurality processing threads in a given period of time; (See Fig 6, paragraph [0103], illustrates data access information 34A may have access count column 66 along with last access column 64 indicating number of times data has been previously access over a time period)
Korzh teaches prefetching or preloading read data based on memory access pattern. However, Korzh does not explicitly teach selecting, based on the tracking, a subset of the plurality of processing threads; and
prefetching data associated with the subset of the plurality of processing threads from a memory device and storing the data in a cache memory
On the other hand, HOLM which also relates to prefetching or preloading read data based on memory access pattern teaches selecting, based on the tracking, a subset of the plurality of processing threads; and (See Fig 1, paragraph [0041], illustrates prefetch unit 19 monitors access requests received and generate prefetch transactions based on access patterns)
prefetching data associated with the subset of the plurality of processing threads from a memory device and storing the data in a cache memory. (See Fig 1, paragraph [0041], illustrates data is prefetched or prepopulated in cache line 20 based on access pattern monitored by prefetch unit 19)
Both Korzh and HOLM relate to prefetching or preloading read data based on memory access pattern (see Korzh, abstract, and see HOLM, abstract, regarding prefetching or preloading read data).
Therefore, it would have been obvious to one of ordinary skill at the time the
invention was effectively filed to combine Korzh with HOLM by incorporating prefetching or preloading read data based on memory access pattern, as taught by
MIZUNO, to enable prefetch unit 19 to monitor access requests received and generate prefetch transactions based on access patterns and data to be prefetched or prepopulated in cache line 20 based on access pattern monitored by prefetch unit 19. The combined system of Korzh – HOLM allows prefetching of a given data value which is predicted to be required by the instruction execution unit is initiated further in advance of its actually being required by the instruction execution unit as mentioned in paragraph [0010]. Therefore, the combination of Korzh - HOLM improves data processing performance. See MIZUNO, paragraph [0011].
Korzh in view of HOLM teaches prefetching or preloading read data based on memory access pattern above. However, Korzh - HOLM combination does not explicitly teach wherein the tracking comprises determining respective submission queue identifiers (SQIDs) for the plurality of memory requests and
incrementing respective counters associated with the respective SQIDs;
On the other hand, Minkin which also relates to prefetching or preloading read data based on memory access pattern teaches wherein the tracking comprises determining respective submission queue identifiers (SQIDs) for the plurality of memory requests and (See Fig 6, paragraph [0086], illustrates request completion tracking circuitry 618 tracks the memory requests from request queue 606 for the sub-blocks and responses received from the memory subsystem)
incrementing respective counters associated with the respective SQIDs; (See Fig 6 and 7B, paragraph [0095], illustrates TMAU have a dedicated counter that keeps track of issued L2 requests and every time request is sent to L2 cache the counter is incremented)
Both Korzh and HOLM and Minkin relate to prefetching or preloading read data based on memory access pattern (see Korzh, abstract, and see HOLM, abstract, and see Minkin, abstract, regarding prefetching or preloading read data).
Therefore, it would have been obvious to one of ordinary skill at the time the
invention was effectively filed to combine Korzh - HOLM combination with Minkin by incorporating prefetching or preloading read data based on memory access pattern, as taught by Minkin, to enable request completion tracking circuitry 618 tracking the memory requests from request queue 606 for the sub-blocks and responses received from the memory subsystem and TMAU to have a dedicated counter that keeps track of issued L2 requests and every time request is sent to L2 cache the counter is incremented. The combined system of Korzh – HOLM - Minkin allows each parallel processor core to be closely coupled to a tensor memory access unit (TMAU) hardware circuitry for moving large data blocks between the shared memory of the parallel processor core and external memory such as, for example, global memory of the parallel processing system as mentioned in paragraph [0038]. Therefore, the combination of Korzh - HOLM - Minkin improves processing efficiency. See Minkin, paragraph [0043].
Regarding claim 9, Korzh in view of HOLM and further in view of Minkin teaches prefetching or preloading read data based on memory access pattern in claim 8. However, Korzh - HOLM - Minkin combination does not explicitly teach The method of claim 8, wherein the plurality of memory access requests comprise requests to read training data from the memory device for at least one of a machine learning (ML) model or an artificial intelligence (AI) framework
On the other hand, Korzh which also relates to prefetching or preloading read data based on memory access pattern teaches The method of claim 8, wherein the plurality of memory access requests comprise requests to read training data from the memory device for at least one of a machine learning (ML) model or an artificial intelligence (AI) framework. (See Fig 1, paragraph [0070], illustrates machine learning block 32 may implement machine learning techniques to facilitate predicting a data access pattern for access requests based at least in part on data access information 34 indicative of a previous data access pattern)
The same motivation that was utilized for combining Korzh - HOLM combination and Minkin as set forth in claim 8 is equally applicable to claim 9.
Regarding claim 13, Korzh in view of HOLM and further in view of Minkin teaches prefetching or preloading read data based on memory access pattern in claim 8. However, Korzh - HOLM - Minkin combination does not explicitly teach The method of claim 8, wherein prefetching the data associated with the subset of the plurality of processing threads from the memory device and storing the data in the cache memory comprises:
subsequent to receiving a request for data at a first memory address in the memory device, retrieving data at a second memory address in the memory device prior to receiving a request for the data at the second memory address
and storing the data at the second memory address in the cache memory, wherein the second memory address is sequential to the first memory address
On the other hand, HOLM which also relates to prefetching or preloading read data based on memory access pattern teaches The method of claim 8, wherein prefetching the data associated with the subset of the plurality of processing threads from the memory device and storing the data in the cache memory comprises:
subsequent to receiving a request for data at a first memory address in the memory device, retrieving data at a second memory address in the memory device prior to receiving a request for the data at the second memory address (See Fig 3, paragraph [0047], illustrates prefetch unit 19 prefetches addresses for a given memory access request for prefetching or retrieving data from memory to cache unit)
and storing the data at the second memory address in the cache memory, wherein the second memory address is sequential to the first memory address. (See Fig 3, paragraph [0047], illustrates prefetches addresses for a given entry in the prefetch table 21 will typically be well within the size of a memory page meaning that the prefetch unit 19 can sequentially issue prefetch transactions for sequential physical addresses)
Both Korzh and HOLM and Minkin relate to prefetching or preloading read data based on memory access pattern (see Korzh, abstract, and see HOLM, abstract, and see Minkin, abstract, regarding prefetching or preloading read data).
Therefore, it would have been obvious to one of ordinary skill at the time the
invention was effectively filed to combine Korzh - HOLM combination with Minkin by incorporating prefetching or preloading read data based on memory access pattern, as taught by HOLM, to enable prefetch unit 19 prefetching addresses for a given memory access request for prefetching or retrieving data from memory to cache unit and prefetching addresses for a given entry in the prefetch table 21 will typically be well within the size of a memory page meaning that the prefetch unit 19 can sequentially issue prefetch transactions for sequential physical addresses. The combined system of Korzh – HOLM - Minkin allows prefetching of a given data value which is predicted to be required by the instruction execution unit is initiated further in advance of its actually being required by the instruction execution unit as mentioned in paragraph [0010]. Therefore, the combination of Korzh - HOLM - Minkin improves data processing performance. See HOLM, paragraph [0011].
Regarding claim 14, Korzh in view of HOLM and further in view of Minkin teaches prefetching or preloading read data based on memory access pattern in claim 13. However, Korzh - HOLM - Minkin combination does not explicitly teach The method of claim 13, further comprising: receiving, from the host system, a memory access request for the data at the second memory address; and
providing the data at the second memory address to the host system from the cache memory
On the other hand, HOLM which also relates to prefetching or preloading read data based on memory access pattern teaches The method of claim 13, further comprising: receiving, from the host system, a memory access request for the data at the second memory address; and (See Fig 1, paragraph [0041] and [0042], illustrates prefetch unit 19 receives memory access requests issued by the processor cores 11, 12 and populate cache line 20 before data is transferred)
providing the data at the second memory address to the host system from the cache memory. (See Fig 1, paragraph [0041], illustrates data is transferred from cache 20 once the data is prepopulated and retrieved from memory 18)
The same motivation that was utilized for combining Korzh - HOLM combination and Minkin as set forth in claim 13 is equally applicable to claim 14.
Regarding claim 15, Korzh teaches A non-transitory computer-readable storage medium comprising instructions that, when executed by a processing device, cause the processing device to perform operations comprising: (See Fig 1, paragraph [0062], illustrates processing sub-system performs various operations like executing instructions to perform a corresponding data processing operation on input data)
receiving, from a host system, a plurality of memory access requests associated with a plurality of processing threads executed by a plurality of processing cores on the host system; (See Fig 1, paragraph [0064], illustrates memory system 14 may receive write and read memory access requests from processing system 12)
identifying the plurality of processing threads with which the plurality of memory access requests are associated; (See Fig 4, paragraph [0087], illustrates metadata 58 may include a process identifier and a thread identifier to indicate processor thread running on computing system 10 (Fig1))
tracking respective numbers of the plurality of memory access requests that are associated with each of the plurality processing threads in a given period of time; (See Fig 6, paragraph [0103], illustrates data access information 34A may have access count column 66 along with last access column 64 indicating number of times data has been previously access over a time period)
Korzh teaches prefetching or preloading read data based on memory access pattern. However, Korzh does not explicitly teach selecting, based on the tracking, a subset of the plurality of processing threads; and
prefetching data associated with the subset of the plurality of processing threads from a memory device and storing the data in a cache memory
On the other hand, HOLM which also relates to prefetching or preloading read data based on memory access pattern teaches selecting, based on the tracking, a subset of the plurality of processing threads; and (See Fig 1, paragraph [0041], illustrates prefetch unit 19 monitors access requests received and generate prefetch transactions based on access patterns)
prefetching data associated with the subset of the plurality of processing threads from a memory device and storing the data in a cache memory. (See Fig 1, paragraph [0041], illustrates data is prefetched or prepopulated in cache line 20 based on access pattern monitored by prefetch unit 19)
Both Korzh and HOLM relate to prefetching or preloading read data based on memory access pattern (see Korzh, abstract, and see HOLM, abstract, regarding prefetching or preloading read data).
Therefore, it would have been obvious to one of ordinary skill at the time the
invention was effectively filed to combine Korzh with HOLM by incorporating prefetching or preloading read data based on memory access pattern, as taught by
MIZUNO, to enable prefetch unit 19 to monitor access requests received and generate prefetch transactions based on access patterns and data to be prefetched or prepopulated in cache line 20 based on access pattern monitored by prefetch unit 19. The combined system of Korzh – HOLM allows prefetching of a given data value which is predicted to be required by the instruction execution unit is initiated further in advance of its actually being required by the instruction execution unit as mentioned in paragraph [0010]. Therefore, the combination of Korzh - HOLM improves data processing performance. See MIZUNO, paragraph [0011].
Korzh in view of HOLM teaches prefetching or preloading read data based on memory access pattern above. However, Korzh - HOLM combination does not explicitly teach wherein the tracking comprises determining respective submission queue identifiers (SQIDs) for the plurality of memory requests and
incrementing respective counters associated with the respective SQIDs;
On the other hand, Minkin which also relates to prefetching or preloading read data based on memory access pattern teaches wherein the tracking comprises determining respective submission queue identifiers (SQIDs) for the plurality of memory requests and (See Fig 6, paragraph [0086], illustrates request completion tracking circuitry 618 tracks the memory requests from request queue 606 for the sub-blocks and responses received from the memory subsystem)
incrementing respective counters associated with the respective SQIDs; (See Fig 6 and 7B, paragraph [0095], illustrates TMAU have a dedicated counter that keeps track of issued L2 requests and every time request is sent to L2 cache the counter is incremented)
Both Korzh and HOLM and Minkin relate to prefetching or preloading read data based on memory access pattern (see Korzh, abstract, and see HOLM, abstract, and see Minkin, abstract, regarding prefetching or preloading read data).
Therefore, it would have been obvious to one of ordinary skill at the time the
invention was effectively filed to combine Korzh - HOLM combination with Minkin by incorporating prefetching or preloading read data based on memory access pattern, as taught by Minkin, to enable request completion tracking circuitry 618 tracking the memory requests from request queue 606 for the sub-blocks and responses received from the memory subsystem and TMAU to have a dedicated counter that keeps track of issued L2 requests and every time request is sent to L2 cache the counter is incremented. The combined system of Korzh – HOLM - Minkin allows each parallel processor core to be closely coupled to a tensor memory access unit (TMAU) hardware circuitry for moving large data blocks between the shared memory of the parallel processor core and external memory such as, for example, global memory of the parallel processing system as mentioned in paragraph [0038]. Therefore, the combination of Korzh - HOLM - Minkin improves processing efficiency. See Minkin, paragraph [0043].
Regarding claim 16 Korzh in view of HOLM and further in view of Minkin teaches prefetching or preloading read data based on memory access pattern in claim 15. However, Korzh - HOLM - Minkin combination does not explicitly teach The non-transitory computer-readable storage medium of claim 15, wherein the plurality of memory access requests comprise requests to read training data from the memory device for at least one of a machine learning (ML) model or an artificial intelligence (AI) framework
On the other hand, Korzh which also relates to prefetching or preloading read data based on memory access pattern teaches The non-transitory computer-readable storage medium of claim 15, wherein the plurality of memory access requests comprise requests to read training data from the memory device for at least one of a machine learning (ML) model or an artificial intelligence (AI) framework. (See Fig 1, paragraph [0070], illustrates machine learning block 32 may implement machine learning techniques to facilitate predicting a data access pattern for access requests based at least in part on data access information 34 indicative of a previous data access pattern)
The same motivation that was utilized for combining Korzh - HOLM combination and Minkin as set forth in claim 15 is equally applicable to claim 16.
Regarding claim 20, Korzh in view of HOLM and further in view of Minkin teaches prefetching or preloading read data based on memory access pattern in claim 15. However, Korzh - HOLM - Minkin combination does not explicitly teach The non-transitory computer-readable storage medium of claim 15, wherein prefetching the data associated with the subset of the plurality of processing threads from the memory device and storing the data in the cache memory comprises:
subsequent to receiving a request for data at a first memory address in the memory device, retrieving data at a second memory address in the memory device prior to receiving a request for the data at the second memory add
and storing the data at the second memory address in the cache memory, wherein the second memory address is sequential to the first memory address
On the other hand, HOLM which also relates to prefetching or preloading read data based on memory access pattern teaches The non-transitory computer-readable storage medium of claim 15, wherein prefetching the data associated with the subset of the plurality of processing threads from the memory device and storing the data in the cache memory comprises:
subsequent to receiving a request for data at a first memory address in the memory device, retrieving data at a second memory address in the memory device prior to receiving a request for the data at the second memory address (See Fig 3, paragraph [0047], illustrates prefetch unit 19 prefetches addresses for a given memory access request for prefetching or retrieving data from memory to cache unit)
and storing the data at the second memory address in the cache memory, wherein the second memory address is sequential to the first memory address. (See Fig 3, paragraph [0047], illustrates prefetches addresses for a given entry in the prefetch table 21 will typically be well within the size of a memory page meaning that the prefetch unit 19 can sequentially issue prefetch transactions for sequential physical addresses)
Both Korzh and HOLM and Minkin relate to prefetching or preloading read data based on memory access pattern (see Korzh, abstract, and see HOLM, abstract, and see Minkin, abstract, regarding prefetching or preloading read data).
Therefore, it would have been obvious to one of ordinary skill at the time the
invention was effectively filed to combine Korzh - HOLM combination with Minkin by incorporating prefetching or preloading read data based on memory access pattern, as taught by HOLM, to enable prefetch unit 19 prefetching addresses for a given memory access request for prefetching or retrieving data from memory to cache unit and prefetching addresses for a given entry in the prefetch table 21 will typically be well within the size of a memory page meaning that the prefetch unit 19 can sequentially issue prefetch transactions for sequential physical addresses. The combined system of Korzh – HOLM - Minkin allows prefetching of a given data value which is predicted to be required by the instruction execution unit is initiated further in advance of its actually being required by the instruction execution unit as mentioned in paragraph [0010]. Therefore, the combination of Korzh - HOLM - Minkin improves data processing performance. See HOLM, paragraph [0011].
Claim(s) 3, 10 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Korzh in view of HOLM and further in view of Minkin and further in view of Vemulapalli et al. (US 20210255957 A1) hereinafter Vemulapalli.
Regarding claim 3, Korzh in view of HOLM and further in view of Minkin teaches prefetching or preloading read data based on memory access pattern in claim 1. However, Korzh - HOLM - Minkin combination does not explicitly teach The system of claim 1, wherein each of the plurality of processing threads is executed by a respective one of the plurality of processing cores and comprises a plurality of sequential memory access requests
On the other hand, Vemulapalli which also relates to prefetching or preloading read data based on memory access pattern teaches The system of claim 1, wherein each of the plurality of processing threads is executed by a respective one of the plurality of processing cores and comprises a plurality of sequential memory access requests. (See Fig 2A-C, paragraph [0077] and [0079], illustrates pipeline manager 232 receives instructions from the scheduler 210 and manages execution of the instructions or group of threads across the set of parallel processing engines in consecutive clock cycles or sequentially)
Both Korzh, HOLM, Minkin and Vemulapalli relate to prefetching or preloading read data based on memory access pattern (see Korzh, abstract, and see HOLM, abstract, and see Minkin, abstract, and see Vemulapalli, abstract, regarding prefetching or preloading read data).
Therefore, it would have been obvious to one of ordinary skill at the time the
invention was effectively filed to combine Korzh - HOLM - Minkin combination and Vemulapalli by incorporating prefetching or preloading read data based on memory access pattern, as taught by Vemulapalli, to enable pipeline manager 232 to receive instructions from the scheduler 210 and manages execution of the instructions or group of threads across the set of parallel processing engines in consecutive clock cycles or sequentially. The combined system of Korzh – HOLM - Minkin - Vemulapalli allows a system or process to provide improvements in data prefetching for graphics data processing as mentioned in paragraph [0045]. Therefore, the combination of Korzh - HOLM - Minkin - Vemulapalli improves training speed for particularly deep neural networks. See Vemulapalli, paragraph [0183].
Regarding claim 10, Korzh in view of HOLM and further in view of Minkin teaches prefetching or preloading read data based on memory access pattern in claim 8. However, Korzh - HOLM - Minkin combination does not explicitly teach The method of claim 8, wherein each of the plurality of processing threads is executed by a respective one of the plurality of processing cores and comprises a plurality of sequential memory access requests
On the other hand, Vemulapalli which also relates to prefetching or preloading read data based on memory access pattern teaches The method of claim 8, wherein each of the plurality of processing threads is executed by a respective one of the plurality of processing cores and comprises a plurality of sequential memory access requests. (See Fig 2A-C, paragraph [0077] and [0079], illustrates pipeline manager 232 receives instructions from the scheduler 210 and manages execution of the instructions or group of threads across the set of parallel processing engines in consecutive clock cycles or sequentially)
Both Korzh, HOLM, Minkin and Vemulapalli relate to prefetching or preloading read data based on memory access pattern (see Korzh, abstract, and see HOLM, abstract, and see Minkin, abstract, and see Vemulapalli, abstract, regarding prefetching or preloading read data).
Therefore, it would have been obvious to one of ordinary skill at the time the
invention was effectively filed to combine Korzh - HOLM - Minkin combination and Vemulapalli by incorporating prefetching or preloading read data based on memory access pattern, as taught by Vemulapalli, to enable pipeline manager 232 to receive instructions from the scheduler 210 and manages execution of the instructions or group of threads across the set of parallel processing engines in consecutive clock cycles or sequentially. The combined system of Korzh – HOLM - Minkin - Vemulapalli allows a system or process to provide improvements in data prefetching for graphics data processing as mentioned in paragraph [0045]. Therefore, the combination of Korzh - HOLM - Minkin - Vemulapalli improves training speed for particularly deep neural networks. See Vemulapalli, paragraph [0183].
Regarding claim 17, Korzh in view of HOLM and further in view of Minkin teaches prefetching or preloading read data based on memory access pattern in claim 15. However, Korzh - HOLM - Minkin combination does not explicitly teach The non-transitory computer-readable storage medium of claim 15, wherein each of the plurality of processing threads is executed by a respective one of the plurality of processing cores and comprises a plurality of sequential memory access requests
On the other hand, Vemulapalli which also relates to prefetching or preloading read data based on memory access pattern teaches The non-transitory computer-readable storage medium of claim 15, wherein each of the plurality of processing threads is executed by a respective one of the plurality of processing cores and comprises a plurality of sequential memory access requests. (See Fig 2A-C, paragraph [0077] and [0079], illustrates pipeline manager 232 receives instructions from the scheduler 210 and manages execution of the instructions or group of threads across the set of parallel processing engines in consecutive clock cycles or sequentially)
Both Korzh, HOLM, Minkin and Vemulapalli relate to prefetching or preloading read data based on memory access pattern (see Korzh, abstract, and see HOLM, abstract, and see Minkin, abstract, and see Vemulapalli, abstract, regarding prefetching or preloading read data).
Therefore, it would have been obvious to one of ordinary skill at the time the
invention was effectively filed to combine Korzh - HOLM - Minkin combination and Vemulapalli by incorporating prefetching or preloading read data based on memory access pattern, as taught by Vemulapalli, to enable pipeline manager 232 to receive instructions from the scheduler 210 and manages execution of the instructions or group of threads across the set of parallel processing engines in consecutive clock cycles or sequentially. The combined system of Korzh – HOLM - Minkin - Vemulapalli allows a system or process to provide improvements in data prefetching for graphics data processing as mentioned in paragraph [0045]. Therefore, the combination of Korzh - HOLM - Minkin - Vemulapalli improves training speed for particularly deep neural networks. See Vemulapalli, paragraph [0183].
Claim(s) 5, 12 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Korzh in view of HOLM and further in view of Minkin and further in view of Nubile et al. (US 20220083241 A1) hereinafter Nubile.
Regarding claim 5, Korzh in view of HOLM and further in view of Minkin teaches prefetching or preloading read data based on memory access pattern in claim 1. However, Korzh - HOLM - Minkin combination does not explicitly teach The system of claim 1, wherein selecting the subset of the plurality of processing threads comprises selecting a number of processing threads that have issued the highest number of memory access requests in the given period of time
On the other hand, Nubile which also relates to prefetching or preloading read data based on memory access pattern teaches The system of claim 1, wherein selecting the subset of the plurality of processing threads comprises selecting a number of processing threads that have issued the highest number of memory access requests in the given period of time. (See Fig 4A-B, paragraph [0053], illustrates data structure 450 can identify a leading thread and the leading thread can be a single or combination of processing threads having the highest priority ring counter 400 corresponds to number of entries)
Both Korzh, HOLM, Minkin and Nubile relate to prefetching or preloading read data based on memory access pattern (see Korzh, abstract, and see HOLM, abstract, and see Minkin, abstract, and see Nubile, abstract, regarding prefetching or preloading read data).
Therefore, it would have been obvious to one of ordinary skill at the time the
invention was effectively filed to combine Korzh - HOLM - Minkin combination and Nubile by incorporating prefetching or preloading read data based on memory access pattern, as taught by Nubile, to enable data structure 450 to identify a leading thread and the leading thread can be a single or combination of processing threads having the highest priority ring counter 400 corresponds to number of entries. The combined system of Korzh – HOLM - Minkin - Nubile allows support for independent parallel plane access in a memory device with significantly reduced hardware resources in the memory sub-system as mentioned in paragraph [0020]. Therefore, the combination of Korzh - HOLM - Minkin - Nubile improves quality of service. See Nubile, paragraph [0020].
Regarding claim 12, Korzh in view of HOLM and further in view of Minkin teaches prefetching or preloading read data based on memory access pattern in claim 8. However, Korzh - HOLM - Minkin combination does not explicitly teach The method of claim 8, wherein selecting the subset of the plurality of processing threads comprises selecting a number of processing threads that have issued the highest number of memory access requests in the given period of time
On the other hand, Nubile which also relates to prefetching or preloading read data based on memory access pattern teaches The method of claim 8, wherein selecting the subset of the plurality of processing threads comprises selecting a number of processing threads that have issued the highest number of memory access requests in the given period of time. (See Fig 4A-B, paragraph [0053], illustrates data structure 450 can identify a leading thread and the leading thread can be a single or combination of processing threads having the highest priority ring counter 400 corresponds to number of entries)
Both Korzh, HOLM, Minkin and Nubile relate to prefetching or preloading read data based on memory access pattern (see Korzh, abstract, and see HOLM, abstract, and see Minkin, abstract, and see Nubile, abstract, regarding prefetching or preloading read data).
Therefore, it would have been obvious to one of ordinary skill at the time the
invention was effectively filed to combine Korzh - HOLM - Minkin combination and Nubile by incorporating prefetching or preloading read data based on memory access pattern, as taught by Nubile, to enable data structure 450 to identify a leading thread and the leading thread can be a single or combination of processing threads having the highest priority ring counter 400 corresponds to number of entries. The combined system of Korzh – HOLM - Minkin - Nubile allows support for independent parallel plane access in a memory device with significantly reduced hardware resources in the memory sub-system as mentioned in paragraph [0020]. Therefore, the combination of Korzh - HOLM - Minkin - Nubile improves quality of service. See Nubile, paragraph [0020].
Regarding claim 19, Korzh in view of HOLM and further in view of Minkin teaches prefetching or preloading read data based on memory access pattern in claim 15. However, Korzh - HOLM - Minkin combination does not explicitly teach The non-transitory computer-readable storage medium of claim 15, wherein selecting the subset of the plurality of processing threads comprises selecting a number of processing threads that have issued the highest number of memory access requests in the given period of time
On the other hand, Nubile which also relates to prefetching or preloading read data based on memory access pattern teaches The non-transitory computer-readable storage medium of claim 15, wherein selecting the subset of the plurality of processing threads comprises selecting a number of processing threads that have issued the highest number of memory access requests in the given period of time. (See Fig 4A-B, paragraph [0053], illustrates data structure 450 can identify a leading thread and the leading thread can be a single or combination of processing threads having the highest priority ring counter 400 corresponds to number of entries)
Both Korzh, HOLM, Minkin and Nubile relate to prefetching or preloading read data based on memory access pattern (see Korzh, abstract, and see HOLM, abstract, and see Minkin, abstract, and see Nubile, abstract, regarding prefetching or preloading read data).
Therefore, it would have been obvious to one of ordinary skill at the time the
invention was effectively filed to combine Korzh - HOLM - Minkin combination and Nubile by incorporating prefetching or preloading read data based on memory access pattern, as taught by Nubile, to enable data structure 450 to identify a leading thread and the leading thread can be a single or combination of processing threads having the highest priority ring counter 400 corresponds to number of entries. The combined system of Korzh – HOLM - Minkin - Nubile allows support for independent parallel plane access in a memory device with significantly reduced hardware resources in the memory sub-system as mentioned in paragraph [0020]. Therefore, the combination of Korzh - HOLM - Minkin - Nubile improves quality of service. See Nubile, paragraph [0020].
Response to Arguments
Applicant’s arguments filed on 06/16/2026 have been fully considered but they
are not persuasive.
Applicant’s first argument is claim 1, 8 and 15 amendments mapping by
primary and secondary references in page 10 of the response: Applicant submits that Holm does not teach selecting a subset of processing threads based on tracking per-thread request counts via SQIDs, as recited by amended claims 1, 8, and 15. Holm's prefetch unit 19 operates by extrapolating memory address stride patterns to predict future data values. As Holm explains, the prefetch unit determines "a 'stride' of 100 which forms another part of the corresponding entry in the prefetch table 21 and on the basis of which it can extrapolate the access pattern to generate prefetch transactions." (Holm, Paragraph [0044].) This is address-based pattern extrapolation where Holm's prefetch unit predicts which memory addresses will be accessed next based on observed stride patterns, not which processing threads should be prioritized for prefetching. Holm has no concept of identifying processing threads, tracking the number of requests per thread, using SQIDs, or selecting a subset of threads based on per-thread request frequency. Holm's prefetching decisions are driven entirely by memory address patterns, rather than by thread-level selection criteria. Therefore Holm does not teach or suggest this feature of the claims.
For at least the foregoing reasons, the combination of Korzh and Holm fails to
teach or suggest the limitations of the claims. Therefore, Applicant respectfully submits that claims 1, 8, and 15, as amended, are patentable over the cited references
In summary, applicant argued that primary reference Korzh and secondary
reference HOLM do not teach amended limitations tracking the number of requests per thread or selecting a subset of threads based on per-thread request frequency. Examiner respectfully disagrees. For further clarification examiner cites portion from HOLM. Also, for applicant’s understanding examiner would like to explain the teachings of HOLM and examiner’s interpretation in more detail here. See Fig 1, paragraph [0043] and [0044], HOLM teaches prefetch unit 19 observes or tracks memory access requests received by the L2 cache using the program counter (PC) and prefetch table 21 prefetch unit 19 determining access pattern frequency of the requests. The cited portion along with fig 1 clearly teaches prefetch unit tracks memory access requests using program counter and prefetch unit determining access pattern frequency of the requests. Thus, the rejection of amended claims 1, 8 and 15 is maintained.
Applicant’s second argument is claim 1, 8 and 15 amendments mapping by
primary and secondary references in page 12 of the response: Applicant submits that Asaad does not teach the amended claim features for at least two independent reasons. First, Asaad's load queue is not a submission queue identifier (SQID). A load queue, as described in Asaad, is a general hardware queue within a processor core that stores pending load requests. (Asaad, Paragraph [0128] describing "a load queue (i.e., a queue storing load requests) in a processor core.") Asaad's load queue is not an identifier at all, but rather includes a hardware buffer structure, and Asaad does not use it to identify or associate memory requests with specific processing threads for the purpose of tracking per-thread request counts.
Second, Asaad's counters are not "periodically decremented based on the given period of time," as recited by claims 1, 8, and 15. Asaad explicitly teaches that "when a request to that master gets scheduled (eager or non eager), the counter gets decremented." (Asaad, Paragraph [0237].) This is event-driven decrementing where the counter is decremented when a specific event occurs (i.e., a request being scheduled), not periodically based on a time period. Claims 1, 8, and 15, as amended, recite that the counters are "periodically decremented based on the given period of time," which implements a rolling time window for tracking request rates over defined time intervals. This is a fundamentally different mechanism from Asaad's event-driven counter management. In Asaad, a counter reflects the number of currently pending requests and is decremented only upon completion of a scheduling event. Accordingly, Asaad does not teach or suggest the periodic time-based decrementing recited by the amended claims
In summary, applicant argued that primary reference Korzh and secondary
references HOLM and Asaad do not teach amended limitations counter reflects the number of currently pending requests and is incremented or decremented only upon completion of a scheduling event. The amendment changes the scope of claim 1 and necessitates adding secondary reference Minkin in this regard. For further clarification examiner cites portion from Minkin. Also, for applicant’s understanding examiner would like to explain the teachings of Minkin and examiner’s interpretation in more detail here. See Fig 6, paragraph [0086], Minkin teaches request completion tracking circuitry 618 tracks the memory requests from request queue 606 for the sub-blocks and responses received from the memory subsystem. Also, See Fig 6 and 7B, paragraph [0095], illustrates TMAU have a dedicated counter that keeps track of issued L2 requests and every time request is sent to L2 cache the counter is incremented and when data come back from L2 cache or it’s completed the counter is decremented. The cited portion along with fig 1 clearly teaches counter reflects the number of currently pending requests and is incremented or decremented only upon completion of a scheduling event. Thus, the rejection of amended claims 1, 8 and 15 is maintained.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SUBIR K CHOWDHURY whose telephone number is (703)756-1207. The examiner can normally be reached Monday-Friday 8:30 - 5:00 CST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hosain Alam can be reached at (571)-272-3978. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/S.K.C./Examiner, Art Unit 2132
/HOSAIN T ALAM/Supervisory Patent Examiner, Art Unit 2132
Prosecution Insights