DETAILED ACTION
Claims 1-22 are presented for examination.
Claims 2, 4-5, 11-12, 14, 16, 18, 19 are canceled.
This office action is in response to request for continued examination of application on 11-MAY-2026.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 11-MAY-2026 has been entered.
Response to Arguments
Applicant’s arguments, see pages 10-20, filed 11-MAY-2026, with respect to the objections to the specification, objections to the claims, rejections under 35 U.S.C. 112 have been fully considered and are persuasive due to amendments. The objections to the specification, objections to the claims, rejections under 35 U.S.C. 112 have been withdrawn.
1. Regarding the rejections under 35 U.S.C. 112, Applicant has amended the claims to remove the limitation of obtaining third data when re-enabling the read ahead mode, while the third data was the data that was analyzed for an access pattern. Applicant has instead recited to obtain “additional data”, which is understood to not be the same “third data” that was determined to lack written description in the previous office action.
2. Regarding the objections to the specification, Applicant has amended the claims to now use language that is used in the specification, and therefore the objections for lack of antecedent basis in the specification are now withdrawn.
Applicant’s arguments, see pages 20-28, filed 11-MAY-2026, with respect to the rejections of claims 1, 15 under 35 U.S.C. 103 have been fully considered and are persuasive due to amendments. Therefore, the rejections have been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of previously applied prior art references.
Regarding Applicant’s arguments that the rejections to the independent claims should be withdrawn because the claims have been amended to incorporate allowable subject matter of claims 5 and 19, Examiner respectfully disagrees. As discussed in the previous office action, the basis for the allowable subject matter and the basis for the rejection under 35 U.S.C. 112(a) were the exact same limitations of obtaining the same data that was analyzed for an access pattern after re-enabling the read ahead mode. By amending the claim to overcome the rejections under 35 U.S.C. 112(a) by obtaining different data instead of the same data that was analyzed, Applicant has necessarily amended the claim to remove the previously identified allowable subject matter.
Regardless, Applicant has amended the claims in a manner which necessitates a new ground of rejection for some claims, which are addressed in the rejections below.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3, 6, 8, 9, 13, 21 are rejected under 35 U.S.C. 103 as being unpatentable over
ODAJIMA, U.S. Pub. No. 20240028517 (hereinafter “Odajima”) in view of
Luo, U.S. Pub. No. 20250054200 (hereinafter “Luo”) further in view of
Eisenman, Assaf et al., “Check-N-Run: a Checkpointing System for Training Deep Learning Recommendation Models”, USENIX Symposium on Networked Systems Design and Implementation (NSDI), (hereinafter “Eisenman”) further in view of
Kodavalla et al., U.S. Pub. No. 20070005664 (hereinafter “Kodavalla”) further in view of
Pinho et al., U.S. Pub. No. 20210149805 (hereinafter “Pinho”).
Regarding claim 1: Odajima teaches A method for executing workloads in a physical server, comprising:
obtaining a first data from a storage system and storing the first data in a cache ([0016], Odajima teaches a prefetching function in which data is copied from a main memory to a cache, based on the regularity of memory access of a program. Furthermore, in [0035-0036], Odajima teaches a variety of different access patterns, including a stream prefetching pattern, which can be used by a number of prefetchers in the system. Further, in [0054], Odajima teaches an initial value for prefetch selection, which is used to execute prefetching. Using the initial value for a prefetch, in which a prefetcher with a particular access pattern copies data from main memory to a cache, as taught by Odajima, is interpreted to be the claimed obtaining a first data from a storage system and storing the first data in a cache).
performing a first analysis of input/output (I/O) statistics associated with the first data to identify an access pattern of the first data as sequential or pseudo-random, wherein the I/O statistics are based on a first set of read requests ([0048-0052], Odajima teaches an operation where a miss rate of address blocks prefetched by each prefetcher is calculated according to the read accesses and the number of prefetch hits for those cache read accesses, and making a selection of a prefetcher to use according to the miss rate of each pattern. The calculating the miss rate is interpreted to be the claimed performing a first analysis of input/output statistics associated with the data, and the selection of the corresponding prefetcher is interpreted to be the claimed identifying an access pattern of the first data. The number of cache read accesses that is used in the calculation of the miss rate is interpreted to be the claimed first set of read requests, and the I/O statistics being based on a first set of read requests is taught. Furthermore, in [0036-0037], Odajima teaches that an example of the prefetching data access pattern is the stream prefetching, where sequential address blocks are saved, and that the prefetchers may use a different pattern.)
identifying, in response to performing the first analysis, that the access pattern of the first data is sequential ([0068], Odajima teaches that, based on the calculation result of calculating miss rates of prefetchers, after performing prefetches with the prefetcher 111, the miss rate of the prefetcher 113 is found to be the new lowest, and is thus selected for performing future prefetching. Further, in [0036], Odajima teaches that the examples of the disclosure consider each of the prefetchers 111-113 are using different access patterns that are not random. Although not explicit, the prefetcher with the lowest miss rate represents the prefetcher with the pattern with the most matches in the calculation, and therefore would be the one associated closest to the particular pattern best matching the data. Hence, identifying that the access pattern of the first data is sequential is taught.)
enabling, in response to identifying that the access pattern of the first data is sequential, a read ahead mode of the cache; predicting, in response to enabling the read ahead mode, a second data that will be used in a next epoch based on the access pattern of the first data; obtaining, in response to predicting the second data, the second data from the storage system based on the access pattern of the first data and storing the second data in the cache (Fig. 5 and [0081], Odajima teaches that after the selection of a prefetcher, the arithmetic processing for which the prefetcher is prefetching data for continues and the whole process repeats. In [0016], Odajima teaches that prefetching is performed to copy data from main memory to the cache, to be used in the future for an arithmetic operation. As previously discussed, the case in which an access pattern of a fetched data is sequential is taught. Although not explicitly stated, selecting a prefetcher to fetch new data to perform another loop of processing after the analysis of the prior arithmetic processing on the prior data, is interpreted to be the claimed enabling a read ahead mode, predicting the second data that will be used in a next epoch, and storing the second data from the storage system based on the access pattern of the first data, and storing the second data in the cache.)
servicing a subsequent file request… using the second data in the cache (Fig. 5 and [0079-0081], Odajima teaches that after a prefetcher is selected (and would have the prefetched data), the entire process of executing a loop of arithmetic processing is repeated. While not explicit, it is obvious from Odajima [0081] that the arithmetic processing using prefetching would mean that the read accesses which the hit rate calculation and subsequent selection occurs, would be repeated when the entire process returns to the start of the loop, and therefore that subsequent file requests would be served using the newly fetched data in the cache, fetched from a selected prefetcher.)
performing a second analysis of second I/O statistics associated with the second data to identify an access pattern of the second data as sequential, pseudo-random, or random, wherein the second I/O statistics are based on a second set of read requests… performing a third analysis of third I/O statistics associated with a third data to identify an access pattern of the third data as sequential, pseudo-random, or random, wherein the third I/O statistics are based on a third set of read requests… and identifying, in response to performing the third analysis, that the access pattern of the third data is sequential; and obtaining… additional data that will be used… from the storage system based on the access pattern of the third data and storing the additional data in the cache (Fig. 5 and [0069-0081], Odajima teaches that the process of performing an analysis of I/O statistics to the last prefetched data, where the I/O statistics are based on a set of read requests, to determine the access pattern to fetch new data, may loop until the operation has ended. Therefore, a second analysis of second I/O statistics, wherein the second I/O statistics are based on a second set of read requests, and making a second determination, and the same for a third, is taught.)
Although Odajima teaches the process as applied to a generic memory system performing arithmetic operations, Odajima does not appear to explicitly disclose a cache associated with a data processing unit (DPU) executing on the physical server;, read requests issued by a graphics processing unit (GPU) executing on the physical server; data that will be used in a next epoch by the GPU; a subsequent file request of the GPU, flushing, concurrent to servicing the subsequent file request, a checkpoint data from the cache to the storage system, identifying, in response to performing the second analysis, that the access pattern of the second data is random; in response to identifying that the access pattern of the second data is random, disabling the read ahead mode of the DPU; while the read ahead mode is disabled in the DPU:… re-enabling, in response to identifying that the access pattern is sequential, the read ahead mode of the DPU;
However, Luo teaches a cache associated with a data processing unit (DPU) executing on the physical server; ([0026], Luo teaches that caches are in DPUs. Furthermore, in [0059], Luo teaches that a DPU may be a card inserted into a server mainboard).
Luo further teaches read requests issued by a graphics processing unit (GPU) executing on the physical server; ([0126], Luo teaches a way of performing iterative training, where model training processors read model training data from the cache in each round of iteration. Furthermore, in [0062], Luo teaches that the model training processor may be a GPU. Furthermore, in [0016], Luo teaches that the model training processor may be located on the same server as the DPU. The GPU reading model training data from the cache of Luo corresponds to the cache read accesses of Odajima. Therefore, a GPU executing on a server, which issues reads to a cache is taught.)
Luo further teaches predicting… a second data that will be used in a next epoch by the GPU… servicing a subsequent file request of the GPU ([0126-0127] Luo teaches that for rounds of iterations of training, the DPU can read a next batch of data when the model training processor trains using the current batch of data, such that the model training processor can read the model training data from the cache of the DPU in the next round of iteration. The reading of the DPU by the GPU is interpreted to be the claimed file requests of the GPU.)
Odajima and Luo are analogous art because they are from the same field of endeavor, cache management for improving operation processing.
Odajima significantly teaches the core steps of the instant invention which provide the technological improvements that the instant invention is directed to. The addition of Luo, therefore, is merely applying those known core steps to a particular physical device comprising fairly generic components. Therefore, the jump from Odajima alone to the combination with Luo is merely taking a known set of methods and incorporating it in a known set of generic components and that hardly be seen as non-obvious.
Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have considered the server with a GPU and DPU associated with a cache that obtains data in batches such that the DPU obtains a next batch of data while the model training processor trains the AI model using a current batch of data, of Luo to be a base device, and the cache prefetching method with access pattern analysis and selection of Odajima as an improvement to cache devices in the same way as the claimed invention. One of ordinary skill in the art could have applied the cache prefetching with access pattern analysis and selection of Odajima to the cache in the device of Luo, to achieve the predictable result of a physical server containing a gpu and a DPU, which contains a cache that obtains batches of data, to obtain batches of data by performing a prefetch with access pattern analysis and selection.
While Odajima/Luo teach returning training data to a storage, Odajima/Luo do not appear to explicitly disclose flushing, concurrent to servicing the subsequent file request, a checkpoint data from the cache to the storage system, identifying, in response to performing the second analysis, that the access pattern of the second data is random; in response to identifying that the access pattern of the data is random, disabling the read ahead mode of the DPU; while the read ahead mode is disabled in the DPU:… re-enabling, in response to identifying that access pattern of the third data is sequential, the read ahead mode of the DPU.
However, Eisenman teaches Checkpoint data (Page 1, left column, first paragraph, Eisenman teaches that checkpoints are a type of data used in training machine learning models. Examiner notes that training machine learning models is a form of AI workload.).
Odajima/Luo and Eisenman are analogous art because they are from the same field of endeavor, using storage elements to enhance workload processing.
With the structure and methods of Odajima and Luo already being directed to the general structure of caching for with machine learning, the step of supplying checkpoint data to be the exact type of data which Odajima/Luo caches is a trivial addition of specifying a data type well-known within machine learning which does not significantly add inventive material to or alter the combination already present.
Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the teachings of Odajima/Luo and Eisenman to achieve the process, which serves file requests associated with machine learning based upon an identified access pattern, to use checkpoint data.
One of ordinary skill in the art would have been motivated to make this modification, as checkpointing provides known benefits for failure recovery, training process shifting, online training, and interim prediction-service as discussed in Eisenman page 1, right column, paragraphs 2-3.
While Eisenman teaches checkpointing as a way to create usable interim models, while the mode is still being trained, Odajima/Luo/Eisenman does not appear to explicitly disclose flushing, concurrent to servicing the subsequent file request, a checkpoint data from the cache to the storage system, identifying, in response to performing the second analysis, that the access pattern of the second data is random; in response to identifying that the access pattern of the data is random, disabling the read ahead mode of the DPU; while the read ahead mode is disabled in the DPU:… re-enabling, in response to identifying that access pattern of the third data is sequential, the read ahead mode of the DPU.
However, Kodavalla teaches flushing, concurrent to servicing a file request, a… data from the cache to the storage system. ([0030-0032], Kodavalla teaches assumptions in the art regarding temporary caching of data before it is written out to more permanent memory, including a Flush-Cache concurrency, where a flush-cache command (writing cached data to storage) can run concurrently with other I/O requests.).
Odajima/Luo/Eisenman and Kodavalla are analogous art because they are from the same field of endeavor, cache management for enhancing operations.
The combination of Odajima/Luo/Eisenman already largely teaches a system of fetching data into a cache, servicing read requests for the cached data, writes new data to the cache, and then flushes the newly written data to the main memory. Kodavalla is merely used to teach that two of these actions can be done concurrently, and that doing so was obvious enough to simply be an assumption of the technology even back in the year 2005 when Kodavalla was filed. The additions from Kodavalla are therefore used to teach well-known concepts that are often already assumed that do not present a significant inventive step from the existing combination of Odajima/Luo/Eisenman.
Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the teachings of Odajima/Luo/Eisenman and Kodavalla, to achieve a system where checkpoint data is able to be transmitted to the storage system while the DPU is servicing at least one of the second set of read requests.
One of ordinary skill in the art would have been motivated to make this modification in order to apply the benefits of not causing timing issues like hangs when Flush-Cache concurrency is not met, as discussed in Kodavalla [0032], to a system where an AI model may need to continue training even after the checkpoint data was sent to the cache to be stored in permanent storage.
While Odajima/Luo teach determinations, based on the analysis of I/O statistics, that data may be associated with a variety of access patterns, Odajima/Luo/Eisenman/Kodavalla do not appear to explicitly disclose identifying, in response to performing the second analysis, that the access pattern of the second data is random; in response to identifying that the access pattern of the data is random, disabling the read ahead mode of the DPU; while the read ahead mode is disabled in the DPU:… re-enabling, in response to identifying that access pattern of the third data is sequential, the read ahead mode of the DPU.
However, Pinho teaches identifying, in response to performing the second analysis, that the access pattern of the second data is random; in response to identifying that the access pattern of the data is random, disabling the read ahead mode of the DPU; ([0034], Pinho teaches a method of automatically switching prefetching on and off, based on a prediction of how much pollution will occur in the cache based on the current workload, and that when the predicted pollution is below the threshold, prefetching is turned off as the workload exhibits primarily random access patterns.).
Pinho further teaches while the read ahead mode is disabled in the DPU:… ([0068], Pinho teaches that a decision on cache policy can last for a period t, before the prediction/decision flow is repeated. In view of the disabling read ahead teachings, a re-performing the prediction/decision flow while the prefetcher is disabled is interpreted to teach the claimed condition of while the read ahead mode is disabled.)
re-enabling, in response to identifying that access pattern of the third data is sequential, the read ahead mode of the DPU ([0034], Pinho teaches that the prefetching can be switched back on when the predicted pollution is below the pollution threshold, and that by doing so, the prefetching is turned on when the workload exhibits primarily sequential access patterns.)
Odajima/Luo/Eisenman/Kodavalla and Pinho are analogous art because they are from the same field of endeavor, managing cache systems for performing operations.
Odajima already largely teaches the method of fitting a prefetching scheme to different types of identified access patterns when they can be identified and matched. The addition of the Pinho reference therefore simply teaches the handling of an extra edge case for when Odajima might find no matches. Therefore, with the existing system from the combination of Odajima/Luo/Eisenman/Kodavalla, which largely matches prefetching to identified access patterns, having another random access pattern match to another condition of disabling prefetching does not significantly alter or add an inventive step to the combination already present.
Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the teachings of Odajima/Luo/Eisenman/Kodavalla and Pinho to achieve performing a second analysis of second I/O statistics associated with second data, based on a second set of read requests, and making a determination that the data is associated with a third random access pattern, disabling the prefetching in response to the second determination, then upon a determination while the prefetching is disabled that the access pattern is sequential again, re-enabling the prefetching.
One of ordinary skill in the art would have been motivated to make this modification in order to prevent pollution from building up in the cache, and preventing performance degradation associated with excessive cache pollution as discussed in Pinho [0033-0034].
Regarding claim 3: The combination of Odajima, Luo, Eisenman, Kodavalla, and Pinho teaches all limitations of claim 1, from which claim 3 depends.
Odajima/Luo/Eisenman/Kodavalla/Pinho further teaches the checkpoint data is associated with an artificial intelligence (AI) workload from the GPU was transmitted to the DPU for storage in the cache prior to servicing the subsequent file request ([0094-0095], Luo teaches that the model training processor may return a trained AI model to the DPU. Further, Eisenman specifically teaches the checkpoint data being used for machine learning, which is a kind of artificial intelligence. Furthermore, in Page 1, right column, paragraphs 2-3, Eisenman specifically describes that with checkpointing, a checkpoint is taken and stored in a persistent storage before resume training. Combined with a teaching from Luo [0071] that training for an AI model is done iteratively, in discrete batches, and that there are cases where processing is done to prepare for a next batch, an embodiment where checkpointing is done between discrete batches, and that a checkpoint would therefore be transmitted to the DPU for storage prior to a subsequent cycle and its associated file requests is obvious. Therefore, the checkpoint data associated with an AI workload being transmitted to the DPU for storage prior to servicing a subsequent file request is taught.).
One of ordinary skill in the art would have been motivated to make this modification, for the same reasons as claim 1.
Regarding claim 6: The combination of Odajima, Luo, Eisenman, Kodavalla, and Pinho teach all limitations of claim 1, from which claim 6 depends.
Odajima/Luo/Eisenman/Kodavalla/Pinho further teaches the cache is located in the DPU ([0026], Luo teaches that caches are in DPUs.)
One of ordinary skill in the art would have bene motivated to make this modification, as one of ordinary skill would have recognized it as an obvious possible physical arrangement of the invention.
Regarding claim 8: The combination of Odajima, Luo, Eisenman, Kodavalla, Pinho teaches all limitations of claim 1, from which claim 8 depends.
Odajima/Luo/Eisenman/Kodavalla/Pinho further teaches each of the I/O statistics specify a number of cache misses that occurred when the DPU attempted to service corresponding read requests using data stored in the cache ([0051], Odajima teaches that the accuracy metric used for selecting the prefetcher is the miss rate, calculated from the number of prefetch misses that occurred when serving a number of cache read accesses. More explicitly, in [0069-0081], Odajima teaches the miss rate is calculated for the number of cache read accesses that occurs during an iteration of the arithmetic processing being performed. In other words, each time this process is looped, there will be I/O statistics that specify cache misses that occurred when attempting to service corresponding reads.)
Regarding claim 9: The combination of Odajima, Luo, Eisenman, Kodavalla, Pinho teaches all limitations of claim 1, from which claim 9 depends.
Odajima/Luo/Eisenman/Kodavalla/Pinho further teaches the second data is associated with an artificial intelligence (AI) workload executing on the GPU ([0071] Luo teaches that the DPU can obtain the model data and process it to be used for an AI model training on the GPU, which would apply to the second data that is obtained from storage.)
One of ordinary skill in the art would have been motivated to make this modification to achieve improved training efficiency by obtaining training samples in a timely manner to continue training after one AI model process was completed, as discussed in Luo [0071].
Regarding claim 13: The combination of Odajima, Luo, Eisenman, Kodavalla, Pinho teaches all limitations of claim 1, from which claim 13 depends.
Odajima/Luo/Eisenman/Kodavalla/Pinho further teaches the read ahead mode of the DPU is alternatively enabled via identifying that the access pattern of the first data is pseudo-random ([0035-0036] and [0039], Odajima teaches that one of the potential data access prediction algorithms that a prefetcher of the disclosure could implement is a Temporal prefetching, which would appear random at a glance, but may contain a repeated data access pattern. Temporal prefetching is interpreted to be the claimed pseudo random pattern, and therefore, a second access pattern being pseudo random is an obvious possible embodiment)
Regarding claim 21: The combination of Odajima, Luo, Eisenman, Kodavalla, and Pinho teaches all limitations of claim 3, from which claim 21 depends.
Odajima/Luo/Eisenman/Kodavalla/Pinho further teaches executing, using the GPU, the AI workload using the first data, wherein the execution of the AI workload is concurrent to obtaining the second data from the storage system; ([0127], Luo teaches that the model training processor can train the AI model using a current batch of data, and that when that happens, the DPU can read a next batch of data from the storage system.).
Odajima/Luo/Eisenman/Kodavalla/Pinho further teaches reaching a synchronization point in the execution of the AI workload using the first data; pausing in response to reaching the synchronization point, the execution of the AI workload; and transmitting, in response to reaching the synchronization point, the checkpoint data from the GPU for storage in the cache. ([0127], Luo teaches particular batches of training, in which the model training processor trains the AI using a current batch, while the DPU reads a next batch from the storage device. Furthermore, in Page 2, right column, under (3) Decoupling, Eisenman teaches that a checkpointing process involves the trainer processes pausing to take a snapshot. Furthermore, in Page 6, 4.2 Decoupled Checkpointing, Eisenman further teaches that checkpointing involves a stalling when creating the copy of the model parameters, and that as soon as the model snapshot is ready, a dedicated CPU processes are in charge of storing checkpoints on a host memory while training continues on the GPU. An obvious combination of these teachings results in reaching a next batch (a synchronization point), in the execution of the AI workload, then pausing the execution of the AI workload to create a snapshot (checkpoint) of the model parameters, then transmitting the checkpoint data from the GPU for storage in the cache.)
One of ordinary skill in the art would have been motivated to make this modification, for the same reasons as claim 1.
Claim 7, 10, 15, 17, 20, 22 are rejected under 35 U.S.C. 103 as being unpatentable over
ODAJIMA, U.S. Pub. No. 20240028517 (hereinafter “Odajima”) in view of
Luo, U.S. Pub. No. 20250054200 (hereinafter “Luo”) further in view of
Eisenman, Assaf et al., “Check-N-Run: a Checkpointing System for Training Deep Learning Recommendation Models”, USENIX Symposium on Networked Systems Design and Implementation (NSDI), (hereinafter “Eisenman”) further in view of
Kodavalla et al., U.S. Pub. No. 20070005664 (hereinafter “Kodavalla”) further in view of
Pinho et al., U.S. Pub. No. 20210149805 (hereinafter “Pinho”) further in view of
ARNDT et al., U.S. Pub. No. 20230094937 (hereinafter “Arndt”)
Regarding Claim 7: The combination of Odajima, Luo, Eisenman, Kodavalla, Pinho teaches all limitations of claim 1, from which claim 7 depends.
While Luo describes an embodiment where the DPU/cache is located on a different server from the model training processor in [0017], Odajima/Luo/Eisenman/Kodavalla/Pinho do not appear to explicitly disclose the cache is located on a Top of Rack (ToR) switch; wherein the ToR switch is connected to the physical server; and wherein the ToR switch is interposed between the storage system and the physical server.
However, Arndt teaches the cache is located on a Top of Rack (ToR) switch; ([0033] and Fig. 4, Arndt teaches that control switches like 425 may comprise a page cache 470. Furthermore, in [0018], Arndt teaches that control switches, which may form a control plane, may be Top-of-Rack ToR switches).
Arndt further teaches wherein the ToR switch is connected to the physical server; and ([0015-0016], Arndt teaches that the control plane, which may comprise ToR switches, may be connected to the data plane, each comprising worker nodes, which include a cluster of servers, to execute the workloads on the data plane. The cluster of servers on the worker nodes are interpreted to be the claimed physical server. Since the control plane is executing workloads, and the workers host the application pods for executing applications, the ToR switch being connected to the physical server is taught.)
Arndt further teaches the ToR switch is interposed between the storage system and the physical server ([0033] and Fig. 4, Arndt teaches that the control switch, which may be the ToR switch, may contain a disk driver 476, which handles a storage disk 477, which is outside of the ToR switch. With the physical server being located on the data plane with the ToR externally executing workloads on it, as explained with respect to claim 1, and the disk driver on the ToR handling the storage disk, the ToR is operatively interposed between the storage disk and the physical server.)
Odajima/Luo/Eisenman/Kodavalla/Pinho and Arndt are analogous art because they are from the same field of endeavor, workload processing with memory.
Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the teachings of Odajima/Luo/Eisenman/Kodavalla/Pinho and Arndt to achieve the method for executing workloads on a physical server of claim 1, but with a configuration where the cache is located on a ToR switch, connected to a physical server on a different plane, and where the ToR switch is operatively interposed between the storage system and physical server.
One of ordinary skill in the art would have been motivated to make this modification as a possible identified implementation of the method, as the described control plane framework may provide a resilient system with scalability, while maintaining a small form factor as discussed in Arndt [0011].
Regarding claim 10: The combination of Odajima, Luo, Eisenman, Kodavalla, Pinho, and Arndt teaches all limitations of claim 7, from which claim 10 depends.
Odajima/Luo/Eisenman/Kodavalla/Pinho/Arndt teaches the checkpoint data is associated with an artificial intelligence (AI) workload from the GPU and was transmitted to the cache for storage in the cache located on the ToR switch for storage prior to servicing the subsequent file request (As discussed with respect to claim 7, the cache is on the ToR switch. Further, in [0094-0095], Luo teaches that the model training processor may return a trained AI model to the DPU. Further, Eisenman specifically teaches the checkpoint data being used for machine learning, which is a kind of artificial intelligence. Furthermore, in Page 1, right column, paragraphs 2-3, Eisenman specifically describes that with checkpointing, a checkpoint is taken and stored in a persistent storage before resume training. Combined with a teaching from Luo [0071] that training for an AI model is done iteratively, in discrete batches, and that there are cases where processing is done to prepare for a next batch, an embodiment where checkpointing is done between discrete batches, and that a checkpoint would therefore be transmitted to the cache in the ToR switch for storage prior to a subsequent cycle and its associated file requests is obvious. Therefore, the checkpoint data associated with an AI workload being transmitted to the cache in the ToR switch for storage prior to servicing a subsequent file request is taught.).
One of ordinary skill in the art would have been motivated to make this modification as model training processors are known to have limited memory space, and should have abundant memory space to train the AI model to improve training efficiency. One of ordinary skill in the art would recognize that transferring data associated with the AI workload from the GPU would achieve the same benefit of reducing occupation of model training processor memory, as discussed in Luo, [0007-0008].
Regarding claim 15: Odajima teaches A method for executing workloads, comprising:
Obtaining a first data from a storage system using and storing the first data in a cache ([0016], Odajima teaches a prefetching function in which data is copied from a main memory to a cache, based on the regularity of memory access of a program. Furthermore, in [0035-0036], Odajima teaches a variety of different access patterns, including a stream prefetching pattern, which can be used by a number of prefetchers in the system. Further, in [0054], Odajima teaches an initial value for prefetch selection, which is used to execute prefetching. Using the initial value for a prefetch, in which a prefetcher with a particular access pattern copies data from main memory to a cache, as taught by Odajima, is interpreted to be the claimed obtaining data from a storage system and storing the data in a cache).
Performing… a first analysis of input/output (I/O) statistics associated with the first data to identify an access pattern of the data as sequential, pseudo-random, or random, wherein the I/O statistics are based on… servicing a first set of read requests… using the cache ([0048-0052], Odajima teaches an operation where a miss rate of address blocks prefetched by each prefetcher is calculated according to the read accesses and the number of prefetch hits for those cache read accesses, and making a selection of a prefetcher to use according to the miss rate of each pattern. The calculating the miss rate is interpreted to be the claimed performing a first analysis of input/output statistics associated with the data, and the selection of the corresponding prefetcher is interpreted to be the claimed identifying an access pattern of the first data. The number of cache read accesses that is used in the calculation of the miss rate is interpreted to be the claimed first set of read requests, and the I/O statistics being based on a first set of read requests is taught. Furthermore, in [0036-0037], Odajima teaches that an example of the prefetching data access pattern is the stream prefetching, where sequential address blocks are saved, and that the prefetchers may use a different pattern.)
identifying, in response to performing the first analysis, that the access pattern of the first data is either sequential or pseudo-random ([0068], Odajima teaches that, based on the calculation result of calculating miss rates of prefetchers, after performing prefetches with the prefetcher 111, the miss rate of the prefetcher 113 is found to be the new lowest, and is thus selected for performing future prefetching. Further, in [0036], Odajima teaches that the examples of the disclosure consider each of the prefetchers 111-113 are using different access patterns that are not random. Although not explicit, the prefetcher with the lowest miss rate represents the prefetcher with the pattern with the most matches in the calculation, and therefore would be the one associated closest to the particular pattern best matching the data. Hence, identifying that the access pattern of the first data is sequential is taught.)
enabling, in response to identifying that the access pattern of the first data is either sequential or pserudo-random, a read ahead mode of the cache; predicting, in response to enabling the read ahead mode, a second data that will be used in a next epoch based on the access pattern of the first data; obtaining, in response to predicting the second data, the second data from the storage system based on the access pattern of the first data and storing the second data in the cache (Fig. 5 and [0081], Odajima teaches that after the selection of a prefetcher, the arithmetic processing for which the prefetcher is prefetching data for continues and the whole process repeats. In [0016], Odajima teaches that prefetching is performed to copy data from main memory to the cache, to be used in the future for an arithmetic operation. As previously discussed, the case in which an access pattern of a fetched data is sequential is taught. Although not explicitly stated, selecting a prefetcher to fetch new data to perform another loop of processing after the analysis of the prior arithmetic processing on the prior data, is interpreted to be the claimed enabling a read ahead mode, predicting the second data that will be used in a next epoch, and storing the second data from the storage system based on the access pattern of the first data, and storing the second data in the cache.)
servicing a subsequent file request… using the second data in the cache (Fig. 5 and [0079-0081], Odajima teaches that after a prefetcher is selected (and would have the prefetched data), the entire process of executing a loop of arithmetic processing is repeated. While not explicit, it is obvious from Odajima [0081] that the arithmetic processing using prefetching would mean that the read accesses which the hit rate calculation and subsequent selection occurs, would be repeated when the entire process returns to the start of the loop, and therefore that subsequent file requests would be served using the newly fetched data in the cache, fetched from a selected prefetcher.)
performing a second analysis of second I/O statistics associated with the second data to identify an access pattern of the second data as sequential, pseudo-random, or random, wherein the second I/O statistics are based on a second set of read requests… performing a third analysis of third I/O statistics associated with a third data to identify an access pattern of the third data as sequential, pseudo-random, or random, wherein the third I/O statistics are based on a third set of read requests… and identifying, in response to performing the third analysis, that the access pattern of the third data is sequential; and obtaining… additional data that will be used… from the storage system based on the access pattern of the third data and storing the additional data in the cache (Fig. 5 and [0069-0081], Odajima teaches that the process of performing an analysis of I/O statistics to the last prefetched data, where the I/O statistics are based on a set of read requests, to determine the access pattern to fetch new data, may loop until the operation has ended. Therefore, a second analysis of second I/O statistics, wherein the second I/O statistics are based on a second set of read requests, and making a second determination, and the same for a third, is taught.)
Although Odajima teaches the process as applied to a generic memory system performing arithmetic operations, Odajima does not appear to explicitly disclose a cache in a Top of Rack (ToR) switch connected to a physical server;, analysis by the ToR switch, a data processing unit (DPU) servicing a first set of read requests issued by a graphics processing unit (GPU) executing on the physical server; data that will be used in a next epoch by the GPU; a subsequent file request of the GPU, or flushing, concurrent to servicing the subsequent file request, a checkpoint data from the cache to the storage system; identifying, in response to performing the second analysis, that the access pattern of the second data is random; in response to identifying that the access pattern of the second data is random, disabling the read ahead mode of the ToR switch; while the read ahead mode is disabled in the ToR switch:… re-enabling, in response to identifying that the access pattern of the third data is sequential or pseudo-random, the read ahead mode of the ToR switch;
However, Luo teaches a data processing unit (DPU) servicing a first set of read requests issued by a graphics processing unit (GPU) ([0126], Luo teaches a way of performing iterative training, where model training processors read model training data from the DPU in each round of iteration. Further, in [0071], Luo teaches that the DPU may specifically output the model training data to the GPU. Furthermore, in [0062], Luo teaches that the model training processor may be a GPU. The GPU reading model training data output from the DPU of Luo, to perform an iteration of training, is interpreted to be the claimed DPU servicing a first set of read requests issued by a GPU.)
Luo further teaches the GPU and DPU are executing on the physical server; ([0016], Luo teaches that the model training processor may be located on the same server as the DPU.)
Odajima and Luo are analogous art because they are from the same field of endeavor, cache management for improving operation processing.
Odajima significantly teaches the core steps of the instant invention which provide the technological improvements that the instant invention is directed to. The addition of Luo, therefore, is merely applying those known core steps to a particular physical device comprising fairly generic components. Therefore, the jump from Odajima alone to the combination with Luo is merely taking a known set of methods and incorporating it in a known set of generic components and that hardly be seen as non-obvious.
Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have considered the server with a GPU and DPU that obtains data in batches of Luo to be a base device, and the cache prefetching method with access pattern analysis and selection of Odajima as an improvement to data fetching devices in the same way as the claimed invention. One of ordinary skill in the art could have applied the cache prefetching with access pattern analysis and selection of Odajima to the DPU of Luo, to achieve the predictable result of a physical server containing a gpu and a DPU, which obtains batches of data, to obtain batches of data by performing a prefetch with access pattern analysis and selection.
While Odajima/Luo teach returning training data to a storage, Odajima/Luo do not appear to explicitly disclose a cache in a Top of Rack (ToR) switch connected to a physical server;, analysis by the ToR switch, flushing, concurrent to servicing the subsequent file request, a checkpoint data from the cache to the storage system; identifying, in response to performing the second analysis, that the access pattern of the second data is random; in response to identifying that the access pattern of the second data is random, disabling the read ahead mode of the ToR switch; while the read ahead mode is disabled in the ToR switch:… re-enabling, in response to identifying that the access pattern of the third data is sequential or pseudo-random, the read ahead mode of the ToR switch;
However, Eisenman teaches Checkpoint data (Page 1, left column, first paragraph, Eisenman teaches that checkpoints are a type of data used in training machine learning models. Examiner notes that training machine learning models is a form of AI workload.).
Odajima/Luo and Eisenman are analogous art because they are from the same field of endeavor, using storage elements to enhance workload processing.
With the structure and methods of Odajima and Luo already being directed to the general structure of caching for with machine learning, the step of supplying checkpoint data to be the exact type of data which Odajima/Luo caches is a trivial addition of specifying a data type well-known within machine learning which does not significantly add inventive material to or alter the combination already present.
Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the teachings of Odajima/Luo and Eisenman to achieve the process, which serves file requests associated with machine learning based upon an identified access pattern, to use checkpoint data.
One of ordinary skill in the art would have been motivated to make this modification, as checkpointing provides known benefits for failure recovery, training process shifting, online training, and interim prediction-service as discussed in Eisenman page 1, right column, paragraphs 2-3.
While Eisenman teaches checkpointing as a way to create usable interim models, while the mode is still being trained, Odajima/Luo/Eisenman does not appear to explicitly disclose a cache in a Top of Rack (ToR) switch connected to a physical server;, analysis by the ToR switch, flushing, concurrent to servicing the subsequent file request, a checkpoint data from the cache to the storage system; identifying, in response to performing the second analysis, that the access pattern of the second data is random; in response to identifying that the access pattern of the second data is random, disabling the read ahead mode of the ToR switch; while the read ahead mode is disabled in the ToR switch:… re-enabling, in response to identifying that the access pattern of the third data is sequential or pseudo-random, the read ahead mode of the ToR switch;
However, Kodavalla teaches flushing, concurrent to servicing a file request, a… data from the cache to the storage system. ([0030-0032], Kodavalla teaches assumptions in the art regarding temporary caching of data before it is written out to more permanent memory, including a Flush-Cache concurrency, where a flush-cache command (writing cached data to storage) can run concurrently with other I/O requests.).
Odajima/Luo/Eisenman and Kodavalla are analogous art because they are from the same field of endeavor, cache management for enhancing operations.
The combination of Odajima/Luo/Eisenman already largely teaches a system of fetching data into a cache, servicing read requests for the cached data, writes new data to the cache, and then flushes the newly written data to the main memory. Kodavalla is merely used to teach that two of these actions can be done concurrently, and that doing so was obvious enough to simply be an assumption of the technology even back in the year 2005 when Kodavalla was filed. The additions from Kodavalla are therefore used to teach well-known concepts that are often already assumed that do not present a significant inventive step from the existing combination of Odajima/Luo/Eisenman.
Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the teachings of Odajima/Luo/Eisenman and Kodavalla, to achieve a system where checkpoint data is able to be transmitted to the storage system while the DPU is servicing at least one of the second set of read requests.
One of ordinary skill in the art would have been motivated to make this modification in order to apply the benefits of not causing timing issues like hangs when Flush-Cache concurrency is not met, as discussed in Kodavalla [0032], to a system where an AI model may need to continue training even after the checkpoint data was sent to the cache to be stored in permanent storage.
Odajima/Luo/Eisenman/Kodavalla do not appear to explicitly disclose a cache in a Top of Rack (ToR) switch connected to a physical server;, analysis by the ToR switch, in response to performing the second analysis, that the access pattern of the second data is random; in response to identifying that the access pattern of the second data is random, disabling the read ahead mode of the ToR switch; while the read ahead mode is disabled in the ToR switch:… re-enabling, in response to identifying that the access pattern of the third data is sequential or pseudo-random, the read ahead mode of the ToR switch;
However, Pinho teaches identifying, in response to performing the second analysis, that the access pattern of the second data is random; in response to identifying that the access pattern of the data is random, disabling the read ahead mode; ([0034], Pinho teaches a method of automatically switching prefetching on and off, based on a prediction of how much pollution will occur in the cache based on the current workload, and that when the predicted pollution is below the threshold, prefetching is turned off as the workload exhibits primarily random access patterns.).
Pinho further teaches while the read ahead mode is disabled ([0068], Pinho teaches that a decision on cache policy can last for a period t, before the prediction/decision flow is repeated. In view of the disabling read ahead teachings, a re-performing the prediction/decision flow while the prefetcher is disabled is interpreted to teach the claimed condition of while the read ahead mode is disabled.)
re-enabling, in response to identifying that access pattern of the third data is sequential, the read ahead mode ([0034], Pinho teaches that the prefetching can be switched back on when the predicted pollution is below the pollution threshold, and that by doing so, the prefetching is turned on when the workload exhibits primarily sequential access patterns.)
Odajima/Luo/Eisenman/Kodavalla and Pinho are analogous art because they are from the same field of endeavor, managing cache systems for performing operations.
Odajima already largely teaches the method of fitting a prefetching scheme to different types of identified access patterns when they can be identified and matched. The addition of the Pinho reference therefore simply teaches the handling of an extra edge case for when Odajima might find no matches. Therefore, with the existing system from the combination of Odajima/Luo/Eisenman/Kodavalla, which largely matches prefetching to identified access patterns, having another random access pattern match to another condition of disabling prefetching does not significantly alter or add an inventive step to the combination already present.
Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the teachings of Odajima/Luo/Eisenman/Kodavalla and Pinho to achieve performing a second analysis of second I/O statistics associated with second data, based on a second set of read requests, and making a determination that the data is associated with a third random access pattern, disabling the prefetching in response to the second determination, then upon a determination while the prefetching is disabled that the access pattern is sequential again, re-enabling the prefetching.
One of ordinary skill in the art would have been motivated to make this modification in order to prevent pollution from building up in the cache, and preventing performance degradation associated with excessive cache pollution as discussed in Pinho [0033-0034].
Odajima/Luo/Eisenman/Kodavalla/Pinho do not appear to explicitly disclose a cache in a Top of Rack (ToR) switch connected to a physical server;, analysis by the ToR switch, disabling the read ahead mode of the ToR switch; while the read ahead mode is disabled in the ToR switch:… re-enabling, in response to identifying that the access pattern of the third data is sequential or pseudo-random, the read ahead mode of the ToR switch;
However, Arndt teaches a cache in a Top of Rack (ToR) switch connected to a physical server ([0033] and Fig. 4, Arndt teaches that control switches like 425 may comprise a page cache 470. Furthermore, in [0018], Arndt teaches that control switches, which may form a control plane, may be Top-of-Rack ToR switches. Further, in [0015-0016], Arndt teaches that the control plane may be connected to the data plane, each comprising worker nodes, which include a cluster of servers, to execute the workloads on the data plane. The cluster of servers on the worker nodes are interpreted to be the claimed physical server. Since the control plane is executing workloads, and the workers host the application pods for executing applications, the ToR switch being connected to the physical server is taught.)
Arndt further teaches processing by the ToR switch ([0021-0023], Arndt teaches that a ToR switch has its own processor which can execute instructions to manage workloads in the worker nodes. While not explicit, the executing instructions to manage workloads of Arndt, may implement the analysis of I/O statistics of taught by Odajima/Luo/Eisenman/Kodavalla/Pinho in an obvious combination.)
Odajima/Luo/Eisenman/Kodavalla/Pinho and Arndt are analogous art because they are from the same field of endeavor, workload processing with memory.
Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the teachings of Odajima/Luo/Eisenman/Kodavalla/Pinho and Arndt, to implement the method of obtaining data using a first sequential access pattern and storing the data into a cache, and performing an analysis of I/O statistics based on a DPU servicing a first set of read requests issued by a GPU using the cache, with the GPU and DPU executing on the same server, and making determinations of the data being associated with a pseudo-random access pattern to obtain a second set of data with, to also be implemented by a ToR switch connected to a physical server, where the ToR switch performs the analysis of I/O statistics.
One of ordinary skill in the art would have been motivated to make this modification as a possible physical implementation of the method, as the described control plane framework may provide a resilient system with scalability, while maintaining a small form factor as discussed in Arndt [0011].
Regarding claim 17: The combination of Odajima, Luo, Pinho, Eisenman, Kodavalla, and Arndt teaches all limitations of claim 15, from which claim 17 depends.
Odajima/Luo/Eisenman/Kodavalla/Pinho/Arndt further teaches the checkpoint data is associated with an artificial intelligence (AI) workload from the GPU was transmitted to the DPU for storage in the cache prior to servicing the subsequent file request ([0094-0095], Luo teaches that the model training processor may return a trained AI model to the DPU. Further, Eisenman specifically teaches the checkpoint data being used for machine learning, which is a kind of artificial intelligence. Furthermore, in Page 1, right column, paragraphs 2-3, Eisenman specifically describes that with checkpointing, a checkpoint is taken and stored in a persistent storage before resume training. Combined with a teaching from Luo [0071] that training for an AI model is done iteratively, in discrete batches, and that there are cases where processing is done to prepare for a next batch, an embodiment where checkpointing is done between discrete batches, and that a checkpoint would therefore be transmitted to the DPU for storage prior to a subsequent cycle and its associated file requests is obvious. Therefore, the checkpoint data associated with an AI workload being transmitted to the DPU for storage prior to servicing a subsequent file request is taught.).
One of ordinary skill in the art would have been motivated to make this modification, for the same reasons as claim 15.
Regarding claim 20: The combination of Odajima, Luo, Eisenman, Kodavalla, Pinho, and Arndt teaches all limitations of claim 15, from which claim 20 depends.
Odajima/Luo/Eisenman/Kodavalla/Pinho/Arndt further teaches each of the I/O statistics specify a number of cache misses that occurred when the DPU attempted to service the first set of corresponding read requests using data stored in the cache ([0051], Odajima teaches that the accuracy metric used for selecting the prefetcher is the miss rate, calculated from the number of prefetch misses that occurred when serving a number of cache read accesses. More explicitly, in [0069-0081], Odajima teaches the miss rate is calculated for the number of cache read accesses that occurs during an iteration of the arithmetic processing being performed. In other words, each time this process is looped, there will be I/O statistics that specify cache misses that occurred when attempting to service corresponding reads.)
Regarding claim 22: The combination of Odajima, Luo, Eisenman, Kodavalla, Pinho, Arndt teaches all limitations of claim 17, from which claim 22 depends.
Odajima/Luo/Pinho/Eisenman/Kodavalla/Arndt further teaches executing, using the GPU, the AI workload using the first data, wherein the execution of the AI workload is concurrent to obtaining the second data from the storage system; ([0127], Luo teaches that the model training processor can train the AI model using a current batch of data, and that when that happens, the DPU can read a next batch of data from the storage system.).
Odajima/Luo/Pinho/Eisenman/Kodavalla/Arndt further teaches reaching a synchronization point in the execution of the AI workload using the first data; pausing in response to reaching the synchronization point, the execution of the AI workload; and transmitting, in response to reaching the synchronization point, the checkpoint data from the GPU for storage in the cache. ([0127], Luo teaches particular batches of training, in which the model training processor trains the AI using a current batch, while the DPU reads a next batch from the storage device. Furthermore, in Page 2, right column, under (3) Decoupling, Eisenman teaches that a checkpointing process involves the trainer processes pausing to take a snapshot. Furthermore, in Page 6, 4.2 Decoupled Checkpointing, Eisenman further teaches that checkpointing involves a stalling when creating the copy of the model parameters, and that as soon as the model snapshot is ready, a dedicated CPU processes are in charge of storing checkpoints on a host memory while training continues on the GPU. An obvious combination of these teachings results in reaching a next batch (a synchronization point), in the execution of the AI workload, then pausing the execution of the AI workload to create a snapshot (checkpoint) of the model parameters, then transmitting the checkpoint data from the GPU for storage in the cache.)
One of ordinary skill in the art would have been motivated to make this modification, for the same reasons as claim 15.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KAITLYN HUNG PHAM whose telephone number is (571)272-6333. The examiner can normally be reached M/Tu/Th/F 8:00-6:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Rocio Del Mar Perez-Velez can be reached at 571-270-5935. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/K.H.P./Examiner, Art Unit 2133
/KHOA D DOAN/Primary Examiner, Art Unit 2133
Prosecution Insights