DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
The Office Action is in response to claims filed 05/06/2026.
Claims 1-20 are pending.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 18, 19, and 20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 6, 6, 6, 1, 5, 6, 6, 9, 8, 9, 8, 11, 9, 9, 9, 19, 10, 14, and 20, respectively, of copending application 18/214,449 (hereafter ‘449) in view of Modukuri et al. Pub. No. US 20210286752 A1 (hereafter Modukuri) as exemplified in the table below.
Instant Application
18/214,449
1. (Currently Amended) One or more processors, comprising: circuitry to:
in response to receiving an application programming interface (API) call comprising arguments including a virtual memory address and a destination device:
cause information to be read from one or more non-uniform memory access (NUMA) storages or one or more graphics processor unit (GPU) physical storages
corresponding to the virtual memory address
and accessible by one or more central processing units (CPUs) and one or more GPUs;
and cause the information to be stored at a physical memory of the destination device.
1. (Currently Amended) One or more processors comprising: circuitry to,
in response to perform an application programming interface (API) call,
to cause information to be stored using one or more virtual memory addresses accessible to one or more non-uniform memory access (NUMA) nodes and one or more graphics processor units (GPUs) to be stored in storage of a NUMA node of the one or more NUMA nodes indicated by one or more parameters of the API.
Modukuri ¶ [0076] states “In at least one embodiment, cuFileWrite is specified as: ssize_t cuFileWrite(CUFileHandle fh, void *devPtr, size_t size, off_t offset), where fh is a file descriptor for a file, devPtr is a start address of a device pointer to read from, size is a size in bytes to write, and offset is an offset in a file to write to.”
6. (Currently Amended) The one or more processors of claim 1, wherein the information is stored using the one or more virtual memory addresses accessible by the one or more GPUs and one or more central processing units (CPUs), and the API call is to cause the information to be stored within a physical memory of the NUMA node.
2. (Currently Amended) The one or more processors of claim 1,
wherein the arguments include one or more pointers to the virtual memory address
6. (Currently Amended) The one or more processors of claim 1,
wherein the information is stored using the one or more virtual memory addresses accessible by the one or more GPUs and one or more central processing units (CPUs), and the API call is to cause the information to be stored within a physical memory of the NUMA node.
Modukuri ¶ [0076] states “In at least one embodiment, cuFileWrite is specified as: ssize_t cuFileWrite(CUFileHandle fh, void *devPtr, size_t size, off_t offset), where fh is a file descriptor for a file, devPtr is a start address of a device pointer to read from, size is a size in bytes to write, and offset is an offset in a file to write to.”
3. (Currently Amended) The one or more processors of claim 1,
wherein the API is to cause the information to be stored within one or more of the one or more NUMA storages.
6. (Currently Amended) The one or more processors of claim 1,
wherein the information is stored using the one or more virtual memory addresses accessible by the one or more GPUs and one or more central processing units (CPUs), and the API call is to cause the information to be stored within a physical memory of the NUMA node.
4. (Currently Amended) The one or more processors of claim 1,
wherein the arguments include information indicating where prefetched data is to be stored in one of the one or more GPUs.
1. (Currently Amended) One or more processors comprising: circuitry to,
in response to perform an application programming interface (API) call,
to cause information to be stored using one or more virtual memory addresses accessible to one or more non-uniform memory access (NUMA) nodes and one or more graphics processor units (GPUs) to be stored in storage of a NUMA node of the one or more NUMA nodes indicated by one or more parameters of the API.
Modukuri ¶ [0055] teaches “In at least one embodiment, a file system, block system, object system, or key-value store system, either in an operating system or a standalone driver, performs a read-ahead operation to accomplish prefetching using an implementation similar to that of transfers initiated with cuFile-based API calls. In at least one embodiment, at least one processor moves data based, at least in part, on an explicit prefetch (e.g., a CUDA cudaMemPrefetch).
5. (Currently Amended) The one or more processors of claim 1,
wherein the arguments include information indicating where prefetched data is to be stored in one of the one or more NUMA storages.
1. (Currently Amended) One or more processors comprising: circuitry to,
in response to perform an application programming interface (API) call,
to cause information to be stored using one or more virtual memory addresses accessible to one or more non-uniform memory access (NUMA) nodes and one or more graphics processor units (GPUs) to be stored in storage of a NUMA node of the one or more NUMA nodes indicated by one or more parameters of the API.
5. (Currently Amended) The one or more processors of claim 1, wherein the API call is to cause the information to be prefetched to a memory of a host NUMA node specified by one or more of one or more parameters.
6. (Currently Amended) The one or more processors of claim 1,
wherein causing the information to be stored at the physical memory of the destination device comprises copying the information to the physical memory of the destination device.
6. (Currently Amended) The one or more processors of claim 1,
wherein the information is stored using the one or more virtual memory addresses accessible by the one or more GPUs and one or more central processing units (CPUs), and the API call is to cause the information to be stored within a physical memory of the NUMA node.
Examiner’s Note: storing information implies copying
7. (Currently Amended) The one or more processors of claim 1,
wherein causing the information to be stored at the physical memory of the destination device comprises moving the information to the physical memory of the destination device.
6. (Currently Amended) The one or more processors of claim 1,
wherein the information is stored using the one or more virtual memory addresses accessible by the one or more GPUs and one or more central processing units (CPUs), and the API call is to cause the information to be stored within a physical memory of the NUMA node.
Examiner’s Note: storing information implies moving
8. (Currently Amended) A system, comprising:
one or more graphics processor units (GPUs); and one or more processors to:
in response to receiving an application programming interface (API) call comprising arguments including a virtual memory address and a destination device:
cause information to be read from one or more non-uniform memory access (NUMA) storages or one or more GPU physical storages corresponding to the
virtual memory address and accessible by one or more central processing units (CPUs) and the one or more GPUs;
and cause the information to be stored at a physical memory of the destination device.
8. A system, comprising:
one or more processors to receive
cause information
Modukuri ¶ [0076] states “In at least one embodiment, cuFileWrite is specified as: ssize_t cuFileWrite(CUFileHandle fh, void *devPtr, size_t size, off_t offset), where fh is a file descriptor for a file, devPtr is a start address of a device pointer to read from, size is a size in bytes to write, and offset is an offset in a file to write to.”
virtual memory addresses accessible one or more (NUMA) nodes and
9. (Currently Amended) The system of claim 8, wherein the information is stored using virtual addresses accessible to one or more central processing units (CPUs) and the one or more GPUs, and the API call is to cause the information to be stored within a physical memory of the NUMA node.
9. (Currently Amended) The system of claim 8,
wherein the destination device comprises one or more of the NUMA storages.
8. A system, comprising:
one or more processors to receive
cause informationone or more (NUMA) nodes and indicated NUMA node of the one or more NUMA nodes.
10. (Currently Amended) The system of claim 8,
wherein the arguments include information indicating a size of the data that is to be stored at the physical memory of the destination device.
9. (Currently Amended) The system of claim 8,
wherein the information is stored using virtual addresses accessible to one or more central processing units (CPUs) and the one or more GPUs, and the API call is to cause the information to be stored within a physical memory of the NUMA node.
Modukuri ¶ [0076] states “In at least one embodiment, cuFileWrite is specified as: ssize_t cuFileWrite(CUFileHandle fh, void *devPtr, size_t size, off_t offset), where fh is a file descriptor for a file, devPtr is a start address of a device pointer to read from, size is a size in bytes to write, and offset is an offset in a file to write to.”
11. (Currently Amended) The system of claim 8,
wherein the arguments include an identifier of a particular NUMA storage of the one or more NUMA storages.
8. A system, comprising:
one or more processors to receive an application programming interface (API) call including one or more parameters indicating at least a non-uniform memory access (NUMA) node and in response to the API call,
cause information stored using one or more virtual memory addresses accessible one or more (NUMA) nodes and one or more graphics processor units (GPUs) to be stored in the indicated NUMA node of the one or more NUMA nodes.
12. (Currently Amended) The system of claim 8,
wherein the API is to cause the information to be read from a NUMA storage of the one or more NUMA storages associated with the one or more CPUs.
11. (Currently Amended) The system of claim 8,
wherein the API call is to cause the information to be prefetched to the NUMA node that includes a central processing unit.
Modukuri ¶ [0076] states “In at least one embodiment, cuFileWrite is specified as: ssize_t cuFileWrite(CUFileHandle fh, void *devPtr, size_t size, off_t offset), where fh is a file descriptor for a file, devPtr is a start address of a device pointer to read from, size is a size in bytes to write, and offset is an offset in a file to write to.”
13. (Currently Amended) The system of claim 8,
wherein one or more physical storages corresponding to the virtual memory address are
further accessible by a plurality of NUMA nodes.
9. (Currently Amended) The system of claim 8,
wherein the information is stored using virtual addresses accessible to one or more central processing units (CPUs) and the one or more GPUs, and the API call is to cause the information to be stored within a physical memory of the NUMA node.
8. A system, comprising: one or more processors to receive an application programming interface (API) call including one or more parameters indicating at least a non-uniform memory access (NUMA) node and in response to the API call,
cause information stored using one or more virtual memory addresses accessible one or more (NUMA) nodes and one or more graphics processor units (GPUs) to be stored in the indicated NUMA node of the one or more NUMA nodes.
14. (Currently Amended) A method, comprising:
receiving an application programming interface (API) call comprising arguments including a virtual memory address and a destination device;
causing, in response to the API call, information to be read from one or more non-uniform memory access (NUMA) storages or one or more graphics processor unit (GPU) physical storages corresponding to the virtual memory address and accessible by one or more central processing units (CPUs) and the one or more GPUs;
and causing the information to be stored at a physical memory of the destination device.
14. (Currently Amended) A method, comprising:
in response to an application programming interface (API) call,
Modukuri ¶ [0076] states “In at least one embodiment, cuFileWrite is specified as: ssize_t cuFileWrite(CUFileHandle fh, void *devPtr, size_t size, off_t offset), where fh is a file descriptor for a file, devPtr is a start address of a device pointer to read from, size is a size in bytes to write, and offset is an offset in a file to write to.”
causing information stored using one or more virtual memory addresses accessible one or more non-uniform memory access (NUMA) nodes and one or more graphics processor units (GPUs) to be stored in storage of a NUMA node of the one or more NUMA nodes indicated by one or more parameters of the API.
9. (Currently Amended) The system of claim 8, wherein the information is stored using virtual addresses accessible to one or more central processing units (CPUs) and the one or more GPUs, and the API call is to cause the information to be stored within a physical memory of the NUMA node.
16. (Currently Amended) The method of claim 14,
wherein causing the information to be stored at the physical memory of the destination device comprises copying the information to the physical memory of the destination device.
9. (Currently Amended) The system of claim 8,
wherein the information is stored using virtual addresses accessible to one or more central processing units (CPUs) and the one or more GPUs, and the API call is to cause the information to be stored within a physical memory of the NUMA node.Examiner’s Note: storing information implies copying
17. (Currently Amended) The method of claim 14,
wherein the arguments include an identifier of a NUMA node indicating where the information is to be stored within
the one or more NUMA storages.
19. (Currently Amended) The method of claim 14,
wherein the information indicates the NUMA node to which the information is to be stored.
14. (Currently Amended) A method, comprising:
in response to an application programming interface (API) call, causing information stored using one or more virtual memory addresses accessible one or more non-uniform memory access (NUMA) nodes and one or more graphics processor units (GPUs) to be stored in storage of a NUMA node of the one or more NUMA nodes indicated by one or more parameters of the API.
18. (Currently Amended) The method of claim 14,
wherein the arguments include a range of virtual memory addresses.
10. (Currently Amended) The system of claim 8,
wherein the one or more indicators one or more parameters include information that indicates a range of memory accessible to one or more central processing units (CPUs) and the one or more GPUs.
19. (Currently Amended) The method of claim 14,
wherein the destination device comprises the one or more NUMA storages
or one or more of the GPUs.
14. (Currently Amended) A method, comprising:
in response to an application programming interface (API) call, causing information stored using one or more virtual memory addresses accessible one or more non-uniform memory access (NUMA) nodes and one or more graphics processor units (GPUs) to be stored in storage of a NUMA node of the one or more NUMA nodes indicated by one or more parameters of the API.
Modukuri ¶ [0075] states “In at least one embodiment, API reads data into GPU memory using dynamic data transfer routing. Modukuri ¶ [0049] states “a first graphics processing unit (GPU) 128, designated at GPU 0, is coupled with third PCIe switch 124, and a second GPU 130, designated as GPU 1,”
20. (Currently Amended) A non-transitory computer-readable medium having stored thereon a set of instructions, which if performed by one or more processors, cause one or more processors to at least perform the method of claim 14.
20. (Original) A non-transitory computer-readable medium having stored thereon a set of instructions, which if performed by one or more processors, cause the one or more processors to at least perform the method of claim 14.
Although the claims at issue are not identical, they are not patentably distinct from each other because ‘449 covers the majority the scope of the instant application using identical or synonymous terms. For example, indicators and parameters are interpreted as the same. Other identical features include having virtual memory be accessible by NUMA nodes and GPUs, storing information, and parameters indicating a NUMA node.
Regarding claims 14, 16, and 18, the claims are rejected because of identical or similar language to claims 9, 9, and 10 of ‘449, respectively. They are not patentably distinct as they only differ in statutory category of invention.
Regarding claims 1-20, ‘449 does not cover the scope related to reading information. However, Modukuri teaches reading information. It would have been obvious to a person having ordinary skill in the art prior to the effective filing date of the claimed invention to substitute the storing of information of ‘449 with the reading of information of Modukuri. Modukuri states that APIs can read or write data (¶ [0071] states “In at least one embodiment API includes APIs for operations such as read or write of data”). A person having ordinary skill in the art would have been motivated to make this simple substitution, with a reasonable expectation of success, as this would merely substitute storing, or writing, information with reading information. The results of reading information instead of writing information would have been predictable and obvious that information could either be read or written.
Regarding claims 2 and 8, ‘449 does not cover the scope related to the arguments including a pointer and a size. However, Modukuri teaches these arguments. It would have been obvious to a person having ordinary skill in the art prior to the effective filing date of the claimed invention to combine the pointer and size parameter of Modukuri with the API of storing information of ‘449. A person having ordinary skill in the art would have been motivated to make this combination to provide more information related to the memory being read so that the function has more utility. The results of using these parameters would have been predictable and obvious, namely that a range of memory could be specified for each read operation.
Regarding claim 4, ‘449 does not cover the scope of prefetching data. However, Modukuri teaches prefetching data. It would have been obvious to a person having ordinary skill in the art prior to the effective filing date of the claimed invention to combine the prefetching of data of Modukuri with the API of strong information of ‘449. A person having ordinary skill in the art would have been motivated to make this combination because one of ordinary skill in the art recognizes the performance benefits of prefetching data.
Regarding claim 19, ‘449 does not cover the scope a destination device of multiple GPUs. However, Modukuri teaches an environment with multiple GPUs. It would have been obvious to a person having ordinary skill in the art prior to the effective filing date of the claimed invention to combine the environment of multiple GPUs of Modukuri with the API of storing data of ‘449. A person having ordinary skill in the art would have been motivated to make this combination because one of ordinary skill in the art would recognize the performance benefits of having multiple GPUs execute in parallel and accelerate specific GPU workloads.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-4, 6-10, 12-14, 16, and 18-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Modukuri et al. Pat. No. US 20210286752 A1 (hereafter Modukuri) in view of Dugast et al. Pat. No. US 20220050722 A1 (hereafter Dugast) and further in view of Roberts Pat. No. US 20220342568 A1 (hereafter Roberts).
With regard to claim 1, Modukuri teaches one or more processors, comprising: circuitry to (¶ [0053] states “an application (e.g., running on first CPU 102, second CPU 104, first GPU 128, and/or second GPU 130) sends an IO function call (e.g., specifying a read or write operation) via an API (e.g., running on first CPU 102, second CPU 104, first GPU 128, and/or second GPU 130)”):
in response to receiving an application programming interface (API) call comprising arguments including a virtual memory address and a destination device (¶ [0076] states “cuFileWrite is specified as: ssize_t cuFileWrite(CUFileHandle fh, void *devPtr, size_t size, off_t offset), where fh is a file descriptor for a file, devPtr is a start address of a device pointer to read from, size is a size in bytes to write, and offset is an offset in a file to write to. In at least one embodiment, based on cuFileWrite call, API writes specified bytes from device memory into a file descriptor using dynamic data transfer routing.” ¶ [0172] states “When performing graphics operations, an effective address 1693 generated by a graphics processing engine is translated to a real address by MMU 1639” and ¶ [0179] states “one or more MMU(s) 1720A-1720B provide for virtual to physical address mapping for graphics processor 1710”. Examiner’s Note: the “effective address” is interpreted to be similar to the “virtual address”. Modukuri teaches a MMU mapping between virtual and physical memory of a graphics processor, so memory addresses can either be virtual or physical. The file descriptor and the offset together represent a destination device because both are used to determine where data should be written to):
cause information to be read from one or more non-uniform memory access (NUMA) storages or one or more graphics processor unit (GPU) physical storages corresponding to the virtual memory address and accessible by one or more central processing units (CPUs) and one or more GPUs (¶ [0076] states “cuFileWrite is specified as: ssize_t cuFileWrite(CUFileHandle fh, void *devPtr, size_t size, off_t offset), where fh is a file descriptor for a file, devPtr is a start address of a device pointer to read from, size is a size in bytes to write, and offset is an offset in a file to write to. In at least one embodiment, based on cuFileWrite call, API writes specified bytes from device memory into a file descriptor using dynamic data transfer routing.” ¶ [0078] states “an API (e.g., running on first CPU 202 or second CPU 204) receives a function call from an application to transfer a set of data (e.g., a write call) from first GPU 228 (e.g., from a source memory) to second set of storage devices 297.” See FIG. 1a. Examiner’s Note: devPtr points to an address of a device pointer to read from. GPUs are an example device in system 100. );
and cause the information to be stored at a physical memory of the destination device (¶ [0076] states “In at least one embodiment, based on cuFileWrite call, API writes specified bytes from device memory into a file descriptor using dynamic data transfer routing.” ¶ [0078] states “In at least one embodiment, an API (e.g., running on first CPU 202 or second CPU 204) receives a function call from an application to transfer a set of data (e.g., a write call) from first GPU 228 (e.g., from a source memory) to second set of storage devices 297.” ¶ [0204] states “include an MMU 1945 that is configured to map virtual addresses into physical addresses.”).
Modukuri does not explicitly teach read information from NUMA storages and reading data.
However, in an analogous art, Dugast teaches cause information to be read from one or more non-uniform memory access (NUMA) storages or one or more graphics processor unit (GPU) physical storages corresponding to the virtual memory address and accessible by one or more central processing units (CPUs) and one or more GPUs (¶ [0039] states “For example, memory pool 302 can provide a local memory pool (to the processors that execute process A and B) with a highest data access rate (e.g., read and/or write rate), but have a smallest size (e.g., amount of data that can read or written) and largest cost relative to memory pools 304 and 306. Memory pool 302 can represent a first non-uniform memory access (NUMA) node”. ¶ [0040] states “A second NUMA node can represent a different NUMA domain than that of the first NUMA node”. Examiner’s Notes: the environment can have a plurality of NUMA nodes. Processes can access NUMA nodes to read or write information).
It would have been obvious to a person having ordinary skill in the art prior to the effective filing date to combine the data access from a NUMA node of Dugast with the API of Modukuri. As a result, the API of Modukuri includes a parameter devPtr that refers to either a GPU memory or a NUMA storage. A person having ordinary skill in the art would have been motivated to make this combination for the purpose of optimizing data placement for performance benefits (Dugast ¶ [0002] states “Frequently accessed memory pages can be stored in local memory of a compute node whereas less frequently accessed memory pages can be stored in a more distant memory pool. Memory pools have different latency characteristics relative to a compute node whereby the compute node can write or read from some memory pools faster than other memory pools”). Improving performance and resource utilization also contributes to meeting service level agreements (Dugast ¶ [0017]).
Although Modukuri describes translating between effective and physical memory addresses (¶ [0179]), Modukuri and Dugast do not explicitly teach the virtual memory addresses corresponding with devices.
However, in an analogous art, Roberts teaches in response to receiving an application programming interface (API) call comprising arguments including a virtual memory address and a destination device (¶ [0020] states “BLOCK_PREFETCH(addr_src, size, tier) can represent a processor instruction with parameters addr_src, size, and tier, where the processor instruction causes generation of one or more memory instructions to a memory sub-system to prefetch (e.g., copy) data (e.g., data block), beginning at a source memory address (e.g., source virtual memory address)”):
cause information to be read from one or more non-uniform memory access (NUMA) storages or one or more graphics processor unit (GPU) physical storages corresponding to the virtual memory address and accessible by one or more central processing units (CPUs) and one or more GPUs (¶ [0020] states “BLOCK_PREFETCH(addr_src, size, tier) can represent a processor instruction with parameters addr_src, size, and tier, where the processor instruction causes generation of one or more memory instructions to a memory sub-system to prefetch (e.g., copy) data (e.g., data block), beginning at a source memory address (e.g., source virtual memory address)”)
It would have been obvious to a person having ordinary skill in the art prior to the effective filing date to combine the source address being a virtual memory address of Roberts with the API of Modukuri and the accessing of data from a NUMA node of Dugast. As a result, data is read from source GPUs and NUMA nodes using a virtual address or effective address associated with the device. A person having ordinary skill in the art would have been motivated to make this combination to “facilitate movement or migration of data across memory tiers while avoiding the overhead of operating system invocation (e.g., via API calls), page table manipulation, use of translational lookaside buffers (TLBs), or page table walks, which are otherwise needed by conventional technologies to facilitate copying, moving, or swapping data across through support of an operating system of a host system” (¶ [0014]).
With regard to claim 2, Modukuri, Dugast, and Roberts teach the one or more processors of claim 1. Roberts additionally teaches wherein the arguments include one or more pointers to the virtual memory address (¶ [0020] states “BLOCK_PREFETCH(addr_src, size, tier) can represent a processor instruction with parameters addr_src, size, and tier, where the processor instruction causes generation of one or more memory instructions to a memory sub-system to prefetch (e.g., copy) data (e.g., data block), beginning at a source memory address (e.g., source virtual memory address) specified by a add_src parameter.” Examiner’s Note: the source virtual memory address is itself a pointer).
With regard to claim 3, Modukuri, Dugast, and Roberts teach the one or more processors of claim 1. Modukuri additionally teaches wherein the API is to cause the information to be stored within one or more of the one or more NUMA storages (¶ [0076] states “In at least one embodiment, based on cuFileWrite call, API writes specified bytes from device memory into a file descriptor using dynamic data transfer routing.” ¶ [0078] states “In at least one embodiment, an API (e.g., running on first CPU 202 or second CPU 204) receives a function call from an application to transfer a set of data (e.g., a write call) from first GPU 228 (e.g., from a source memory) to second set of storage devices 297.”).
Dugast additionally teaches wherein the API is to cause the information to be stored within one or more of the one or more NUMA storages (¶ [0039] states “memory pool 302 can provide a local memory pool (to the processors that execute process A and B) with a highest data access rate (e.g., read and/or write rate), but have a smallest size (e.g., amount of data that can read or written) and largest cost relative to memory pools 304 and 306. Memory pool 302 can represent a first non-uniform memory access (NUMA) node”).
With regard to claim 4, Modukuri, Dugast, and Roberts teach the one or more processors of claim 1. Modukuri additionally teaches wherein the arguments include information indicating where prefetched data is to be stored in one of the one or more GPUs (¶ [0075] states “In at least one embodiment, cuFileRead is specified as: ssize_t cuFileRead (CUFileHandle fh, void *devPtr, size_t size, off_t offset), where fh is a file descriptor for a file, devPtr is a start address of a device pointer to read into, size is a size in bytes to read, and offset is an offset in a file to read from” and “In at least one embodiment, API reads data into GPU memory using dynamic data transfer routing.” ¶ [0055] states “In at least one embodiment, a file system, block system, object system, or key-value store system, either in an operating system or a standalone driver, performs a read-ahead operation to accomplish prefetching using an implementation similar to that of transfers initiated with cuFile-based API calls. In at least one embodiment, at least one processor moves data based, at least in part, on an explicit prefetch (e.g., a CUDA cudaMemPrefetch).” Examiner’s Note: devPtr is used to read data into GPU memory. Data is prefeteched using cudaMemPrefetch).
It would have been obvious to a person having ordinary skill in the art prior to the effective filing date to combine the devPtr parameter indicating a destination of where to read data into with the API specifying a different devPtr to read data from. Additionally, data can be prefetched using the cudaMemPrefetch. A person having ordinary skill in the art would have been motivated to make this combination for the purpose to transfer data between GPU memories. Modukuri suggests this by stating “communication path 292 includes a set of switches that enables data communication between any GPU in first set of GPUs 224 and any other GPU in first set of GPUs 224” (¶ [0068]). Modukuri ¶ [0061] additionally states “GPU-GPU link 176 is a memory fabric (e.g., Nvidia NVLink, Intel compute express link (CXL), AMD Infinity) rather than a network fabric, and allows compute elements in a first OS instance to directly perform loads, stores, and atomic operations in memory controlled by another OS instance.” By using a memory fabric, the data transfer bandwidth between GPUs is increased.
With regard to claim 6, Modukuri, Dugast, and Roberts teach the one or more processors of claim 1. Modukuri additionally teaches wherein causing the information to be stored at the physical memory of the destination device comprises copying the information to the physical memory of the destination device (¶ [0055] states “In at least one embodiment, API allocates additional memory as part of data transfer operations (e.g., read/write operations such as cuFileRead or cuFileWrite).” ¶ [0110] states “performing data transfer includes one or more copy operations.”).
With regard to claim 7, Modukuri, Dugast, and Roberts teach the one or more processors of claim 1. Modukuri additionally teaches wherein causing the information to be stored at the physical memory of the destination device comprises moving the information to the physical memory of the destination device (¶ [0055] states “In at least one embodiment, API allocates additional memory as part of data transfer operations (e.g., read/write operations such as cuFileRead or cuFileWrite).”).
With regard to claim 8, Modukuri teaches a system, comprising (¶ [0048] states “FIG. 1A is a block diagram illustrating a computer system 100):
one or more graphics processor units (GPUs) (¶ [0049] states “In at least one embodiment, a first graphics processing unit (GPU) 128, designated at GPU 0, is coupled with third PCIe switch 124, and a second GPU 130, designated as GPU 1, is coupled with fourth PCIe switch 126.”);
and one or more processors to (¶ [0048] states “computer system 100 includes a first central processing unit (CPU) 102 in a first CPU socket, designated as CPU Socket 0, and a second CPU 104 in a second CPU socket, designated as CPU Socket 1.”):
in response to receiving an application programming interface (API) call comprising arguments including a virtual memory address and a destination device (¶ [0076] states “cuFileWrite is specified as: ssize_t cuFileWrite(CUFileHandle fh, void *devPtr, size_t size, off_t offset), where fh is a file descriptor for a file, devPtr is a start address of a device pointer to read from, size is a size in bytes to write, and offset is an offset in a file to write to. In at least one embodiment, based on cuFileWrite call, API writes specified bytes from device memory into a file descriptor using dynamic data transfer routing.” ¶ [0172] states “When performing graphics operations, an effective address 1693 generated by a graphics processing engine is translated to a real address by MMU 1639” and ¶ [0179] states “one or more MMU(s) 1720A-1720B provide for virtual to physical address mapping for graphics processor 1710”. Examiner’s Note: the “effective address” is interpreted to be similar to the “virtual address”. Modukuri teaches a MMU mapping between virtual and physical memory of a graphics processor, so memory addresses can either be virtual or physical. The file descriptor and the offset together represent a destination device because both are used to determine where data should be written to):
cause information to be read from one or more non-uniform memory access (NUMA) storages or one or more graphics processor unit (GPU) physical storages corresponding to the virtual memory address and accessible by one or more central processing units (CPUs) and one or more GPUs (¶ [0076] states “cuFileWrite is specified as: ssize_t cuFileWrite(CUFileHandle fh, void *devPtr, size_t size, off_t offset), where fh is a file descriptor for a file, devPtr is a start address of a device pointer to read from, size is a size in bytes to write, and offset is an offset in a file to write to. In at least one embodiment, based on cuFileWrite call, API writes specified bytes from device memory into a file descriptor using dynamic data transfer routing.” ¶ [0078] states “an API (e.g., running on first CPU 202 or second CPU 204) receives a function call from an application to transfer a set of data (e.g., a write call) from first GPU 228 (e.g., from a source memory) to second set of storage devices 297.” See FIG. 1a. Examiner’s Note: devPtr points to an address of a device pointer to read from. GPUs are an example device in system 100.);
and cause the information to be stored at a physical memory of the destination device (¶ [0076] states “In at least one embodiment, based on cuFileWrite call, API writes specified bytes from device memory into a file descriptor using dynamic data transfer routing.” ¶ [0078] states “In at least one embodiment, an API (e.g., running on first CPU 202 or second CPU 204) receives a function call from an application to transfer a set of data (e.g., a write call) from first GPU 228 (e.g., from a source memory) to second set of storage devices 297.”).
Modukuri does not explicitly teach read information from NUMA storages and reading data.
However, in an analogous art, Dugast teaches cause information to be read from one or more non-uniform memory access (NUMA) storages or one or more graphics processor unit (GPU) physical storages corresponding to the virtual memory address and accessible by one or more central processing units (CPUs) and one or more GPUs (¶ [0039] states “For example, memory pool 302 can provide a local memory pool (to the processors that execute process A and B) with a highest data access rate (e.g., read and/or write rate), but have a smallest size (e.g., amount of data that can read or written) and largest cost relative to memory pools 304 and 306. Memory pool 302 can represent a first non-uniform memory access (NUMA) node”. ¶ [0040] states “A second NUMA node can represent a different NUMA domain than that of the first NUMA node”. Examiner’s Note: the environment can have a plurality of NUMA nodes. Processes can access NUMA nodes to read or write information).
It would have been obvious to a person having ordinary skill in the art prior to the effective filing date to combine the data access from a NUMA node of Dugast with the API of Modukuri. As a result, the API of Modukuri includes a parameter devPtr that refers to either a GPU memory or a NUMA storage. A person having ordinary skill in the art would have been motivated to make this combination for the purpose of optimizing data placement for performance benefits (Dugast ¶ [0002] states “Frequently accessed memory pages can be stored in local memory of a compute node whereas less frequently accessed memory pages can be stored in a more distant memory pool. Memory pools have different latency characteristics relative to a compute node whereby the compute node can write or read from some memory pools faster than other memory pools”). Improving performance and resource utilization also contributes to meeting service level agreements (Dugast ¶ [0017]).
Although Modukuri describes translating between effective and physical memory addresses (¶ [0179]), Modukuri and Dugast do not explicitly teach the virtual memory addresses corresponding with devices.
However, in an analogous art, Roberts teaches in response to receiving an application programming interface (API) call comprising arguments including a virtual memory address and a destination device (¶ [0020] states “BLOCK_PREFETCH(addr_src, size, tier) can represent a processor instruction with parameters addr_src, size, and tier, where the processor instruction causes generation of one or more memory instructions to a memory sub-system to prefetch (e.g., copy) data (e.g., data block), beginning at a source memory address (e.g., source virtual memory address)”):
cause information to be read from one or more non-uniform memory access (NUMA) storages or one or more graphics processor unit (GPU) physical storages corresponding to the virtual memory address and accessible by one or more central processing units (CPUs) and one or more GPUs (¶ [0020] states “BLOCK_PREFETCH(addr_src, size, tier) can represent a processor instruction with parameters addr_src, size, and tier, where the processor instruction causes generation of one or more memory instructions to a memory sub-system to prefetch (e.g., copy) data (e.g., data block), beginning at a source memory address (e.g., source virtual memory address)”)
It would have been obvious to a person having ordinary skill in the art prior to the effective filing date to combine the source address being a virtual memory address of Roberts with the API of Modukuri and the accessing of data from a NUMA node of Dugast. As a result, data is read from source GPUs and NUMA nodes using a virtual address or effective address associated with the device. A person having ordinary skill in the art would have been motivated to make this combination to “facilitate movement or migration of data across memory tiers while avoiding the overhead of operating system invocation (e.g., via API calls), page table manipulation, use of translational lookaside buffers (TLBs), or page table walks, which are otherwise needed by conventional technologies to facilitate copying, moving, or swapping data across through support of an operating system of a host system” (¶ [0014]).
With regard to claim 9, Modukuri, Dugast, and Roberts teach the system of claim 8. Dugast additionally teaches wherein the destination device comprises one or more of the NUMA storages (¶ [0039] states “For example, memory pool 302 can provide a local memory pool (to the processors that execute process A and B) with a highest data access rate (e.g., read and/or write rate), but have a smallest size (e.g., amount of data that can read or written) and largest cost relative to memory pools 304 and 306. Memory pool 302 can represent a first non-uniform memory access (NUMA) node”. ¶ [0040] states “A second NUMA node can represent a different NUMA domain than that of the first NUMA node”. Examiner’s Note: the environment can have a plurality of NUMA nodes. Processes can access NUMA nodes to read or write information).
With regard to claim 10, Modukuri, Dugast, and Roberts teach the system of claim 8. Modukuri additionally teaches wherein the arguments include information indicating a size of the data that is to be stored at the physical memory of the destination device (¶ [0076] states “cuFileWrite is specified as: ssize_t cuFileWrite(CUFileHandle fh, void *devPtr, size_t size, off_t offset), where fh is a file descriptor for a file, devPtr is a start address of a device pointer to read from, size is a size in bytes to write”).
With regard to claim 12, Modukuri, Dugast, and Roberts teach the system of claim 8. Dugast additionally teaches wherein the API is to cause the information to be read from a NUMA storage of the one or more NUMA storages associated with the one or more CPUs (¶ [0039] states “For example, memory pool 302 can provide a local memory pool (to the processors that execute process A and B) with a highest data access rate (e.g., read and/or write rate), but have a smallest size (e.g., amount of data that can read or written) and largest cost relative to memory pools 304 and 306. Memory pool 302 can represent a first non-uniform memory access (NUMA) node”. ¶ [0040] states “A second NUMA node can represent a different NUMA domain than that of the first NUMA node”. ¶ [0026] states “A memory pool node 150 can include a network or fabric interface 152, processors 154 (e.g., CPUs, GPUs, accelerators, and so forth).” Examiner’s Note: memory pools provide memory to processors that execute processes A and B. Processors are either CPUs, GPUs, accelerators, and so forth.).
With regard to claim 13, Modukuri, Dugast, and Roberts teach the system of claim 8. Dugast additionally teaches wherein one or more physical storages corresponding to the virtual memory address are further accessible by a plurality of NUMA nodes (¶ [0026] states “A memory pool node 150 can include a network or fabric interface 152, processors 154 (e.g., CPUs, GPUs, accelerators, and so forth), and memory pool resources 156 (e.g., memory, persistent memory, storage, and/or cache)” and “one or more of memory pool nodes 150-0 to 150-N can be coupled to network interface 106 using a network interface and one or more switches or routers.” Examiner’s Note: memory pool resources are physical storages and are accessible to the NUMA node).
Roberts additionally teaches wherein one or more physical storages corresponding to the virtual memory address are further accessible by a plurality of NUMA nodes (¶ [0017] states “The source memory address, the target memory address, or both can be a virtual memory address, which can be translated by the hardware processor of the host system into one or more memory addresses (e.g., intermediate memory addresses) used by the memory sub-system with one or more memory instructions (of an embodiment) sent from the host system to the memory sub-system, where the one or more memory addresses received by the memory sub-system can be mapped to one or more memory locations (e.g., physical memory locations) on one or more memory devices of the memory sub-system.” ¶ [0013] states “data movement/migration/placement across memory tiers often involves a software application invoking the operating system (e.g., via Application Program Interface (API) calls), manipulating page table, using translational lookaside buffers (TLBs), or walking a page table. An existing example of this is non-uniform memory access (NUMA)”).
It would have been obvious to a person having ordinary skill in the art prior to the effective filing date to combine the physical memory addresses corresponding to virtual memory addresses of Roberts with the memory pool nodes of Dugast and the API of Modukuri. As a result, NUMA nodes access the physical storages that correspond to virtual memory addresses by translating between virtual and physical memory. A person having ordinary skill in the art would have been motivated to make this combination to “facilitate movement or migration of data across memory tiers while avoiding the overhead of operating system invocation (e.g., via API calls), page table manipulation, use of translational lookaside buffers (TLBs), or page table walks, which are otherwise needed by conventional technologies to facilitate copying, moving, or swapping data across through support of an operating system of a host system” (¶ [0014]).
With regard to claim 14, Modukuri teaches a method, comprising (¶ [0076] states “API receives a write function call (e.g., cuFileWrite) and performs dynamic data transfer routing based, at least in part, on received write function call.”):
receiving an application programming interface (API) call comprising arguments including a virtual memory address and a destination device (¶ [0076] states “cuFileWrite is specified as: ssize_t cuFileWrite(CUFileHandle fh, void *devPtr, size_t size, off_t offset), where fh is a file descriptor for a file, devPtr is a start address of a device pointer to read from, size is a size in bytes to write, and offset is an offset in a file to write to. In at least one embodiment, based on cuFileWrite call, API writes specified bytes from device memory into a file descriptor using dynamic data transfer routing.” ¶ [0172] states “When performing graphics operations, an effective address 1693 generated by a graphics processing engine is translated to a real address by MMU 1639” and ¶ [0179] states “one or more MMU(s) 1720A-1720B provide for virtual to physical address mapping for graphics processor 1710”. Examiner’s Note: the “effective address” is interpreted to be similar to the “virtual address”. Modukuri teaches a MMU mapping between virtual and physical memory of a graphics processor, so memory addresses can either be virtual or physical. The file descriptor and the offset together represent a destination device because both are used to determine where data should be written to);
causing, in response to the API call, information to be read from one or more non-uniform memory access (NUMA) storages or one or more graphics processor unit (GPU) physical storages corresponding to the virtual memory address and accessible by one or more central processing units (CPUs) and the one or more GPUs (¶ [0076] states “cuFileWrite is specified as: ssize_t cuFileWrite(CUFileHandle fh, void *devPtr, size_t size, off_t offset), where fh is a file descriptor for a file, devPtr is a start address of a device pointer to read from, size is a size in bytes to write, and offset is an offset in a file to write to. In at least one embodiment, based on cuFileWrite call, API writes specified bytes from device memory into a file descriptor using dynamic data transfer routing.” ¶ [0078] states “an API (e.g., running on first CPU 202 or second CPU 204) receives a function call from an application to transfer a set of data (e.g., a write call) from first GPU 228 (e.g., from a source memory) to second set of storage devices 297.” See FIG. 1a. Examiner’s Note: devPtr points to an address of a device pointer to read from. GPUs are an example device in system 100.);
and causing the information to be stored at a physical memory of the destination device (¶ [0076] states “In at least one embodiment, based on cuFileWrite call, API writes specified bytes from device memory into a file descriptor using dynamic data transfer routing.” ¶ [0078] states “In at least one embodiment, an API (e.g., running on first CPU 202 or second CPU 204) receives a function call from an application to transfer a set of data (e.g., a write call) from first GPU 228 (e.g., from a source memory) to second set of storage devices 297.”).
Modukuri does not explicitly teach read information from NUMA storages and reading data.
However, in an analogous art, Dugast teaches causing, in response to the API call, information to be read from one or more non-uniform memory access (NUMA) storages or one or more graphics processor unit (GPU) physical storages corresponding to the virtual memory address and accessible by one or more central processing units (CPUs) and the one or more GPUs (¶ [0039] states “For example, memory pool 302 can provide a local memory pool (to the processors that execute process A and B) with a highest data access rate (e.g., read and/or write rate), but have a smallest size (e.g., amount of data that can read or written) and largest cost relative to memory pools 304 and 306. Memory pool 302 can represent a first non-uniform memory access (NUMA) node”. ¶ [0040] states “A second NUMA node can represent a different NUMA domain than that of the first NUMA node”. Examiner’s Note: the environment can have a plurality of NUMA nodes. Processes can access NUMA nodes to read or write information).
It would have been obvious to a person having ordinary skill in the art prior to the effective filing date to combine the data access from a NUMA node of Dugast with the API of Modukuri. As a result, the API of Modukuri includes a parameter devPtr that refers to either a GPU memory or a NUMA storage. A person having ordinary skill in the art would have been motivated to make this combination for the purpose of optimizing data placement for performance benefits (Dugast ¶ [0002] states “Frequently accessed memory pages can be stored in local memory of a compute node whereas less frequently accessed memory pages can be stored in a more distant memory pool. Memory pools have different latency characteristics relative to a compute node whereby the compute node can write or read from some memory pools faster than other memory pools”). Improving performance and resource utilization also contributes to meeting service level agreements (Dugast ¶ [0017]).
Although Modukuri describes translating between effective and physical memory addresses (¶ [0179]), Modukuri and Dugast do not explicitly teach the virtual memory addresses corresponding with devices.
However, in an analogous art, Roberts teaches receiving an application programming interface (API) call comprising arguments including a virtual memory address and a destination device
causing, in response to the API call, information to be read from one or more non-uniform memory access (NUMA) storages or one or more graphics processor unit (GPU) physical storages corresponding to the virtual memory address and accessible by one or more central processing units (CPUs) and the one or more GPUs (¶ [0020] states “BLOCK_PREFETCH(addr_src, size, tier) can represent a processor instruction with parameters addr_src, size, and tier, where the processor instruction causes generation of one or more memory instructions to a memory sub-system to prefetch (e.g., copy) data (e.g., data block), beginning at a source memory address (e.g., source virtual memory address)”).
It would have been obvious to a person having ordinary skill in the art prior to the effective filing date to combine the source address being a virtual memory address of Roberts with the API of Modukuri and the accessing of data from a NUMA node of Dugast. As a result, data is read from source GPUs and NUMA nodes using a virtual address or effective address associated with the device. A person having ordinary skill in the art would have been motivated to make this combination to “facilitate movement or migration of data across memory tiers while avoiding the overhead of operating system invocation (e.g., via API calls), page table manipulation, use of translational lookaside buffers (TLBs), or page table walks, which are otherwise needed by conventional technologies to facilitate copying, moving, or swapping data across through support of an operating system of a host system” (¶ [0014]).
With regard to claim 16, Modukuri, Dugast, and Roberts teach the method of claim 14. Modukuri additionally teaches wherein causing the information to be stored at the physical memory of the destination device comprises copying the information to the physical memory of the destination device (¶ [0055] states “In at least one embodiment, API allocates additional memory as part of data transfer operations (e.g., read/write operations such as cuFileRead or cuFileWrite).” ¶ [0110] states “performing data transfer includes one or more copy operations.”).
With regard to claim 18, Modukuri, Dugast, and Roberts teach the method of claim 14. Roberts additionally teaches wherein the arguments include a range of virtual memory addresses (¶ [0020] states “BLOCK_PREFETCH(addr_src, size, tier) can represent a processor instruction with parameters addr_src, size, and tier, where the processor instruction causes generation of one or more memory instructions to a memory sub-system to prefetch (e.g., copy) data (e.g., data block), beginning at a source memory address (e.g., source virtual memory address) specified by a add_src parameter and having a size specified (e.g., in number of bytes) by a size parameter.” Examiner’s Note: source address and size indicate a range of virtual memory addresses).
With regard to claim 19, Modukuri, Dugast, and Roberts teach the method of claim 14. Modukuri additionally teaches wherein the destination device comprises the one or more NUMA storages or one or more of the GPUs (¶ [0075] states “In at least one embodiment, cuFileRead is specified as: ssize_t cuFileRead (CUFileHandle fh, void *devPtr, size_t size, off_t offset), where fh is a file descriptor for a file, devPtr is a start address of a device pointer to read into, size is a size in bytes to read, and offset is an offset in a file to read from” and “In at least one embodiment, API reads data into GPU memory using dynamic data transfer routing.”).
It would have been obvious to a person having ordinary skill in the art prior to the effective filing date to combine the devPtr parameter indicating a destination of where to read data into with the API specifying a different devPtr to read data from. A person having ordinary skill in the art would have been motivated to make this combination for the purpose to transfer data between GPU memories. Modukuri suggests this by stating “communication path 292 includes a set of switches that enables data communication between any GPU in first set of GPUs 224 and any other GPU in first set of GPUs 224” (¶ [0068]). Modukuri ¶ [0061] additionally states “GPU-GPU link 176 is a memory fabric (e.g., Nvidia NVLink, Intel compute express link (CXL), AMD Infinity) rather than a network fabric, and allows compute elements in a first OS instance to directly perform loads, stores, and atomic operations in memory controlled by another OS instance.” By using a memory fabric, the data transfer bandwidth between GPUs is increased.
Dugast additionally teaches wherein the destination device comprises the one or more NUMA storages or one or more of the GPUs (¶ [0039] states “For example, memory pool 302 can provide a local memory pool (to the processors that execute process A and B) with a highest data access rate (e.g., read and/or write rate), but have a smallest size (e.g., amount of data that can read or written) and largest cost relative to memory pools 304 and 306. Memory pool 302 can represent a first non-uniform memory access (NUMA) node”. ¶ [0040] states “A second NUMA node can represent a different NUMA domain than that of the first NUMA node”. Examiner’s Note: the environment can have a plurality of NUMA nodes. Processes can access NUMA nodes to read or write information).
With regard to claim 20, Modukuri, Dugast, and Roberts teach the method of claim 14. Modukuri additionally teaches a non-transitory computer-readable medium having stored thereon a set of instructions, which if performed by one or more processors, cause the one or more processors to at least perform the method of claim 14 (¶ [0393] states “code (e.g., executable code or source code) is stored on a set of one or more non-transitory computer-readable storage media having stored thereon executable instructions (or other memory to store executable instructions) that, when executed (i.e., as a result of being executed) by one or more processors of a computer system, cause computer system to perform operations described herein”).
Claim(s) 5, 11, and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Modukuri in view of Dugast and Roberts and further in view of Wagle et al. Pat. No. US 20160371194 A1 (hereafter Wagle).
With regard to claim 5, Modukuri, Dugast, and Roberts teach the one or more processors of claim 1. Dugast additionally teaches wherein the arguments include information indicating where prefetched data is to be stored in one of the one or more NUMA storages (¶ [0039] states “memory pool 302 can provide a local memory pool (to the processors that execute process A and B) with a highest data access rate (e.g., read and/or write rate), but have a smallest size (e.g., amount of data that can read or written) and largest cost relative to memory pools 304 and 306. Memory pool 302 can represent a first non-uniform memory access (NUMA) node”. ¶ [0064] states “Some examples utilize artificial intelligence (AI)-based technologies to detect memory access patterns per clustered pages for a workload and use this information to determine whether to pre-fetch data to nearer memory to the processor that executes the workload or migrate data to further memory away from the processor that executes the workload". Examiner’s Note: information is read from or written to a NUMA node. Information is also be prefetched to a NUMA node).
Modukuri, Dugast, and Roberts do not explicitly teach an argument indicating where data is to be stored in the one or more NUMA storages.
However, in an analogous art, Wagle teaches wherein the arguments include information indicating where prefetched data is to be stored in one of the one or more NUMA storages (¶ [0033] states “As shown in FIG. 4, worker thread 405 calls “ALLOC_ON_NUMA_NODE (<0>)” to allocate memory addresses of memory 1124 (e.g., DRAM DIMMs) for use by cores 0, 4, 8 and 12. With reference to FIG. 1, memory 1124 and cores 0, 4, 8 and 12 are all located on the same node (i.e., node 0)”. Examiner’s Note: the ALLOC_ON_NUMA_NODE includes a parameter to specify which NUMA node to allocate memory on).
It would have been obvious to a person having ordinary skill in the art prior to the effective filing date to combine the indicator of a NUMA node of Wagle with the storing of prefetched data of Dugast and the API of Modukuri and virtual memory address of Roberts. A person having ordinary skill in the art would have been motivated to make this combination for the purpose of “[using] this information to determine whether to pre-fetch data to nearer memory to the processor that executes the workload or migrate data to further memory away from the processor that executes the workload" (¶ [0064]). Placement decisions are important to overcoming memory fragmentation and the associated remote access penalties of accessing memory within another NUMA node (Wagle ¶ [0025] states that “with multiple per-CPU allocators working on the memory (DRAM) of a single NUMA node, memory fragmentation or remote access penalties are possible” and ¶ [0027] states “preferred: Try to allocate on a node first”. By setting the policy to preferred, memory will be allocated on the same NUMA node that the thread is executing in).
With regard to claim 11, Modukuri, Dugast, and Roberts teach the system of claim 8. Modukuri, Dugast, and Roberts do not explicitly teach an argument including an identifier of a particular NUMA storage of the one or more NUMA storages.
However, in an analogous art, Wagle additionally teaches wherein the arguments include an identifier of a particular NUMA storage of the one or more NUMA storages (¶ [0033] states “As shown in FIG. 4, worker thread 405 calls “ALLOC_ON_NUMA_NODE (<0>)” to allocate memory addresses of memory 1124 (e.g., DRAM DIMMs) for use by cores 0, 4, 8 and 12. With reference to FIG. 1, memory 1124 and cores 0, 4, 8 and 12 are all located on the same node (i.e., node 0)”. Examiner’s Note: the ALLOC_ON_NUMA_NODE includes a parameter to specify which NUMA node to allocate memory on).
It would have been obvious to a person having ordinary skill in the art prior to the effective filing date to combine the indicator of a NUMA node of Wagle with the API of Modukuri, the accessing of NUMA nodes of Dugast, and the virtual memory addresses of Roberts. A person having ordinary skill in the art would have been motivated to make this combination for the purpose of “[using] this information to determine whether to pre-fetch data to nearer memory to the processor that executes the workload or migrate data to further memory away from the processor that executes the workload" (¶ [0064]). Placement decisions are important to overcoming memory fragmentation and the associated remote access penalties of accessing memory within another NUMA node (Wagle ¶ [0025] states that “with multiple per-CPU allocators working on the memory (DRAM) of a single NUMA node, memory fragmentation or remote access penalties are possible” and ¶ [0027] states “preferred: Try to allocate on a node first”. By setting the policy to preferred, memory will be allocated on the same NUMA node that the thread is executing in).
With regard to claim 17, Modukuri, Dugast, and Roberts teach the method of claim 14. Modukuri, Dugast, and Roberts do not explicitly teach an argument including an identifier of a particular NUMA storage of the one or more NUMA storages.
Wagle additionally teaches wherein the arguments include an identifier of a NUMA node indicating where the information is to be stored within the one or more NUMA storages (¶ [0033] states “As shown in FIG. 4, worker thread 405 calls “ALLOC_ON_NUMA_NODE (<0>)” to allocate memory addresses of memory 1124 (e.g., DRAM DIMMs) for use by cores 0, 4, 8 and 12. With reference to FIG. 1, memory 1124 and cores 0, 4, 8 and 12 are all located on the same node (i.e., node 0)”. Examiner’s Note: the ALLOC_ON_NUMA_NODE includes a parameter to specify which NUMA node to allocate memory on).
It would have been obvious to a person having ordinary skill in the art prior to the effective filing date to combine the indicator of a NUMA node of Wagle with the API of Modukuri, the accessing of NUMA nodes of Dugast, and the virtual memory addresses of Roberts. A person having ordinary skill in the art would have been motivated to make this combination for the purpose of “[using] this information to determine whether to pre-fetch data to nearer memory to the processor that executes the workload or migrate data to further memory away from the processor that executes the workload" (¶ [0064]). Placement decisions are important to overcoming memory fragmentation and the associated remote access penalties of accessing memory within another NUMA node (Wagle ¶ [0025] states that “with multiple per-CPU allocators working on the memory (DRAM) of a single NUMA node, memory fragmentation or remote access penalties are possible” and ¶ [0027] states “preferred: Try to allocate on a node first”. By setting the policy to preferred, memory will be allocated on the same NUMA node that the thread is executing in).
Claim(s) 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Modukuri in view of Dugast and Roberts and further in view of Eickemeyer et al. Pat. No. US 20170147493 A1 (hereafter Eickemeyer).
With regard to claim 15, Modukuri, Dugast, and Roberts teach the method of claim 14. Modukuri, Dugast, and Roberts do not explicitly teach a stream identifier.
However, in an analogous art, Eickemeyer teaches wherein the arguments include an identifier of one or more streams corresponding to the information (¶ [0027] states “As shown, the prefetch request 300 includes request data 305 and a confidence level 315. The request data 305 may specify information associated with the request, such as a data address, thread identifier, stream identifier, and a prefetch source.” Examiner’s Note: to create the request, the data address, thread identifier, stream identifier, and prefetch source are considered arguments to a function that creates the request).
It would have been obvious to a person having ordinary skill in the art prior to the effective filing date to combine the prefetch request with a stream identifier of Eickemeyer with the API with arguments of Modukuri, the accessing of NUMA nodes of Dugast, and the virtual memory address of Roberts. As a result, the API of Modukuri includes an argument that is the stream identifier of the prefetch request of Eickemeyer. A person having ordinary skill in the art would have been motivated to make this combination “to optimize prefetch efficiency relative to current memory resource utilization and constraints. By assigning a confidence level to each prefetch request in a stream based on depth of the stream, the prefetcher prioritizes streams that are likely to benefit from prefetching and streams where prefetching does not result in much performance benefit. Accordingly, the prefetcher may adaptively adjust rates at which prefetching for a given stream occurs. In addition, the memory controller can determine, based on confidence level, prefetch requests to drop to conserve memory usage bandwidth” (¶ [0017]).
Response to Arguments
The nonstatutory double patenting rejection with copending application 18/214,447 in view of Modukuri et al. Pub. No. US 20210286752 A1 has been withdrawn.
Applicant's arguments filed 05/06/2026 have been fully considered but they are not persuasive.
With regard to the 35 U.S.C. § 103, applicant argues that Modukuri does not teach “at least "in response to receiving an application programming interface (API) call comprising arguments including a virtual memory address and a destination device: cause information to be read from one or more [NUMA] storages or one or more [GPU] storages corresponding to the virtual memory address.... and cause the information to be stored at a physical memory of the destination device," as claimed.”
In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986).
Examiner finds applicant’s argument unpersuasive. Examiner does not solely rely on Modukuri to teach the entirety of the claim. Rather, it is the combination of Modukuri, Dugast, and Roberts that fully teach claim 1. Modukuri teaches the API call comprising arguments including a memory address indicating a source address and a file descriptor and offset which represent a destination device (¶ [0076] states “cuFileWrite is specified as: ssize_t cuFileWrite(CUFileHandle fh, void *devPtr, size_t size, off_t offset), where fh is a file descriptor for a file, devPtr is a start address of a device pointer to read from, size is a size in bytes to write, and offset is an offset in a file to write to.”). The information is written to the location indicated by the file descriptor and offset. Modukuri ¶ [0204] teaches an MMU that maps virtual addresses to physical addresses, so it understood that data is stored at a physical address. Dugast teaches memory pools that are NUMA nodes and that allow data accesses (¶ [0039] states “For example, memory pool 302 can provide a local memory pool (to the processors that execute process A and B) with a highest data access rate (e.g., read and/or write rate), but have a smallest size (e.g., amount of data that can read or written) and largest cost relative to memory pools 304 and 306.). Lastly, Roberts explicitly teaches that the source address is a virtual memory address (¶ [0020] states “BLOCK_PREFETCH(addr_src, size, tier) can represent a processor instruction with parameters addr_src, size, and tier, where the processor instruction causes generation of one or more memory instructions to a memory sub-system to prefetch (e.g., copy) data (e.g., data block), beginning at a source memory address (e.g., source virtual memory address)”). See 35 U.S.C. § 103 of claim 1 for more details and rationales to combine. Together in combination, the references teach limitations of claim 1.
Examiner maintains 35 U.S.C. § 103 rejection of claim 1 and similar independent claims 8 and 14. Examiner maintains 35 U.S.C. § 103 rejection of dependent claims 2-7, 9-13, and 15-20.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PETER L YUAN whose telephone number is (571)272-5737. The examiner can normally be reached Mon-Fri 7:30am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Bradley Teets can be reached at 571-272-3338. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PETER LI YUAN/Examiner, Art Unit 2197
/BRADLEY A TEETS/Supervisory Patent Examiner, Art Unit 2197
Prosecution Insights