The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Claims 1-13 and 21-27 are presented for examination in this application (19/177,363) as per the response to election/restriction filed on 8/6/2026.
The Examiner cites particular sections in the references as applied to the claims below for the convenience of the applicant(s). Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested that, in preparing responses, the applicant(s) fully consider the references in their entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the Examiner.
Claims 1-13 and 21-27 are pending for consideration.
Drawings
The drawings submitted on April 11, 2025 have been considered and accepted.
Information Disclosure Statement
Acknowledgment is made of the information disclosure statements filed on August 6, 2026. U.S. patents and Foreign Patents have been considered.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 7-11, 21-22 and 25-27 are rejected under 35 U.S.C. 102(a)(1) and (a)(2) as being anticipated by Markthub et al. (US PGPUB 2023/0176741) (hereinafter ‘Markthub’).
As per independent claim 7, Markthub discloses a host system comprising: memory; and at least one processing device comprising a plurality of streaming multiprocessors [(Paragraphs 0004-0009) where Markthub teaches where the Communication protocols such as GDA-KI, which utilize GPUs instead of CPUs, leverage a GPU streaming multiprocessor (SM) to submit WQEs to the NIC. If traditional WQE submission algorithms are strictly followed, the GPU will need to (1) lock the network QP, which limits the concurrency, or (2) create one QP per thread, which may consume hundreds GB of GPU memory in real applications. In addition, each WQE submission will require issuing a memory barrier, which incurs significant latency for the GPU SMS to correspond to the claimed limitation], wherein: each streaming multiprocessor is configured to use a respective plurality of submission queues [(Paragraphs 0010-0012) where Markthub teaches where the multiple threads (from the same or different thread blocks (CTAs)) should be able to create WQEs in parallel and submit them to the same QP without locking. Also, the number of issued memory barriers should be reduceable (e.g., by issuing one memory barrier for submitting multiple WQEs). Embodiments of the present disclosure aim to improve communication efficiencies by increasing the parallelism of the process described above for threads within the same thread block. Embodiments of the present disclosure further contemplate improving communication efficiencies while working within the framework of existing post-send protocol(s). Aspects of the present disclosure may include two levels: (1) warp-level coalescing of WQE slot reservation and WQE creation and (2) coalescing of memory barriers and doorbell updates. Embodiments of the present disclosure are contemplated for use in an architecture having a scalable array of multithreaded SMs. Each SM may include a set of execution units, a set of registers, and a chunk of shared memory. In some embodiments, the basic unit of execution for a processing unit (e.g., a CPU or GPU) may be referred to as a warp. A warp may correspond to a collection of threads (e.g., 32 threads may belong to a warp) that are executed simultaneously by an SM. Multiple warps can be executed on an SM at once. When a program on a host CPU invokes a kernel grid, the blocks of the grid may be enumerated and distributed to SMs with available execution capacity. The threads of a thread block execute concurrently on one SM, and multiple thread blocks can execute concurrently on one SM. As thread blocks terminate, new blocks are launched on the vacated SMs to correspond to the claimed limitation]; and the processing device is configured to, for each submission queue, submit commands for processing by a memory sub-system in response to determining that a first batch of the commands has been written to the submission queue [(Paragraphs 0010-0012, 0028-0030 and 0101) where Markthub teaches where the multiple threads (from the same or different thread blocks (CTAs)) should be able to create WQEs in parallel and submit them to the same QP without locking. Also, the number of issued memory barriers should be reduceable (e.g., by issuing one memory barrier for submitting multiple WQEs). Embodiments of the present disclosure aim to improve communication efficiencies by increasing the parallelism of the process described above for threads within the same thread block. Embodiments of the present disclosure further contemplate improving communication efficiencies while working within the framework of existing post-send protocol(s). Aspects of the present disclosure may include two levels: (1) warp-level coalescing of WQE slot reservation and WQE creation and (2) coalescing of memory barriers and doorbell updates. Embodiments of the present disclosure are contemplated for use in an architecture having a scalable array of multithreaded SMs. Each SM may include a set of execution units, a set of registers, and a chunk of shared memory. In some embodiments, the basic unit of execution for a processing unit (e.g., a CPU or GPU) may be referred to as a warp. The processing unit 300 may transfer data 312 to the IO device 304 and/or receive data 312 from the IO device 304 using the WQ 316, the DBR 320, and a completion queue (CQ) 324. The WQ 316, DBR 320, and CQ 324 may enable the SMs 328 of the processing unit 300 to interact directly with the IO device 304 to correspond to the claimed limitation].
As per claim 8, Markthub discloses wherein a plurality of threads running on the each streaming multiprocessor are configured to write the commands into the respective plurality of submission queues [(Paragraphs 0010-0012, 0028-0030 and 0101) where Markthub teaches where the multiple threads (from the same or different thread blocks (CTAs)) should be able to create WQEs in parallel and submit them to the same QP without locking. Also, the number of issued memory barriers should be reduceable (e.g., by issuing one memory barrier for submitting multiple WQEs). Embodiments of the present disclosure aim to improve communication efficiencies by increasing the parallelism of the process described above for threads within the same thread block. Embodiments of the present disclosure further contemplate improving communication efficiencies while working within the framework of existing post-send protocol(s). Aspects of the present disclosure may include two levels: (1) warp-level coalescing of WQE slot reservation and WQE creation and (2) coalescing of memory barriers and doorbell updates. Embodiments of the present disclosure are contemplated for use in an architecture having a scalable array of multithreaded SMs. Each SM may include a set of execution units, a set of registers, and a chunk of shared memory. In some embodiments, the basic unit of execution for a processing unit (e.g., a CPU or GPU) may be referred to as a warp. The processing unit 300 may transfer data 312 to the IO device 304 and/or receive data 312 from the IO device 304 using the WQ 316, the DBR 320, and a completion queue (CQ) 324. The WQ 316, DBR 320, and CQ 324 may enable the SMs 328 of the processing unit 300 to interact directly with the IO device 304 to correspond to the claimed limitation].
As per claim 9, Markthub discloses wherein the processing device is a graphics processing unit (GPU) [(Paragraphs 0007-0012 and 0028-0030) where Markthub teaches where the multiple threads (from the same or different thread blocks (CTAs)) should be able to create WQEs in parallel and submit them to the same QP without locking. Also, the number of issued memory barriers should be reduceable (e.g., by issuing one memory barrier for submitting multiple WQEs). Embodiments of the present disclosure aim to improve communication efficiencies by increasing the parallelism of the process described above for threads within the same thread block. Embodiments of the present disclosure further contemplate improving communication efficiencies while working within the framework of existing post-send protocol(s). Aspects of the present disclosure may include two levels: (1) warp-level coalescing of WQE slot reservation and WQE creation and (2) coalescing of memory barriers and doorbell updates. Embodiments of the present disclosure are contemplated for use in an architecture having a scalable array of multithreaded SMs. Each SM may include a set of execution units, a set of registers, and a chunk of shared memory. In some embodiments, the basic unit of execution for a processing unit (e.g., a CPU or GPU) may be referred to as a warp. A warp may correspond to a collection of threads (e.g., 32 threads may belong to a warp) that are executed simultaneously by an SM. Multiple warps can be executed on an SM at once. When a program on a host CPU invokes a kernel grid, the blocks of the grid may be enumerated and distributed to SMs with available execution capacity. The threads of a thread block execute concurrently on one SM, and multiple thread blocks can execute concurrently on one SM. As thread blocks terminate, new blocks are launched on the vacated SMs; the processing unit coalesces two or more work queue element submissions from different threads and allocates memory to the two or more work queue element submissions and the threads associated therewith. In some embodiments, the processing unit checks whether a queue pair associated with a first submission and a queue pair associated with a second submission are shared between a common manager unit from the processing unit. In some embodiments, the processing unit issues a single work queue element write back to the IO device for both the first submission and the second submission if all threads associated with the first submission and the second submission are shared between the common manager unit; work queue entry (WQE) submission involves enqueueing a WQE into a ring buffer and updating the head pointer to submit work to a network interface card (NIC) or similar type of Input/Output (IO) device. In particular, a post-send protocol may include: (1) writing the WQE (or WQEs) in a work queue (WQ) buffer; (2) updating the doorbell record (DBR); and ringing the doorbell (DB). In the GPUDirect Async-Kernel Initiated networking protocol (GDA-KI), WQ and DBR are in GPU memory. The DB is usually provided on the NIC. In CPU-centric libraries such as libibverbs, WQ and DBR are in host memory and the DB is on the NIC to correspond to the claimed limitation].
As per claim 10, Markthub discloses wherein the host system is configured to write to a doorbell register of the memory sub-system to cause the memory sub-system to retrieve the commands for execution in the memory sub-system [(Paragraphs 0007-0008 and 0101-0104) where Markthub teaches work queue entry (WQE) submission involves enqueueing a WQE into a ring buffer and updating the head pointer to submit work to a network interface card (NIC) or similar type of Input/Output (IO) device. In particular, a post-send protocol may include: (1) writing the WQE (or WQEs) in a work queue (WQ) buffer; (2) updating the doorbell record (DBR); and ringing the doorbell (DB). In the GPUDirect Async-Kernel Initiated networking protocol (GDA-KI), WQ and DBR are in GPU memory. The DB is usually provided on the NIC. In CPU-centric libraries such as libibverbs, WQ and DBR are in host memory and the DB is on the NIC, DBRs 320 may be used as a notification mechanism when new WQEs are put on the WQs 316 or to solicit completion notifications from the CQs 324. Each QP may be associated with a corresponding DBR to correspond to the claimed limitation].
As per claim 11, Markthub discloses a respective completion queue for each submission queue [(Paragraphs 0010-0012, 0028-0030 and 0101) where Markthub teaches where the multiple threads (from the same or different thread blocks (CTAs)) should be able to create WQEs in parallel and submit them to the same QP without locking. Also, the number of issued memory barriers should be reduceable (e.g., by issuing one memory barrier for submitting multiple WQEs). Embodiments of the present disclosure aim to improve communication efficiencies by increasing the parallelism of the process described above for threads within the same thread block. Embodiments of the present disclosure further contemplate improving communication efficiencies while working within the framework of existing post-send protocol(s). Aspects of the present disclosure may include two levels: (1) warp-level coalescing of WQE slot reservation and WQE creation and (2) coalescing of memory barriers and doorbell updates. Embodiments of the present disclosure are contemplated for use in an architecture having a scalable array of multithreaded SMs. Each SM may include a set of execution units, a set of registers, and a chunk of shared memory. In some embodiments, the basic unit of execution for a processing unit (e.g., a CPU or GPU) may be referred to as a warp. The processing unit 300 may transfer data 312 to the IO device 304 and/or receive data 312 from the IO device 304 using the WQ 316, the DBR 320, and a completion queue (CQ) 324. The WQ 316, DBR 320, and CQ 324 may enable the SMs 328 of the processing unit 300 to interact directly with the IO device 304 to correspond to the claimed limitation], wherein the processing device is further configured to, for each completion queue, write to a doorbell register of the memory sub-system in response to obtaining a second batch of completion records from the completion queue [(Paragraphs 0007-0008 and 0101-0104) where Markthub teaches work queue entry (WQE) submission involves enqueueing a WQE into a ring buffer and updating the head pointer to submit work to a network interface card (NIC) or similar type of Input/Output (IO) device. In particular, a post-send protocol may include: (1) writing the WQE (or WQEs) in a work queue (WQ) buffer; (2) updating the doorbell record (DBR); and ringing the doorbell (DB). In the GPUDirect Async-Kernel Initiated networking protocol (GDA-KI), WQ and DBR are in GPU memory. The DB is usually provided on the NIC. In CPU-centric libraries such as libibverbs, WQ and DBR are in host memory and the DB is on the NIC, DBRs 320 may be used as a notification mechanism when new WQEs are put on the WQs 316 or to solicit completion notifications from the CQs 324. Each QP may be associated with a corresponding DBR to correspond to the claimed limitation].
As per independent claim 21, Markthub discloses a host system comprising: memory; and at least one processing device comprising a plurality of streaming multiprocessors [(Paragraphs 0004-0009) where Markthub teaches where the Communication protocols such as GDA-KI, which utilize GPUs instead of CPUs, leverage a GPU streaming multiprocessor (SM) to submit WQEs to the NIC. If traditional WQE submission algorithms are strictly followed, the GPU will need to (1) lock the network QP, which limits the concurrency, or (2) create one QP per thread, which may consume hundreds GB of GPU memory in real applications. In addition, each WQE submission will require issuing a memory barrier, which incurs significant latency for the GPU SMS to correspond to the claimed limitation], wherein: each streaming multiprocessor is configured to use a respective plurality of queue pairs [(Paragraphs 0010-0012) where Markthub teaches where the multiple threads (from the same or different thread blocks (CTAs)) should be able to create WQEs in parallel and submit them to the same QP without locking. Also, the number of issued memory barriers should be reduceable (e.g., by issuing one memory barrier for submitting multiple WQEs). Embodiments of the present disclosure aim to improve communication efficiencies by increasing the parallelism of the process described above for threads within the same thread block. Embodiments of the present disclosure further contemplate improving communication efficiencies while working within the framework of existing post-send protocol(s). Aspects of the present disclosure may include two levels: (1) warp-level coalescing of WQE slot reservation and WQE creation and (2) coalescing of memory barriers and doorbell updates. Embodiments of the present disclosure are contemplated for use in an architecture having a scalable array of multithreaded SMs. Each SM may include a set of execution units, a set of registers, and a chunk of shared memory. In some embodiments, the basic unit of execution for a processing unit (e.g., a CPU or GPU) may be referred to as a warp. A warp may correspond to a collection of threads (e.g., 32 threads may belong to a warp) that are executed simultaneously by an SM. Multiple warps can be executed on an SM at once. When a program on a host CPU invokes a kernel grid, the blocks of the grid may be enumerated and distributed to SMs with available execution capacity. The threads of a thread block execute concurrently on one SM, and multiple thread blocks can execute concurrently on one SM. As thread blocks terminate, new blocks are launched on the vacated SMs to correspond to the claimed limitation]; and the processing device is configured to, for each queue pair, assign threads executing on a streaming multiprocessor to one of the queue pairs [(Paragraphs 0010-0012, 0028-0030 and 0101) where Markthub teaches where the multiple threads (from the same or different thread blocks (CTAs)) should be able to create WQEs in parallel and submit them to the same QP without locking. Also, the number of issued memory barriers should be reduceable (e.g., by issuing one memory barrier for submitting multiple WQEs). Embodiments of the present disclosure aim to improve communication efficiencies by increasing the parallelism of the process described above for threads within the same thread block. Embodiments of the present disclosure further contemplate improving communication efficiencies while working within the framework of existing post-send protocol(s). Aspects of the present disclosure may include two levels: (1) warp-level coalescing of WQE slot reservation and WQE creation and (2) coalescing of memory barriers and doorbell updates. Embodiments of the present disclosure are contemplated for use in an architecture having a scalable array of multithreaded SMs. Each SM may include a set of execution units, a set of registers, and a chunk of shared memory. In some embodiments, the basic unit of execution for a processing unit (e.g., a CPU or GPU) may be referred to as a warp. The processing unit 300 may transfer data 312 to the IO device 304 and/or receive data 312 from the IO device 304 using the WQ 316, the DBR 320, and a completion queue (CQ) 324. The WQ 316, DBR 320, and CQ 324 may enable the SMs 328 of the processing unit 300 to interact directly with the IO device 304 to correspond to the claimed limitation].
As per claim 22, Markthub discloses wherein a plurality of threads running on each streaming multiprocessor are configured to write commands into a respective plurality of submission queues [(Paragraphs 0010-0012, 0028-0030 and 0101) where Markthub teaches where the multiple threads (from the same or different thread blocks (CTAs)) should be able to create WQEs in parallel and submit them to the same QP without locking. Also, the number of issued memory barriers should be reduceable (e.g., by issuing one memory barrier for submitting multiple WQEs). Embodiments of the present disclosure aim to improve communication efficiencies by increasing the parallelism of the process described above for threads within the same thread block. Embodiments of the present disclosure further contemplate improving communication efficiencies while working within the framework of existing post-send protocol(s). Aspects of the present disclosure may include two levels: (1) warp-level coalescing of WQE slot reservation and WQE creation and (2) coalescing of memory barriers and doorbell updates. Embodiments of the present disclosure are contemplated for use in an architecture having a scalable array of multithreaded SMs. Each SM may include a set of execution units, a set of registers, and a chunk of shared memory. In some embodiments, the basic unit of execution for a processing unit (e.g., a CPU or GPU) may be referred to as a warp. The processing unit 300 may transfer data 312 to the IO device 304 and/or receive data 312 from the IO device 304 using the WQ 316, the DBR 320, and a completion queue (CQ) 324. The WQ 316, DBR 320, and CQ 324 may enable the SMs 328 of the processing unit 300 to interact directly with the IO device 304 to correspond to the claimed limitation].
As per claim 25, Markthub discloses wherein the processing device is further configured to, for each queue pair, cause the assigned threads to write commands into different slots of a submission queue of the assigned queue pair [(Paragraphs 0010-0012 and 0115-0116) where Markthub teaches where the multiple threads (from the same or different thread blocks (CTAs)) should be able to create WQEs in parallel and submit them to the same QP without locking. Also, the number of issued memory barriers should be reduceable (e.g., by issuing one memory barrier for submitting multiple WQEs). Embodiments of the present disclosure aim to improve communication efficiencies by increasing the parallelism of the process described above for threads within the same thread block. Embodiments of the present disclosure further contemplate improving communication efficiencies while working within the framework of existing post-send protocol(s). Aspects of the present disclosure may include two levels: (1) warp-level coalescing of WQE slot reservation and WQE creation and (2) coalescing of memory barriers and doorbell updates. Embodiments of the present disclosure are contemplated for use in an architecture having a scalable array of multithreaded SMs. Each SM may include a set of execution units, a set of registers, and a chunk of shared memory. In some embodiments, the basic unit of execution for a processing unit (e.g., a CPU or GPU) may be referred to as a warp. A warp may correspond to a collection of threads (e.g., 32 threads may belong to a warp) that are executed simultaneously by an SM. Multiple warps can be executed on an SM at once. When a program on a host CPU invokes a kernel grid, the blocks of the grid may be enumerated and distributed to SMs with available execution capacity. The threads of a thread block execute concurrently on one SM, and multiple thread blocks can execute concurrently on one SM. As thread blocks terminate, new blocks are launched on the vacated SMs to correspond to the claimed limitation].
As per claim 26, Markthub discloses wherein the processing device is further configured to, for each queue pair, determine that a first batch of commands has been written to a submission queue [(Paragraphs 0010-0012, 0028-0030 and 0101) where Markthub teaches where the multiple threads (from the same or different thread blocks (CTAs)) should be able to create WQEs in parallel and submit them to the same QP without locking. Also, the number of issued memory barriers should be reduceable (e.g., by issuing one memory barrier for submitting multiple WQEs). Embodiments of the present disclosure aim to improve communication efficiencies by increasing the parallelism of the process described above for threads within the same thread block. Embodiments of the present disclosure further contemplate improving communication efficiencies while working within the framework of existing post-send protocol(s). Aspects of the present disclosure may include two levels: (1) warp-level coalescing of WQE slot reservation and WQE creation and (2) coalescing of memory barriers and doorbell updates. Embodiments of the present disclosure are contemplated for use in an architecture having a scalable array of multithreaded SMs. Each SM may include a set of execution units, a set of registers, and a chunk of shared memory. In some embodiments, the basic unit of execution for a processing unit (e.g., a CPU or GPU) may be referred to as a warp. The processing unit 300 may transfer data 312 to the IO device 304 and/or receive data 312 from the IO device 304 using the WQ 316, the DBR 320, and a completion queue (CQ) 324. The WQ 316, DBR 320, and CQ 324 may enable the SMs 328 of the processing unit 300 to interact directly with the IO device 304 to correspond to the claimed limitation], and in response write a value to a corresponding submission queue doorbell register of a memory sub-system [(Paragraphs 0007-0008 and 0101-0104) where Markthub teaches work queue entry (WQE) submission involves enqueueing a WQE into a ring buffer and updating the head pointer to submit work to a network interface card (NIC) or similar type of Input/Output (IO) device. In particular, a post-send protocol may include: (1) writing the WQE (or WQEs) in a work queue (WQ) buffer; (2) updating the doorbell record (DBR); and ringing the doorbell (DB). In the GPUDirect Async-Kernel Initiated networking protocol (GDA-KI), WQ and DBR are in GPU memory. The DB is usually provided on the NIC. In CPU-centric libraries such as libibverbs, WQ and DBR are in host memory and the DB is on the NIC, DBRs 320 may be used as a notification mechanism when new WQEs are put on the WQs 316 or to solicit completion notifications from the CQs 324. Each QP may be associated with a corresponding DBR to correspond to the claimed limitation].
As per claim 27, Markthub discloses wherein writing the value to the corresponding submission queue doorbell register causes the memory sub-system to retrieve the first batch of commands from the submission queue [(Paragraphs 0007-0008, 0101-0104 and 0110-0111) where Markthub teaches work queue entry (WQE) submission involves enqueueing a WQE into a ring buffer and updating the head pointer to submit work to a network interface card (NIC) or similar type of Input/Output (IO) device. In particular, a post-send protocol may include: (1) writing the WQE (or WQEs) in a work queue (WQ) buffer; (2) updating the doorbell record (DBR); and ringing the doorbell (DB). In the GPUDirect Async-Kernel Initiated networking protocol (GDA-KI), WQ and DBR are in GPU memory. The DB is usually provided on the NIC. In CPU-centric libraries such as libibverbs, WQ and DBR are in host memory and the DB is on the NIC, DBRs 320 may be used as a notification mechanism when new WQEs are put on the WQs 316 or to solicit completion notifications from the CQs 324. Each QP may be associated with a corresponding DBR the IO device 304 reads that data 312 from the memory of the processing unit 300. The data 312 may also be read using RDMA or a similar approach. The location of the data 312 may be obtained, at least in part, with data retrieved from the corresponding WQE. In a seventh operation (7), the IO device 304 may transfer the data 312 to a remote node. For instance, the IO device 304 may transfer the data 312 to a peer computing system 108a-d, a peer device 112, or the like to correspond to the claimed limitation].
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-6 are rejected under 35 U.S.C. 103(a) as being unpatentable over Qiu et al. (US PGPUB 2020/00019433) (hereinafter ‘Qiu’), in view of Markthub et al. (US PGPUB 2025/0077444) (hereinafter ‘Markthub’).
As per independent claim 1, Qiu discloses a host system comprising: memory; and at least one processing device configured to: allocate a portion of the memory for a queue pair [(Paragraphs 0004-0009 and 0064-0070; FIGs 1-6) where Qiu teaches where the signaling exchange between the host and the controller or between the remote host and the NVM subsystem may be implemented using a queue resource configured for the host and the controller or for the remote host and the NVM subsystem. In an embodiment, two queue pairs, namely, an input/output queue pair and an administration queue pair are defined in a data processing process. An administration queue pair is used to store a to-be-processed management command and a completed management command. Only one administration queue pair is configured for each controller and each host. Each administration queue pair includes an administration submission queue and an administration completion queue. An I/O queue pair is used to store a to-be-processed read/write access request and a completed read/write access request. One or more I/O queue pairs may be configured for each host and each controller. Each I/O queue pair includes a submission queue (including an SQ and an ASQ) and a completion queue (including a CQ and an ACQ). An access request message to be processed by the host is stored in the SQ, and a result of an access request message delivered by the host and processed completely by the controller is stored in the CQ. The submission queue and the completion queue are used as a bridge for communication between the host and the controller to correspond to the claimed limitation].
Qiu does not appear to explicitly disclose batch submitting of commands for processing by a memory sub-system via the queue pair.
However, Markthub discloses batch submitting of commands for processing by a memory sub-system via the queue pair [(Paragraphs 0010-0012) where Markthub teaches where the multiple threads (from the same or different thread blocks (CTAs)) should be able to create WQEs in parallel and submit them to the same QP without locking. Also, the number of issued memory barriers should be reduceable (e.g., by issuing one memory barrier for submitting multiple WQEs). Embodiments of the present disclosure aim to improve communication efficiencies by increasing the parallelism of the process described above for threads within the same thread block. Embodiments of the present disclosure further contemplate improving communication efficiencies while working within the framework of existing post-send protocol(s). Aspects of the present disclosure may include two levels: (1) warp-level coalescing of WQE slot reservation and WQE creation and (2) coalescing of memory barriers and doorbell updates. Embodiments of the present disclosure are contemplated for use in an architecture having a scalable array of multithreaded SMs. Each SM may include a set of execution units, a set of registers, and a chunk of shared memory. In some embodiments, the basic unit of execution for a processing unit (e.g., a CPU or GPU) may be referred to as a warp. A warp may correspond to a collection of threads (e.g., 32 threads may belong to a warp) that are executed simultaneously by an SM. Multiple warps can be executed on an SM at once. When a program on a host CPU invokes a kernel grid, the blocks of the grid may be enumerated and distributed to SMs with available execution capacity. The threads of a thread block execute concurrently on one SM, and multiple thread blocks can execute concurrently on one SM. As thread blocks terminate, new blocks are launched on the vacated SMs to correspond to the claimed limitation].
Qiu and Markthub are analogous art because they are from the same field of endeavor of data storage management.
At the time of the invention, it would have been obvious to one of ordinary skill in the art, having the teachings of Qiu and Markthub before him or her, to modify the method of Qiu to include the command batching of Markthub because it will enhance system performance.
The motivation for doing so would be [“improve communication efficiencies by increasing the parallelism of the process described above for threads within the same thread block” (Paragraph 0011 by Markthub)].
Therefore, it would have been obvious to combine Qiu and Markthub to obtain the invention as specified in the instant claim.
As per claim 2, Qiu discloses wherein the queue pair comprises a submission queue and a completion queue [(Paragraphs 0004-0009 and 0064-0070; FIGs 1-6) where Qiu teaches where the signaling exchange between the host and the controller or between the remote host and the NVM subsystem may be implemented using a queue resource configured for the host and the controller or for the remote host and the NVM subsystem. In an embodiment, two queue pairs, namely, an input/output queue pair and an administration queue pair are defined in a data processing process. An administration queue pair is used to store a to-be-processed management command and a completed management command. Only one administration queue pair is configured for each controller and each host. Each administration queue pair includes an administration submission queue and an administration completion queue. An I/O queue pair is used to store a to-be-processed read/write access request and a completed read/write access request. One or more I/O queue pairs may be configured for each host and each controller. Each I/O queue pair includes a submission queue (including an SQ and an ASQ) and a completion queue (including a CQ and an ACQ). An access request message to be processed by the host is stored in the SQ, and a result of an access request message delivered by the host and processed completely by the controller is stored in the CQ. The submission queue and the completion queue are used as a bridge for communication between the host and the controller to correspond to the claimed limitation].
As per claim 3, Markthub discloses wherein batching the submitting of commands comprises synchronizing threads using the queue pair for input/output operations [(Paragraphs 0010-0012 and 0028-0030) where Markthub teaches where the multiple threads (from the same or different thread blocks (CTAs)) should be able to create WQEs in parallel and submit them to the same QP without locking. Also, the number of issued memory barriers should be reduceable (e.g., by issuing one memory barrier for submitting multiple WQEs). Embodiments of the present disclosure aim to improve communication efficiencies by increasing the parallelism of the process described above for threads within the same thread block. Embodiments of the present disclosure further contemplate improving communication efficiencies while working within the framework of existing post-send protocol(s). Aspects of the present disclosure may include two levels: (1) warp-level coalescing of WQE slot reservation and WQE creation and (2) coalescing of memory barriers and doorbell updates. Embodiments of the present disclosure are contemplated for use in an architecture having a scalable array of multithreaded SMs. Each SM may include a set of execution units, a set of registers, and a chunk of shared memory. In some embodiments, the basic unit of execution for a processing unit (e.g., a CPU or GPU) may be referred to as a warp. A warp may correspond to a collection of threads (e.g., 32 threads may belong to a warp) that are executed simultaneously by an SM. Multiple warps can be executed on an SM at once. When a program on a host CPU invokes a kernel grid, the blocks of the grid may be enumerated and distributed to SMs with available execution capacity. The threads of a thread block execute concurrently on one SM, and multiple thread blocks can execute concurrently on one SM. As thread blocks terminate, new blocks are launched on the vacated SMs; the processing unit coalesces two or more work queue element submissions from different threads and allocates memory to the two or more work queue element submissions and the threads associated therewith. In some embodiments, the processing unit checks whether a queue pair associated with a first submission and a queue pair associated with a second submission are shared between a common manager unit from the processing unit. In some embodiments, the processing unit issues a single work queue element write back to the IO device for both the first submission and the second submission if all threads associated with the first submission and the second submission are shared between the common manager unit to correspond to the claimed limitation].
As per claim 4, Markthub discloses wherein the processing device is a graphics processing unit running threads that write the commands to a submission queue of the queue pair [(Paragraphs 0010-0012, 0028-0030 and 0101) where Markthub teaches where the multiple threads (from the same or different thread blocks (CTAs)) should be able to create WQEs in parallel and submit them to the same QP without locking. Also, the number of issued memory barriers should be reduceable (e.g., by issuing one memory barrier for submitting multiple WQEs). Embodiments of the present disclosure aim to improve communication efficiencies by increasing the parallelism of the process described above for threads within the same thread block. Embodiments of the present disclosure further contemplate improving communication efficiencies while working within the framework of existing post-send protocol(s). Aspects of the present disclosure may include two levels: (1) warp-level coalescing of WQE slot reservation and WQE creation and (2) coalescing of memory barriers and doorbell updates. Embodiments of the present disclosure are contemplated for use in an architecture having a scalable array of multithreaded SMs. Each SM may include a set of execution units, a set of registers, and a chunk of shared memory. In some embodiments, the basic unit of execution for a processing unit (e.g., a CPU or GPU) may be referred to as a warp. The processing unit 300 may transfer data 312 to the IO device 304 and/or receive data 312 from the IO device 304 using the WQ 316, the DBR 320, and a completion queue (CQ) 324. The WQ 316, DBR 320, and CQ 324 may enable the SMs 328 of the processing unit 300 to interact directly with the IO device 304 to correspond to the claimed limitation].
As per claim 5, Markthub discloses wherein batching the submitting of commands comprises writing to a submission queue doorbell register of the memory sub-system in response to determining that a batch of threads has completed writing the commands to a submission queue [(Paragraphs 0007-0008) where Markthub teaches work queue entry (WQE) submission involves enqueueing a WQE into a ring buffer and updating the head pointer to submit work to a network interface card (NIC) or similar type of Input/Output (IO) device. In particular, a post-send protocol may include: (1) writing the WQE (or WQEs) in a work queue (WQ) buffer; (2) updating the doorbell record (DBR); and ringing the doorbell (DB). In the GPUDirect Async-Kernel Initiated networking protocol (GDA-KI), WQ and DBR are in GPU memory. The DB is usually provided on the NIC. In CPU-centric libraries such as libibverbs, WQ and DBR are in host memory and the DB is on the NIC to correspond to the claimed limitation].
As per claim 6, Markthub discloses wherein the processing device is further configured to write to a completion queue doorbell register of the memory sub-system in response to determining that the batch of threads has completed obtaining completion records from a completion queue [(Paragraphs 0007-0008 and 0101-0104) where Markthub teaches work queue entry (WQE) submission involves enqueueing a WQE into a ring buffer and updating the head pointer to submit work to a network interface card (NIC) or similar type of Input/Output (IO) device. In particular, a post-send protocol may include: (1) writing the WQE (or WQEs) in a work queue (WQ) buffer; (2) updating the doorbell record (DBR); and ringing the doorbell (DB). In the GPUDirect Async-Kernel Initiated networking protocol (GDA-KI), WQ and DBR are in GPU memory. The DB is usually provided on the NIC. In CPU-centric libraries such as libibverbs, WQ and DBR are in host memory and the DB is on the NIC, DBRs 320 may be used as a notification mechanism when new WQEs are put on the WQs 316 or to solicit completion notifications from the CQs 324. Each QP may be associated with a corresponding DBR to correspond to the claimed limitation].
Claim 12 is rejected under 35 U.S.C. 103(a) as being disclosed by Markthub, as applied to claim 11, and further in view of Jiang et al. (US 8,208,406 hereinafter referred to as Jiang).
As per dependent claim 12, Markthub discloses the host system of claim 11.
Markthub does not appear to explicitly disclose wherein the host system is further configured to cause the memory sub-system to service one submission queue for each of the plurality of streaming multiprocessors in a round-robin scheme and service the respective plurality of submission queues for the each streaming multiprocessor in a round-robin scheme.
However, Jiang discloses wherein the host system is further configured to cause the memory sub-system to service one submission queue for each of the plurality of streaming multiprocessors in a round-robin scheme and service the respective plurality of submission queues for the each streaming multiprocessor in a round-robin scheme [(Column 2, lines 52-65, Column 3, lines 55-67 and Column 2, lines 52-65) where Jiang teaches where the network device comprising a plurality of packet processing nodes may employ the dynamic queue management techniques. The packet processing nodes may each comprise a highly parallelized multi-core processor. The multi-core processor may execute a queue maintenance software module that monitors and determines, for each of the plurality of queues, a relative amount of time the multi-core processor consumes to service the respective queue. The queue maintenance module of each packet processing node determines, based on the assignment, a queue weight for each of the queues of that processing node. The queue weight defines a number of the plurality of packets de-queued and forwarded by the processor during each round robin iteration through the queues to correspond to the claimed limitation].
Markthub and Jiang are analogous art because they are from the same field of endeavor of data storage management.
Before the effective filing date of the claimed inventions, it would have been obvious to one of ordinary skill in the art, having the teachings of Markthub and Jiang before him or her, to modify the device of Markthub to include the dynamic queue management techniques of Jiang because it will enhance storage operations.
The motivation for doing so would be [“perform the dynamic queue management techniques described herein to adjust weights 52 so as to improve the utilization of multi-core processor 38” (Column 14, lines 20-22 by Jiang)].
Therefore, it would have been obvious to combine Markthub and Jiang to obtain the invention as specified in the instant claim.
Claim 13 is rejected under 35 U.S.C. 103(a) as being disclosed by Markthub, as applied to claim 7, and further in view of Qureshi et al. (“GPU-Initiated On-Demand High-Throughput Storage Access in the BaM System Architecture” hereinafter referred to as Qureshi).
As per dependent claim 13, Markthub discloses the host system of claim 7.
Markthub does not appear to explicitly disclose wherein the processing device is further configured to manage a software cache to implement an array abstraction in the memory, and each command causes writing by the memory sub-system of a page of data into the software cache when the command is executed by the memory sub-system.
However, Qureshi discloses wherein the processing device is further configured to manage a software cache to implement an array abstraction in the memory [(BaM System Overview) where Qureshi teaches where the BaM presents the bam::array high-level programming abstraction, enabling programmers to easily integrate BaM into their existing GPU applications. An application can call BaM APIs to map the bam::array to data on storage, akin to mmap’ing a file. Figure 2 shows how a GPU thread uses BaM to access data. When a GPU thread accesses data with the bam::array abstraction ❶, it uses the abstraction to determine the offset, i.e. cache line, for the data being accessed ❷. Threads in a warp can coalesce their accesses ❸ if multiple threads access the same cache line. For each unique cache line being accessed, a single thread probes the cache line’s metadata ❹ on behalf of the rest of the threads, improving cache access efficiency to correspond to the claimed limitation], and each command causes writing by the memory sub-system of a page of data into the software cache when the command is executed by the memory sub-system [(Introduction, BaM System Overview and BaM Sotfware Cache) where Qureshi teaches where the software cache coalesces redundant on-demand accesses and facilitates reuse of storage data while providing high application-perceived throughput. Second, while the CPU-centric approaches suffer from the low degree of CPU thread-level parallelism available to page fault handlers and device drivers, there is currently a lack of GPU mechanisms for orchestrating storage accesses without relying on the CPU. To address this issue, BaM provides a user-level GPU library of highly concurrent submission/completion queues in GPU memory that enables GPU threads whose on-demand accesses do not hit in the software cache to make storage accesses in a high-throughput manner. This user-level approach removes the page fault handling bottleneck, and uses fine-grained synchronization and minimal critical sections to reduce software overhead for each storage access and support a high-degree of thread-level parallelism; The BaM software cache is designed to optimize the bandwidth utilization to the backing memory in two ways: (1) by eliminating redundant requests to the backing memory and (2) by allowing users to configure the cache for their application’s needs; The BaM cache aims to minimize the number of redundant I/O requests to the storage, avoiding unnecessary I/O traffic. To this end, when a thread probes the cache with an offset, it checks the corresponding cache line’s state. If the accessed cache line is not in the cache, the thread locks the cache line, finds a victim to evict, and requests the cache line from the backing memory. When the request completes, the requesting thread unlocks the cache line by making its state valid and incrementing its reference count to correspond to the claimed limitation].
Markthub and Qureshi are analogous art because they are from the same field of endeavor of data storage management.
Before the effective filing date of the claimed inventions, it would have been obvious to one of ordinary skill in the art, having the teachings of Markthub and Qureshi before him or her, to modify the device of Markthub to include the dynamic queue management techniques of Qureshi because it will enhance storage operations.
The motivation for doing so would be [(BaM System Overview) improving cache access efficiency to correspond to the claimed limitation].
Therefore, it would have been obvious to combine Markthub and Qureshi to obtain the invention as specified in the instant claim.
Claims 23 and 24 are rejected under 35 U.S.C. 103(a) as being disclosed by Markthub, as applied to claim 21, and further in view of Qiu et al. (US PGPUB 2020/0019433 hereinafter referred to as Qiu).
As per dependent claim 23, Markthub discloses the host system of claim 21.
Markthub does not appear to explicitly disclose wherein each queue pair comprises a submission queue and a completion queue.
However, Qiu discloses wherein each queue pair comprises a submission queue and a completion queue [(Paragraphs 0004-0009 and 0064-0070; FIGs 1-6) where Qiu teaches where the signaling exchange between the host and the controller or between the remote host and the NVM subsystem may be implemented using a queue resource configured for the host and the controller or for the remote host and the NVM subsystem. In an embodiment, two queue pairs, namely, an input/output queue pair and an administration queue pair are defined in a data processing process. An administration queue pair is used to store a to-be-processed management command and a completed management command. Only one administration queue pair is configured for each controller and each host. Each administration queue pair includes an administration submission queue and an administration completion queue. An I/O queue pair is used to store a to-be-processed read/write access request and a completed read/write access request. One or more I/O queue pairs may be configured for each host and each controller. Each I/O queue pair includes a submission queue (including an SQ and an ASQ) and a completion queue (including a CQ and an ACQ). An access request message to be processed by the host is stored in the SQ, and a result of an access request message delivered by the host and processed completely by the controller is stored in the CQ. The submission queue and the completion queue are used as a bridge for communication between the host and the controller to correspond to the claimed limitation].
Markthub and Qiu are analogous art because they are from the same field of endeavor of data storage management.
Before the effective filing date of the claimed inventions, it would have been obvious to one of ordinary skill in the art, having the teachings of Markthub and Qiu before him or her, to modify the device of Markthub to include the queue pair of Qiu because it will enhance storage operations.
The motivation for doing so would be [“provides a resource management method and apparatus, to release an idle resource in time based on current resource utilization, thereby improving resource utilization” (Paragraph 0006 by Qiu)].
Therefore, it would have been obvious to combine Markthub and Qiu to obtain the invention as specified in the instant claim.
As per claim 24, Qiu discloses wherein the processing device is further configured to, for each queue pair, allocate the queue pair in the memory [(Paragraphs 0004-0009 and 0064-0070; FIGs 1-6) where Qiu teaches where the signaling exchange between the host and the controller or between the remote host and the NVM subsystem may be implemented using a queue resource configured for the host and the controller or for the remote host and the NVM subsystem. In an embodiment, two queue pairs, namely, an input/output queue pair and an administration queue pair are defined in a data processing process. An administration queue pair is used to store a to-be-processed management command and a completed management command. Only one administration queue pair is configured for each controller and each host. Each administration queue pair includes an administration submission queue and an administration completion queue. An I/O queue pair is used to store a to-be-processed read/write access request and a completed read/write access request. One or more I/O queue pairs may be configured for each host and each controller. Each I/O queue pair includes a submission queue (including an SQ and an ASQ) and a completion queue (including a CQ and an ACQ). An access request message to be processed by the host is stored in the SQ, and a result of an access request message delivered by the host and processed completely by the controller is stored in the CQ. The submission queue and the completion queue are used as a bridge for communication between the host and the controller to correspond to the claimed limitation].
Pertinent Prior art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Canepa et al., US PGPUB 2014/0281083– teaches ENHANCED QUEUE MANAGEMENT.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MOHAMED GEBRIL whose telephone number is (571)270-1857. The examiner can normally be reached on Monday-Friday, 8:00am-5:00pm.ALT. Friday.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jared Rutz can be reached on 571-272-5535. The fax phone number for the organization where this application or proceeding is assigned is 571-270-2857.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MOHAMED M GEBRIL/Primary Examiner, Art Unit 2135
