DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Claims 1-20 are pending.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on September 05, 2025 is/are in compliance with the provisional of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Priority
Acknowledgment is made of applicant's claim for foreign priority based on an application filed in Korea on February 21, 2025. It is noted, however, that applicant has not filed a certified copy of the KR10-2025-0022862 application as required by 37 CFR 1.55.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 1, 2, 4, and 10-12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Mizrahi (US 2020/0201575) (hereinafter Mizrahi) (published June 25, 2020) in view of PINTO (US 2018/0307521) (hereinafter Pinto) (published October 25, 2018) and Fang et al. (US 2024/0078045) (hereinafter Fang) (published March 07, 2024).
Regarding Claim 1, Mizrahi discloses hub device, comprising:
“The interface switch (“switch”) 110 may be positioned between the one or more hosts 102 and the SSDs 130, 140, and 150, and may be configured to communicatively couple the one or more hosts with these SSDs” (Mizrahi [0017])
a hub controller configured to: select a target processor from the plurality of processors upon a request of a host, select a target queue from at least one software queue corresponding to the target processor, and support communication between the host and the target processor using the target queue.
“In various examples, the command processor module of processor circuitry 114 may terminate commands, such as an NVMe command, received from the host 102 associated a request for access to an SSD by distributing an indication in the SQ for a particular one of the queue pairs 121, 122, or 123 based on the SSD 130, 140, 150, respectively, to which the request for the data storage service is being directed” (Mizrahi [0022] the communication between host and the processor of the SSD is facilitated via the selected queue pair)
“As an non-limiting example of system 100, SSD 130 includes a local processor 132, a memory array 135, a submission queue (SQ) 133, and a completion queue (CQ) 134, SSD 140 includes a local processor 142, a memory array 145, a submission queue (SQ) 143, and a completion queue (CQ) 144, and SSD 150 includes a local processor 152, a memory array 155, a submission queue (SQ) 153. and a completion queue (CQ) 154 … The local processors 132, 142, and 152 for each SSD may be configured to manage data operations associated with the respective SSD where the local processor is located, such as reading, writing, and erasing data operations associated with the memory array of the SSD where the local processor is located” (Mizrahi [0028] due to each queue pair being associated with a respective SSD, the selection of the queue corresponds to the selection of the processor associated with that SSD)
But does not explicitly state a first memory configured to: store queue setting information for setting a plurality of software queues corresponding to a plurality of processors, and store doorbell information generated based on the queue setting information and indicating statuses of the plurality of software queues; a second memory configured to: store tag information triggering operations of the plurality of processors.
Pinto discloses queue setting information for setting a plurality of software queues corresponding to a plurality of processors, and
“virtual I/O queue creation command may include parameters that are part of the conventional I/O queue creation command, such as queue size 906 (how much space should be allocated for the queue being established), queue identifier 909 (an identifier for the submission queue being established), completion queue identifier 912 (an identifier for the completion queue), queue priority 915 (the relative priority of the submission queue being established), and physically contiguous flag 918 (indicating whether the submission queue and the completion queue are to be physically contiguous or not). Virtual I/O queue creation command 903 may also include extended attributes, such as LBA range attribute 921, QoS attribute 924, and shared namespace attribute 927” (Pinto [0118])
doorbell information generated based on the queue setting information and indicating statuses of the plurality of software queues;
“FIG. 8 shows memory mapping of doorbells in storage device 120 of FIG. 1 to support VM isolation. In conventional storage devices that use doorbells to communicate between host device 105 of FIG. 1 and storage device 120 of FIG. 1, there may be multiple doorbells: for example, one for each of I/O queues 520-1, 520-2 and 520-3 of FIG. 5. To support access to these doorbells by VMs 305-1, 305-2, and 305-3, storage device 120 of FIG. 1 may request that a portion of the address space of host device 105 of FIG. 1 be allocated to storage device 120. The addresses in the address space of host device 105 of FIG. 1 may then be mapped to the memory addresses of storage device 105. Hypervisor 310 of FIG. 3 may then provide the addresses for the doorbells to VMs 305-1, 305-2, and 305-3 of FIG. 3, enabling VMs 305-1, 305-2, and 305-3 of FIG. 3 to access the doorbells by using the addresses in the address space of host device 105 of FIG. 1” (Pinto [0112])
It would have been obvious before the effective filing date of the invention to one of ordinary skill in the art to combine the queue-setting information and doorbell information of Pinto with the system of Mizrahi, such that the queue-setting information and doorbell information are maintained for the plurality of queues corresponding to the plurality of processors. The motivation for making this modification would be to allow the queues and their corresponding doorbell information to be identified and managed according to their respective configurations and statuses, thereby facilitating efficient management and processing of the plurality of processor-associated queues.
It would have further been obvious to include a first memory to store the queue-setting information and doorbell information in memory so that the controller has a readily accessible location for retaining the information used to configure and manage the respective queues and for accessing the information during subsequent queue-management operations. Such storage would provide a conventional and predictable use of memory for maintaining queue configuration and status information and would not change the underlying operation of processor-associated queues. The combination would therefore provide a first memory configured to store queue setting information for setting the plurality of software queues corresponding to the plurality of processors and store doorbell information generated based on the queue setting information and indicating statuses of the plurality of software queues.
Fang discloses a second memory configured to: store tag information triggering operations of the plurality of processors; and
“The hardware module detects the empty flag of the corresponding SQ hardware queue handler control module (main entity). If it is not empty, it indicates that there are pending commands in the queue; otherwise, the hardware module detects it again next time” (Fang [0210] the second memory is the register or location storing the flag)
“The CPU detects the empty flag of the corresponding CQ hardware queue handler control module (mapping entity). If it is not empty, it indicates that there are pending messages in the queue; otherwise, the CPU detects it again next time” (Fang [0219] the second memory is the register or location storing the flag)
It would have been obvious before the effective filing date of the invention to one of ordinary skill in the art to further modify the system in the combination of Mizrahi and Pinto in view of Fang to store tag information corresponding to the status of the respective queues and use the tag information to trigger operations of the corresponding processors. By storing such queue-status information as a flag in a second memory, the processors would have readily accessible information indicating when processing of a corresponding queue is required. The motivation for making this modification would be to allow the processor to determine whether its corresponding queue contains pending information and initiate processing based on the queue-status indication, thereby facilitating efficient queue processing and reducing the need for repeated queue checking.
Regarding Claim 2, Mizrahi and Pinto further discloses further comprising: an input queue configured to receive a request from the host or the plurality of processors; and
“The host may generate a command, such as an NVMe command, including a data storage service request to fetch data stored in the SSDs or store data to the SSDs, wherein the command is placed in the submission queue 105. The command is forwarded from the host to the switch 110, as illustratively represented by arrow 104” (Mizrahi [0020])
“The device queue manager module may provide queue pairs 118 to facilitate communication of commands between the switch 110 and the SSDs 130, 140, and 150. The device queue manager module may maintain corresponding submission queues and completion queues for each of the SSDs 130, 140, 150 that are coupled to the switch through interconnect 137” (Mizrahi [0021])
an output queue configured to: receive doorbell information corresponding to the target queue from the first memory, and
“The device queue manager module may provide queue pairs 118 to facilitate communication of commands between the switch 110 and the SSDs 130, 140, and 150. The device queue manager module may maintain corresponding submission queues and completion queues for each of the SSDs 130, 140, 150 that are coupled to the switch through interconnect 137” (Mizrahi [0021])
“To support access to these doorbells by VMs 305-1, 305-2, and 305-3, storage device 120 of FIG. 1 may request that a portion of the address space of host device 105 of FIG. 1 be allocated to storage device 120. The addresses in the address space of host device 105 of FIG. 1 may then be mapped to the memory addresses of storage device 105. Hypervisor 310 of FIG. 3 may then provide the addresses for the doorbells to VMs 305-1, 305-2, and 305-3 of FIG. 3, enabling VMs 305-1, 305-2, and 305-3 of FIG. 3 to access the doorbells by using the addresses in the address space of host device 105 of FIG. 1” (Pinto [0112] communication of doorbell information would be through the queue pair of Mizrahi)
provide the doorbell information to the target processor.
“At block 1415, FPGA 315 of FIG. 3 may request an address space from host device 105 of FIG. 1. At block 1420, FPGA 315 may map memory addresses 820 of FIG. 8 to memory addresses in the requested address space. In this manner, VM isolation may be maintained, since no two VMs 305 may access their doorbells on a common memory page. At block 1425, FPGA 315 of FIG. 3 may provide VM 305 of FIG. 3 with a new memory address 820 of FIG. 8” (Pinto [0140] the address of the doorbell is provided to the processor)
Regarding Claim 4, Pinto further discloses wherein the target queue is set to a complete queue or a submission queue based on queue setting information corresponding to the target queue.
“FIG. 6 shows extended I/O queue creation command 535 of FIG. 5 for storage device 120 of FIG. 1. In FIG. 6, I/O queue creation command 535 is shown. I/O queue creation command 535 may include parameters that are part of the conventional NVMe specification-defined I/O queue creation command, such as queue size 603 (how much space should be allocated for the queue being established), queue identifier 606 (an identifier for the submission queue being established), completion queue identifier 609 (an identifier for the completion queue), queue priority 612 (the relative priority of the submission queue being established), and physically contiguous flag 615 (indicating whether the submission queue and the completion queue are to be physically contiguous or not)” (Pinto [0104] the queue identifier defines a submission queue while the completion queue identifier defines a completion queue)
Regarding Claim 7, Pinto further disclose wherein the hub controller is configured to write the queue setting information received from the host to the first memory in response to a setup request from the host.
“virtual I/O queue creation command may include parameters that are part of the conventional I/O queue creation command, such as queue size 906 (how much space should be allocated for the queue being established), queue identifier 909 (an identifier for the submission queue being established), completion queue identifier 912 (an identifier for the completion queue), queue priority 915 (the relative priority of the submission queue being established), and physically contiguous flag 918 (indicating whether the submission queue and the completion queue are to be physically contiguous or not). Virtual I/O queue creation command 903 may also include extended attributes, such as LBA range attribute 921, QoS attribute 924, and shared namespace attribute 927” (Pinto [0118] the creation of the queue is the setup of the queue)
It would have been obvious to use the first memory of the combination described above to store the queue-setting information received in Pinto's queue creation command during queue establishment. Doing so would allow the controller to retain the queue-setting information in the first memory for subsequent identification and management of the corresponding queue, consistent with the memory-storage configuration discussed above for Claim 1.
Regarding Claim 10, Mizrahi further discloses wherein the hub device is configured to communicate with the host through a first interface and communicate with the plurality of processors through a second interface different from the first interface.
“In various examples, switch 110 includes processor circuitry 114 communicatively coupled to a host interface 112, a SSD interface 116 and a memory containing queues 118” (Mizrahi [0017] see fig. 1, the host is connected via the host interface and the processors in the SSDs are connected via SSD interface)
Regarding Claim 11, Mizrahi further discloses wherein the first interface comprises an Advanced eXtensible Interface (AXI), and wherein the second interface comprises a direct interface.
“In some examples, the host interface may be an endpoint, such as but not limited to a PCIe endpoint cluster, which facilitates communication between the switch 110 and the one or more hosts 102, while the SSD interface, which in some examples may be a PCIe root complex cluster, facilitates communication between the switch 110 and the SSDs 130, 140, and 150” (Mizrahi [0018])
“Bus 303 is not limited to any particular type of bus, and may for example be compliant with Peripheral Component Interconnect (PCI), Industry Standard Architecture (ISA), PCI-Express, New Bus (NuBus), Advanced Extensible Bus AXI Bus, and Ethernet standards” (Mizrahi [0058] substituting the PCIe with AXI would have been predictable use of aa known interface protocol to provide communications between the host and the switch)
Regarding Claim 12, Mizrahi further discloses wherein the host comprises a central processing unit (CPU), and
“FIG. 1 illustrates an example configuration of a data storage system 100 which provides various data storage services to a host, while also providing SSDs configured to operate as initiators of commands or requests for data storage services or for performance of a data storage procedure, the commands or requests being communicated directly between SSDs without the need for the command or request to be originated by a host coupled to the system, and in some examples without the need for the SSDs to communicate between the SSDs through an intermediary device, such as a host processor” (Mizrahi [0015])
wherein each of the plurality of processors comprises a Tiny Processing Unit (TPU).
“The local processors 132, 142, and 152 for each SSD may be configured to manage data operations associated with the respective SSD where the local processor is located, such as reading, writing, and erasing data operations associated with the memory array of the SSD where the local processor is located” (Mizrahi [0028] paragraph [0027] of applicant’s specification defines TPU as low-power processor optimized for specific tasks, the local processors perform the specific tasks of data accesses to the SSD)
Claims 3, 5, 6, and 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Mizrahi (published June 25, 2020), Pinto (published October 25, 2018) and Fang (published March 07, 2024) as applied to claims 1 and 2 above, and further in view of Chaturvedi et al. (US 2019/0278514) (hereinafter Chat) (published September 12, 2019).
Regarding Claim 3, the combination of Mizrahi, Pinto, and Fang disclosed the device of claim 2, but does not explicitly state wherein the request comprises at least one of a head pointer write request of the target queue, a tail pointer write request of the target queue, and a read request for the doorbell information.
Chat discloses wherein the request comprises at least one of a head pointer write request of the target queue, a tail pointer write request of the target queue, and a read request for the doorbell information.
“In step 540 of FIG. 8, Host 120 writes a command to a Submission Queue. For example, host 120 can add a command to the Submission Queue (S) for Core 0. In step 542, host 120 adjusts the Tail Pointer for the Submission Queue to reflect the command added to the Submission Queue. For example, host 120 will update SQTP0 (see FIG. 6). In step 544, host 120 rings the doorbell for the Submission Queue by writing the updated Submission Queue Tail Pointer (SQTP0) to the Submission Queue Tail Doorbell (SQTDB) on Controller 102” (Chat [0049])
“In step 564, in response to the interrupt, Host 120 checks the appropriate Completion Queue at the entry pointed to by the Completion Queue Head Pointer. In step 566, host 120 processes the Completion Queue entry. In step 568, Host 120 updates the Completion Queue Head Pointer (CQHP0). In step 570, Host 120 writes the updated Completion Queue Head Pointer (CQHP0) to the Completion Queue Head Doorbell on the controller (CQHDB0)” (Chat [0051])
It would have been obvious before the effective filing date of the invention to one of ordinary skill in the art to further modify the system in the combination of Mizrahi, Pinto, and Fang further in view of Chat such that requests received by the input queue include requests to update the head pointer or tail pointer of a target queue and requests to read the corresponding doorbell information. The motivation for doing so would be to provide a standardized mechanism for managing the state and notification information of the respective queues, thereby facilitating efficient queue management and reducing unnecessary queue-status checking. Configuring the input queue to receive such pointer-update and doorbell-information requests would allow queue state and corresponding doorbell information to be updated and accessed through the existing communication path between the host, controller, and processor-associated queues.
Regarding Claim 5, the combination of Mizrahi, Pinto, and Fang disclosed the device of claim 2, and Fang further discloses update the tag information to indicate that the target queue is not empty.
“The hardware module detects the empty flag of the corresponding SQ hardware queue handler control module (main entity). If it is not empty, it indicates that there are pending commands in the queue; otherwise, the hardware module detects it again next time” (Fang [0210] the flag would be updated upon status change)
“The CPU detects the empty flag of the corresponding CQ hardware queue handler control module (mapping entity). If it is not empty, it indicates that there are pending messages in the queue; otherwise, the CPU detects it again next time” (Fang [0219] the flag would be updated upon status change)
But does not explicitly state wherein, in response to receiving a tail pointer write request of the target queue from the host or the target processor through the input queue, the hub controller is configured to: update a tail pointer of the target queue in the doorbell information,
Chat discloses wherein, in response to receiving a tail pointer write request of the target queue from the host or the target processor through the input queue, the hub controller is configured to: update a tail pointer of the target queue in the doorbell information,
“In step 540 of FIG. 8, Host 120 writes a command to a Submission Queue. For example, host 120 can add a command to the Submission Queue (S) for Core 0. In step 542, host 120 adjusts the Tail Pointer for the Submission Queue to reflect the command added to the Submission Queue. For example, host 120 will update SQTP0 (see FIG. 6). In step 544, host 120 rings the doorbell for the Submission Queue by writing the updated Submission Queue Tail Pointer (SQTP0) to the Submission Queue Tail Doorbell (SQTDB) on Controller 102” (Chat [0049])
It would have been obvious before the effective filing date of the invention to one of ordinary skill in the art to further modify the system in the combination of Mizrahi, Pinto, and Fang further in view of Chat such that, in response to receiving a tail pointer write request through the input queue, the hub controller updates the tail pointer of the target queue in the corresponding doorbell information. The motivation for doing so would be to maintain the queue's current state in the doorbell information and provide an updated indication of the queue to the corresponding processor, thereby facilitating timely identification and processing of newly added queue entries. Updating the tail pointer in the doorbell information when a new entry is added would allow the controller and processor to access current queue information through the existing queue-management and notification mechanisms.
Regarding Claim 6, the combination of Mizrahi, Pinto, and Fang disclosed the device of claim 2, and Fang further discloses update the tag information to indicate that the target queue is empty.
“The hardware module detects the empty flag of the corresponding SQ hardware queue handler control module (main entity). If it is not empty, it indicates that there are pending commands in the queue; otherwise, the hardware module detects it again next time” (Fang [0210] the flag would be updated upon status change)
“The CPU detects the empty flag of the corresponding CQ hardware queue handler control module (mapping entity). If it is not empty, it indicates that there are pending messages in the queue; otherwise, the CPU detects it again next time” (Fang [0219] the flag would be updated upon status change)
But does not explicitly state wherein, in response to receiving a head pointer write request of the target queue from the host or the target processor through the input queue, the hub controller is configured to: update a head pointer of the target queue in the doorbell information,
Chat discloses wherein, in response to receiving a head pointer write request of the target queue from the host or the target processor through the input queue, the hub controller is configured to: update a head pointer of the target queue in the doorbell information,
“In step 564, in response to the interrupt, Host 120 checks the appropriate Completion Queue at the entry pointed to by the Completion Queue Head Pointer. In step 566, host 120 processes the Completion Queue entry. In step 568, Host 120 updates the Completion Queue Head Pointer (CQHP0). In step 570, Host 120 writes the updated Completion Queue Head Pointer (CQHP0) to the Completion Queue Head Doorbell on the controller (CQHDB0)” (Chat [0051])
It would have been obvious before the effective filing date of the invention to one of ordinary skill in the art to further modify the system in the combination of Mizrahi, Pinto, and Fang further in view of Chat such that, in response to receiving a head pointer write request through the input queue, the hub controller updates the head pointer of the target queue in the corresponding doorbell information. The motivation for doing so would be to maintain the queue's current state in the doorbell information and provide an updated indication of the queue to the corresponding processor, thereby facilitating timely identification and processing of newly consumed queue entries. Updating the head pointer in the doorbell information when an entry is consumed would allow the controller and processor to access current queue information through the existing queue-management and notification mechanisms.
Regarding Claim 8, the combination of Mizrahi, Pinto, and Fang disclosed the device of claim 1, but does not explicitly state wherein the hub controller is configured to read or update the doorbell information in response to a request from the host or the plurality of processors.
Chat discloses wherein the hub controller is configured to read or update the doorbell information in response to a request from the host or the plurality of processors.
“As discussed above, when host 120 adds an entry to a submission queue or consumes an entry on a completion queue it will ring an appropriate doorbell by writing the updated pointer to that doorbell” (Chat [0046])
It would have been obvious before the effective filing date of the invention to one of ordinary skill in the art to further modify the system in the combination of Mizrahi, Pinto, and Fang further in view of Chat such that the hub controller reads or updates the doorbell information in response to a request from the host or the plurality of processors. The motivation for doing so would be to maintain and provide current queue-status and notification information in response to queue-management requests. Configuring the hub controller to read or update the doorbell information in response to such requests would allow the queue state and corresponding notification information to be accessed and maintained through the existing communication path between the host, controller, and processor-associated queues.
Claim 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Mizrahi (published June 25, 2020), Pinto (published October 25, 2018) and Fang (published March 07, 2024) as applied to claim 1 above, and further in view of Bugge (US 2013/0054858) (hereinafter Bugge) (published February 28, 2013).
Regarding Claim 9, the combination of Mizrahi, Pinto, and Fang disclosed the device of claim 1, but does not explicitly state wherein the second memory is configured to provide the target processor with an interrupt signal indicating whether the target queue is empty based on the tag information.
Bugge discloses wherein the second memory is configured to provide the target processor with an interrupt signal indicating whether the target queue is empty based on the tag information.
“By way of another example, upon receiving a message, the receiving communication adapter may determine whether the completion queue corresponding to the message transitioned from a state of empty to non-empty. The empty state is when there were no messages waiting to be processed by the receiving entity. The non-empty state is when messages are waiting to be processed by the receiving entity. If the completion queue corresponding to the message transitions from a state of empty to non-empty, then an interrupt is triggered to the receiving entity. The interrupt allows the receiving entity to start processing messages” (Bugge [0013])
It would have been obvious before the effective filing date of the invention to one of ordinary skill in the art to modify the system in the combination of Mizrahi, Pinto, and Fang further in view of Bugge to provide the target processor with an interrupt signal indicating whether the target queue is empty according to the tag information. The motivation for doing so would be to notify the target processor of a change in the status of its corresponding queue, thereby facilitating timely processing of information in the target queue and reducing the need for repeated queue-status checking. Applying Bugge's queue-status-based interrupt mechanism to the tag information in the system of Mizrahi, Pinto, and Fang would cause the target processor to receive an interrupt according to the queue-status indication represented by the tag information, thereby notifying the target processor when the target queue contains information to be processed.
Claims 13 and 16-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Mizrahi (published June 25, 2020) in view of Chat (published September 12, 2019) and Fang (published March 07, 2024).
Regarding Claim 13, Mizrahi discloses an operating method of a hub device including a first memory, the operating method comprising:
“In various examples, switch 110 includes processor circuitry 114 communicatively coupled to a host interface 112, a SSD interface 116 and a memory containing queues 118” (Mizrahi [0017])
a plurality of software queues corresponding to a plurality of processors stored in the first memory in response to a request from a host or the plurality of processors;
“In various examples, the command processor module of processor circuitry 114 may terminate commands, such as an NVMe command, received from the host 102 associated a request for access to an SSD by distributing an indication in the SQ for a particular one of the queue pairs 121, 122, or 123 based on the SSD 130, 140, 150, respectively, to which the request for the data storage service is being directed” (Mizrahi [0022])
selecting a target processor among the plurality of processors;
selecting a target queue from at least one software queue corresponding to the target processor; and
“In various examples, the command processor module of processor circuitry 114 may terminate commands, such as an NVMe command, received from the host 102 associated a request for access to an SSD by distributing an indication in the SQ for a particular one of the queue pairs 121, 122, or 123 based on the SSD 130, 140, 150, respectively, to which the request for the data storage service is being directed” (Mizrahi [0022] the communication between host and the processor of the SSD is facilitated via the selected queue pair)
“As an non-limiting example of system 100, SSD 130 includes a local processor 132, a memory array 135, a submission queue (SQ) 133, and a completion queue (CQ) 134, SSD 140 includes a local processor 142, a memory array 145, a submission queue (SQ) 143, and a completion queue (CQ) 144, and SSD 150 includes a local processor 152, a memory array 155, a submission queue (SQ) 153. and a completion queue (CQ) 154 … The local processors 132, 142, and 152 for each SSD may be configured to manage data operations associated with the respective SSD where the local processor is located, such as reading, writing, and erasing data operations associated with the memory array of the SSD where the local processor is located” (Mizrahi [0028] due to each queue pair being associated with a respective SSD, the selection of the queue corresponds to the selection of the processor associated with that SSD)
But does not explicitly state a second memory; updating doorbell information indicating statuses of a plurality of software queues; updating tag information triggering operations of the plurality of processors stored in the second memory based on the doorbell information; selecting a target processor among the plurality of processors based on the tag information; providing the target processor with doorbell information corresponding to the target queue.
Chat discloses updating doorbell information indicating statuses of a plurality of software queues;
“Doorbell registers 444 are a set of registers that are operated as doorbells. As discussed above, when host 120 adds an entry to a submission queue or consumes an entry on a completion queue it will ring an appropriate doorbell by writing the updated pointer to that doorbell” (Chat [0046])
providing the target processor with doorbell information corresponding to the target queue.
“One or more of SSDs may receive the command, as directed by the switch 110. Upon receiving the command, the respective SSD or SSDs may update the submission queue located in that SSD to indicated that a command has been received, and then the local processor for that SSD may begin performing the actual data storage procedure indicated to be performed by the command” (Mizrahi [0023])
“In response to the host writing to the Submission Queue Tail Doorbell and in response to the arbitration, Controller 102 fetches the next command based on the value of the Submission Queue Head Pointer (SQHPO) and the arbitration performed in step 546” (Chat [0049] the receiving/fetching of commands in Mizrahi is based on the provided doorbell)
It would have been obvious before the effective filing date of the invention to one of ordinary skill in the art to further modify the system of Mizrahi in view of Chat such that doorbell information corresponding to the respective queues is updated and provided to the processor associated with a selected queue. The motivation for doing so would be to provide the processor with current queue-status information when queue entries are added or consumed, thereby facilitating timely processing of commands associated with the respective queues. Updating and providing the corresponding doorbell information would allow the processor associated with the selected queue to receive current information for managing and processing the queue through the existing communication path between the host, controller, and processor-associated queues.
Chat, Fang, and Mizrahi discloses a second memory; updating tag information triggering operations of the plurality of processors stored in the second memory based on the doorbell information;
“The queue is considered empty if the head and tail pointers are equal. The consumer uses the head pointer to determine where to start reading from the queue, after examining the tail pointer and determining that the queue is non-empty” (Chat [0040])
“After one or more entries have been pushed into the queue, the tail pointer (that was incremented) is written to the controller via a submission queue doorbell register residing on the controller” (Chat [0041])
“The hardware module detects the empty flag of the corresponding SQ hardware queue handler control module (main entity). If it is not empty, it indicates that there are pending commands in the queue; otherwise, the hardware module detects it again next time” (Fang [0210] the flag would be updated upon status change)
“The CPU detects the empty flag of the corresponding CQ hardware queue handler control module (mapping entity). If it is not empty, it indicates that there are pending messages in the queue; otherwise, the CPU detects it again next time” (Fang [0219] the flag would be updated upon status change)
selecting a target processor among the plurality of processors based on the tag information; (emphasis added)
“The hardware module detects the empty flag of the corresponding SQ hardware queue handler control module (main entity). If it is not empty, it indicates that there are pending commands in the queue; otherwise, the hardware module detects it again next time” (Fang [0210])
“The CPU detects the empty flag of the corresponding CQ hardware queue handler control module (mapping entity). If it is not empty, it indicates that there are pending messages in the queue; otherwise, the CPU detects it again next time” (Fang [0219])
“In various examples, the command processor module of processor circuitry 114 may terminate commands, such as an NVMe command, received from the host 102 associated a request for access to an SSD by distributing an indication in the SQ for a particular one of the queue pairs 121, 122, or 123 based on the SSD 130, 140, 150, respectively, to which the request for the data storage service is being directed” (Mizrahi [0022] the processors in the SSD will be selected based on a queue that has pending commands or messages)
It would have been obvious before the effective filing date of the invention to one of ordinary skill in the art to modify the system in the combination of Chat and Mizrahi further in view of Fang such that the tag information stored in the second memory is updated based on the doorbell information and used to select the processor corresponding to a queue having pending commands or messages. The motivation for doing so would be to provide an indication of the current status of the respective queues and use the indication to identify the processor associated with a queue requiring processing, thereby facilitating timely processing of pending commands or messages and avoiding unnecessary checking of queues that do not contain pending information. Using queue-status information to identify a processor associated with pending queue entries would allow the existing queue and doorbell mechanisms to coordinate queue-status detection with selection and operation of the corresponding processor.
Regarding Claim 16, Chat and Fang further discloses wherein the updating the doorbell information comprises updating a tail pointer of the target queue in the doorbell information in response to a tail pointer write request of the target queue received from the host or the target processor, and
“In step 540 of FIG. 8, Host 120 writes a command to a Submission Queue. For example, host 120 can add a command to the Submission Queue (S) for Core 0. In step 542, host 120 adjusts the Tail Pointer for the Submission Queue to reflect the command added to the Submission Queue. For example, host 120 will update SQTP0 (see FIG. 6). In step 544, host 120 rings the doorbell for the Submission Queue by writing the updated Submission Queue Tail Pointer (SQTP0) to the Submission Queue Tail Doorbell (SQTDB) on Controller 102” (Chat [0049])
wherein the updating the tag information comprises updating the tag information to indicate that the target queue is not empty.
“The hardware module detects the empty flag of the corresponding SQ hardware queue handler control module (main entity). If it is not empty, it indicates that there are pending commands in the queue; otherwise, the hardware module detects it again next time” (Fang [0210] the flag would be updated upon status change)
“The CPU detects the empty flag of the corresponding CQ hardware queue handler control module (mapping entity). If it is not empty, it indicates that there are pending messages in the queue; otherwise, the CPU detects it again next time” (Fang [0219] the flag would be updated upon status change)
Regarding Claim 17, Chat and Fang further discloses wherein the updating the doorbell information comprises updating a head pointer of the target queue in the doorbell information in response to a head pointer write request for the target queue received from the host or the target processor, and
“In step 564, in response to the interrupt, Host 120 checks the appropriate Completion Queue at the entry pointed to by the Completion Queue Head Pointer. In step 566, host 120 processes the Completion Queue entry. In step 568, Host 120 updates the Completion Queue Head Pointer (CQHP0). In step 570, Host 120 writes the updated Completion Queue Head Pointer (CQHP0) to the Completion Queue Head Doorbell on the controller (CQHDB0)” (Chat [0051])
wherein the updating the tag information comprises updating the tag information to indicate that the target queue is empty.
“The hardware module detects the empty flag of the corresponding SQ hardware queue handler control module (main entity). If it is not empty, it indicates that there are pending commands in the queue; otherwise, the hardware module detects it again next time” (Fang [0210] the flag would be updated upon status change)
“The CPU detects the empty flag of the corresponding CQ hardware queue handler control module (mapping entity). If it is not empty, it indicates that there are pending messages in the queue; otherwise, the CPU detects it again next time” (Fang [0219] the flag would be updated upon status change)
Regarding Claim 18, Mizrahi discloses a cluster system comprising: a plurality of processors configured to communicate with a host; and
“The interface switch (“switch”) 110 may be positioned between the one or more hosts 102 and the SSDs 130, 140, and 150, and may be configured to communicatively couple the one or more hosts with these SSDs. In various examples, switch 110 is an NVMe switch configured to processes NVMe commands to control PCIe based point-to-point switch connections between the one or more hosts 102 and the SSDs 130, 140, and 150” (Mizrahi [0017])
“As an non-limiting example of system 100, SSD 130 includes a local processor 132, a memory array 135, a submission queue (SQ) 133, and a completion queue (CQ) 134, SSD 140 includes a local processor 142, a memory array 145, a submission queue (SQ) 143, and a completion queue (CQ) 144, and SSD 150 includes a local processor 152, a memory array 155, a submission queue (SQ) 153. and a completion queue (CQ) 154” (Mizrahi [0028])
a hub device configured to: select a target processor from the plurality of processors in response to a request from the host, select a target queue from at least one software queue corresponding to the target processor, and
“In various examples, the command processor module of processor circuitry 114 may terminate commands, such as an NVMe command, received from the host 102 associated a request for access to an SSD by distributing an indication in the SQ for a particular one of the queue pairs 121, 122, or 123 based on the SSD 130, 140, 150, respectively, to which the request for the data storage service is being directed” (Mizrahi [0022] the communication between host and the processor of the SSD is facilitated via the selected queue pair)
“As an non-limiting example of system 100, SSD 130 includes a local processor 132, a memory array 135, a submission queue (SQ) 133, and a completion queue (CQ) 134, SSD 140 includes a local processor 142, a memory array 145, a submission queue (SQ) 143, and a completion queue (CQ) 144, and SSD 150 includes a local processor 152, a memory array 155, a submission queue (SQ) 153. and a completion queue (CQ) 154 … The local processors 132, 142, and 152 for each SSD may be configured to manage data operations associated with the respective SSD where the local processor is located, such as reading, writing, and erasing data operations associated with the memory array of the SSD where the local processor is located” (Mizrahi [0028] due to each queue pair being associated with a respective SSD, the selection of the queue corresponds to the selection of the processor associated with that SSD)
perform communication between the host and the target processor using the target queue.
“In various examples, the command processor module of processor circuitry 114 may terminate commands, such as an NVMe command, received from the host 102 associated a request for access to an SSD by distributing an indication in the SQ for a particular one of the queue pairs 121, 122, or 123 based on the SSD 130, 140, 150, respectively, to which the request for the data storage service is being directed” (Mizrahi [0022])
But does not explicitly state store doorbell information indicating statuses of a plurality of software queues corresponding to the plurality of processors, store tag information triggering operations of the plurality of processors.
Chat discloses store doorbell information indicating statuses of a plurality of software queues corresponding to the plurality of processors,
“Doorbell registers 444 are a set of registers that are operated as doorbells. As discussed above, when host 120 adds an entry to a submission queue or consumes an entry on a completion queue it will ring an appropriate doorbell by writing the updated pointer to that doorbell” (Chat [0046])
It would have been obvious before the effective filing date of the invention to one of ordinary skill in the art to further modify the system of Mizrahi in view of Chat such that doorbell information corresponding to the respective queues is updated and provided to the processor associated with a selected queue. The motivation for doing so would be to provide the processor with current queue-status information when queue entries are added or consumed, thereby facilitating timely processing of commands associated with the respective queues. Updating and providing the corresponding doorbell information would allow the processor associated with the selected queue to receive current information for managing and processing the queue through the existing communication path between the host, controller, and processor-associated queues.
Fang discloses store tag information triggering operations of the plurality of processors,
“The hardware module detects the empty flag of the corresponding SQ hardware queue handler control module (main entity). If it is not empty, it indicates that there are pending commands in the queue; otherwise, the hardware module detects it again next time” (Fang [0210])
“The CPU detects the empty flag of the corresponding CQ hardware queue handler control module (mapping entity). If it is not empty, it indicates that there are pending messages in the queue; otherwise, the CPU detects it again next time” (Fang [0219])
It would have been obvious before the effective filing date of the invention to one of ordinary skill in the art to modify the system in the combination of Chat and Mizrahi further in view of Fang such that the stored tag information is used to trigger operations of the processor corresponding to a queue having pending commands or messages. The motivation for doing so would be to provide an indication of the current status of the respective queues, thereby facilitating timely processing of pending commands or messages and avoiding unnecessary checking of queues that do not contain pending information. Using queue-status information to identify pending queue entries would allow the existing queue and doorbell mechanisms to coordinate queue-status detection with operation of the processors.
Regarding Claim 19, Chat and Mizrahi further discloses wherein, in response to a request from the host or the target processor, the hub device is configured to update the doorbell information, and
“Doorbell registers 444 are a set of registers that are operated as doorbells. As discussed above, when host 120 adds an entry to a submission queue or consumes an entry on a completion queue it will ring an appropriate doorbell by writing the updated pointer to that doorbell” (Chat [0046])
provide doorbell information corresponding to the target queue to the target processor.
“One or more of SSDs may receive the command, as directed by the switch 110. Upon receiving the command, the respective SSD or SSDs may update the submission queue located in that SSD to indicated that a command has been received, and then the local processor for that SSD may begin performing the actual data storage procedure indicated to be performed by the command” (Mizrahi [0023])
“In response to the host writing to the Submission Queue Tail Doorbell and in response to the arbitration, Controller 102 fetches the next command based on the value of the Submission Queue Head Pointer (SQHPO) and the arbitration performed in step 546” (Chat [0049] the receiving/fetching of commands in Mizrahi is based on the provided doorbell)
Claims 14 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Mizrahi (published June 25, 2020), Chat (published September 12, 2019), and Fang (published March 07, 2024) as applied to claim 13 and 19 above, and further in view of Bugge (published February 28, 2013).
Regarding Claim 14, the combination of Mizrahi, Chat, and Fang disclosed the method of claim 13, but does not explicitly state further comprising providing the target processor with an interrupt signal indicating whether the target queue is empty based on the tag information.
Bugge discloses further comprising providing the target processor with an interrupt signal indicating whether the target queue is empty based on the tag information.
“By way of another example, upon receiving a message, the receiving communication adapter may determine whether the completion queue corresponding to the message transitioned from a state of empty to non-empty. The empty state is when there were no messages waiting to be processed by the receiving entity. The non-empty state is when messages are waiting to be processed by the receiving entity. If the completion queue corresponding to the message transitions from a state of empty to non-empty, then an interrupt is triggered to the receiving entity. The interrupt allows the receiving entity to start processing messages” (Bugge [0013])
It would have been obvious before the effective filing date of the invention to one of ordinary skill in the art to modify the system in the combination of Mizrahi, Chat, and Fang further in view of Bugge to provide the target processor with an interrupt signal indicating whether the target queue is empty according to the tag information. The motivation for doing so would be to notify the target processor of a change in the status of its corresponding queue, thereby facilitating timely processing of information in the target queue and reducing the need for repeated queue-status checking. Applying Bugge's queue-status-based interrupt mechanism to the tag information in the system of Mizrahi, Chat, and Fang would cause the target processor to receive an interrupt according to the queue-status indication represented by the tag information, thereby notifying the target processor when the target queue contains information to be processed.
Regarding Claim 20, the combination of Mizrahi, Chat, and Fang disclosed the system of claim 19, and Chat and Fang further discloses wherein the hub device is configured to update the tag information based on the doorbell information and
“The queue is considered empty if the head and tail pointers are equal. The consumer uses the head pointer to determine where to start reading from the queue, after examining the tail pointer and determining that the queue is non-empty” (Chat [0040])
“After one or more entries have been pushed into the queue, the tail pointer (that was incremented) is written to the controller via a submission queue doorbell register residing on the controller” (Chat [0041])
“The hardware module detects the empty flag of the corresponding SQ hardware queue handler control module (main entity). If it is not empty, it indicates that there are pending commands in the queue; otherwise, the hardware module detects it again next time” (Fang [0210] the flag would be updated upon status change)
“The CPU detects the empty flag of the corresponding CQ hardware queue handler control module (mapping entity). If it is not empty, it indicates that there are pending messages in the queue; otherwise, the CPU detects it again next time” (Fang [0219] the flag would be updated upon status change)
But does not explicitly state provide the target processor with an interrupt signal indicating whether the target queue is empty based on the tag information.
Fang and Bugge discloses provide the target processor with an interrupt signal indicating whether the target queue is empty based on the tag information.
“The hardware module detects the empty flag of the corresponding SQ hardware queue handler control module (main entity). If it is not empty, it indicates that there are pending commands in the queue; otherwise, the hardware module detects it again next time” (Fang [0210])
“The CPU detects the empty flag of the corresponding CQ hardware queue handler control module (mapping entity). If it is not empty, it indicates that there are pending messages in the queue; otherwise, the CPU detects it again next time” (Fang [0219])
“By way of another example, upon receiving a message, the receiving communication adapter may determine whether the completion queue corresponding to the message transitioned from a state of empty to non-empty. The empty state is when there were no messages waiting to be processed by the receiving entity. The non-empty state is when messages are waiting to be processed by the receiving entity. If the completion queue corresponding to the message transitions from a state of empty to non-empty, then an interrupt is triggered to the receiving entity. The interrupt allows the receiving entity to start processing messages” (Bugge [0013])
It would have been obvious before the effective filing date of the invention to one of ordinary skill in the art to modify the system in the combination of Mizrahi, Chat, and Fang further in view of Bugge to provide the target processor with an interrupt signal indicating whether the target queue is empty according to the tag information. The motivation for doing so would be to notify the target processor of a change in the status of its corresponding queue, thereby facilitating timely processing of information in the target queue and reducing the need for repeated queue-status checking. Applying Bugge's queue-status-based interrupt mechanism to the tag information in the system of Mizrahi, Chat, and Fang would cause the target processor to receive an interrupt according to the queue-status indication represented by the tag information, thereby notifying the target processor when the target queue contains information to be processed.
Claim 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Mizrahi (published June 25, 2020), Chat (published September 12, 2019), and Fang (published March 07, 2024) as applied to claim 13 above, and further in view of Pinto (published October 25, 2018).
Regarding Claim 15, the combination of Mizrahi, Chat, and Fang disclosed the method of claim 13, but does not explicitly state further comprising: writing queue setting information for setting up the plurality of software queues to the first memory in response to a setup request from the host; and generating the doorbell information based on the queue setting information.
Pinto discloses further comprising: writing queue setting information for setting up the plurality of software queues to the first memory in response to a setup request from the host; and
“virtual I/O queue creation command may include parameters that are part of the conventional I/O queue creation command, such as queue size 906 (how much space should be allocated for the queue being established), queue identifier 909 (an identifier for the submission queue being established), completion queue identifier 912 (an identifier for the completion queue), queue priority 915 (the relative priority of the submission queue being established), and physically contiguous flag 918 (indicating whether the submission queue and the completion queue are to be physically contiguous or not). Virtual I/O queue creation command 903 may also include extended attributes, such as LBA range attribute 921, QoS attribute 924, and shared namespace attribute 927” (Pinto [0118])
generating the doorbell information based on the queue setting information.
“FIG. 8 shows memory mapping of doorbells in storage device 120 of FIG. 1 to support VM isolation. In conventional storage devices that use doorbells to communicate between host device 105 of FIG. 1 and storage device 120 of FIG. 1, there may be multiple doorbells: for example, one for each of I/O queues 520-1, 520-2 and 520-3 of FIG. 5. To support access to these doorbells by VMs 305-1, 305-2, and 305-3, storage device 120 of FIG. 1 may request that a portion of the address space of host device 105 of FIG. 1 be allocated to storage device 120. The addresses in the address space of host device 105 of FIG. 1 may then be mapped to the memory addresses of storage device 105. Hypervisor 310 of FIG. 3 may then provide the addresses for the doorbells to VMs 305-1, 305-2, and 305-3 of FIG. 3, enabling VMs 305-1, 305-2, and 305-3 of FIG. 3 to access the doorbells by using the addresses in the address space of host device 105 of FIG. 1” (Pinto [0112])
It would have been obvious before the effective filing date of the invention to one of ordinary skill in the art to modify the method of the combination of Mizrahi, Chat, and Fang further in view of Pinto such that, in response to a setup request from the host, queue setting information is written to the first memory and doorbell information is generated based on the queue setting information. The motivation for doing so would be to establish and retain the configuration of the plurality of software queues and their corresponding doorbell information during queue initialization, thereby facilitating identification, management, and notification of the respective queues during subsequent queue operations. Writing the queue setting information to the first memory and generating the corresponding doorbell information during queue establishment would allow the controller to retain and access the information needed to configure the queues and manage their corresponding status and notification information.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Margetts (US 2018/0260145) discloses multiple local processors, submission and completion queues corresponding to the processors, and also connection to a host device
CHO (US 2022/0083271) discloses submission and completion queue along with doorbell registers used to indicate the tail and/or head pointers.
Hahn et al. (US 2023/0101626) discloses the initialization of submission and completion queues by sending information such as base address for each queue.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SIDNEY LI whose telephone number is (571)270-5967. The examiner can normally be reached Monday to Friday 10:00 AM to 6:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Arpan P Savla can be reached at (571) 272-1077. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/S.L./Examiner, Art Unit 2137
/RYAN BERTRAM/Primary Examiner, Art Unit 2137
Prosecution Insights