DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Examiner Notes
Examiner cites particular columns and line numbers in the references as applied to the claims below for the convenience of the applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested that, in preparing responses, the applicant fully consider the references in entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the examiner.
The examiner encourages Applicant to submit an authorization to communicate with the examiner via the Internet by making the following statement (from MPEP 502.03):
“Recognizing that Internet communications are not secure, I hereby authorize the USPTO to communicate with the undersigned and practitioners in accordance with 37 CFR 1.33 and 37 CFR 1.34 concerning any subject matter of this application by video conferencing, instant messaging, or electronic mail. I understand that a copy of these communications will be made of record in the application file.”
Please note that the above statement can only be submitted via Central Fax, Regular postal mail, or EFS Web (PTO/SB/439).
Examiner did not make a 35 USC § 101 rejection with respect to claim 19 because Applicant’s specification define computer storage media as physical storage media and hardware storage. (Applicant’s Specification, [072], Computer storage media are physical storage media that store computer-executable instructions and/or data structures. Physical storage media include computer hardware, such as random access memory (RAM), read-only memory (ROM), electrically erasable programmable ROM (EEPROM), solid state drives (SSDs), flash memory, phase-change memory (PCM), optical disk storage, magnetic disk storage or other magnetic storage devices, or any other hardware storage device(s) which can be used to store program code in the form of computer-executable instructions or data structures, which can be accessed and executed by a general-purpose or special-purpose computer system to implement the disclosed functionality).
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 10-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The following claim language is unclear and indefinite:
As per claim 10, it is unclear what is meant by:
“based on the first completion queue entry including the command identifier:
fetching the first submission queue entry from the virtual administrative submission queue; and
inserting a second completion queue entry into a virtual administrative completion queue of the virtual NVMe controller, the second completion queue entry including: the command identifier, and the result of the physical NVMe controller having executed the administrative command.”
According to the claims, queue entries are fetched from the first submission queue in order to be processed prior to inserting corresponding completion queue entries. Processed queue entries are then placed in a completion queue. It is unclear how the completion queue entry including the command identifier influences fetching submission queue entries and completing the other above-referenced steps following the “fetching” step for the same command. For examination purposes, the examiner assumes that the phrase “based on the first completion queue entry including the command identifier” was written in error). Claims 11-18 are rejected due to their dependency on independent claim 10.
As per claim 19, it is unclear what is meant by :
“based on the first completion queue entry including the command identifier:
fetch the first submission queue entry from the virtual administrative submission queue;
determine, based on the opcode, either: that the administrative command is to be emulated by the virtual NVMe controller, or that the administrative command has been executed by the physical NVMe controller; and
insert a second completion queue entry into a virtual administrative completion queue of the virtual NVMe controller, the second completion queue entry including the command identifier and a result that as based on one of: emulating the administrative command by the virtual NVMe controller, or the physical NVMe controller having executed the administrative command.”
According to the claims, queue entries are fetched from the first submission queue in order to be processed prior to inserting corresponding completion queue entries. Processed queue entries are then placed in a completion queue. It is unclear how the completion queue entry including the command identifier influences fetching submission queue entries and completing the other above-referenced steps following the “fetching” step for the same command. For examination purposes, the examiner assumes that the phrase “based on the first completion queue entry including the command identifier” was written in error). Claim 20 is rejected due to its dependency on independent claim 19.
The following claim language lacks sufficient antecedent basis:
Claim 6, Line 3: the command identifier of the first completion queue entry.
Claim 15, Line 3: the command identifier of the first completion queue entry. Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 2 , 3-6, 9-10, 12-15, 18-20 rejected under 35 U.S.C. 103 as being unpatentable over Dong et al. (US 20200174819 A1) in view of Esaka (US 20230070397 A1). Examiner Note: The following reference is cited as an evidentiary reference to illustrate the features of the NVMe protocol and is not relied upon for the rejections recited below: NVM Express: NVM Express NVM Command Set Specification, July 23, 2021, Revision 1.0a, Pgs. 20-21)
As per claim 1, Dong teaches the invention substantially as claimed including a method, implemented at a computer system that includes a processor ([0082], processor platform 1300 of the illustrated example includes one or more processor(s) 1312), comprising:
identifying a first submission queue entry at a virtual administrative submission queue ([0071], guest queue manager 216 determines whether an I/O command or an administrative command has been submitted to the guest queues 226a (FIG. 2); and [0072], the example guest queue manager 216 accesses the command in the guest queue 226a (block 1104)) of a virtual non-volatile memory express (NVMe) controller ([0071], guest queue manager 216 of the virtual NVMe device 224 (FIG. 2) determines whether a command has been submitted), the first submission queue entry having been written by a virtual machine (VM) guest ([0093], the guest queue manager to access a first command submitted to a guest queue by a native nonvolatile memory driver executing in a guest virtual machine) to the virtual administrative submission queue [0051], the guest queues 226a... include administrative queues (e.g., ASQ, ACQ); and [0093], a first command submitted to a guest queue), and including: ... a first opcode of an administrative command ([0095], the first command is at least one of an administrative command; Examiner Note: The NVMe protocol uses opcodes to describe commands as shown in the following evidentiary reference: NVM Express: NVM Express NVM Command Set Specification, Pg. 20, Section 3.2, The NVM Command Set includes the commands listed in Figure 18. This section describes the definition for each of the commands defined by this specification. Commands are submitted as described in the NVMe Base Specification; and Fig. 18: Opcodes for NVM Commands);
determining, based on the first opcode, that the administrative command is to be emulated by the virtual NVMe controller ([0052], the guest queue manager 216 (FIG. 2) processes administrative commands in the guest queues 226a to determine which administrative commands need to be forwarded to the shadow queues 230a (and, thus, the physical queues 231), ...This determination is made based on the type of administrative command; [0093], a mediator to generate a translated command based on the first command by translating a virtual parameter of the first command to a physical parameter associated with a physical nonvolatile memory device; and [0099], the guest queue manager is further to determine that the first command is to be handled by the physical nonvolatile memory device before translating the first command, the determination based on ...the first command being an administrative command to access a configuration of the physical nonvolatile memory device);
inserting a second submission queue entry into a physical administrative submission queue of a physical NVMe controller ([0050], when the shadow queue manager 220 makes a change to the shadow queues 230a, the host native NVMe driver 222 (FIG. 2) propagates or synchronizes the change to the physical queues 231; [0102], submit, by the virtual nonvolatile memory device, the translated command to a shadow queue to be processed by the physical nonvolatile memory device based on the physical parameter; and [0103], the translated command is to be processed by the physical nonvolatile memory device after the translated command is synchronized from the shadow queue in the virtual machine monitor to a physical queue in the physical nonvolatile memory device), the second submission queue entry including: ...a second opcode of a placeholder command that is different than the administrative command ([0103], the translated command is to be processed by the physical nonvolatile memory device after the translated command is synchronized from the shadow queue in the virtual machine monitor to a physical queue in the physical nonvolatile memory device; Examiner Note: NVMe uses opcodes to describe commands: NVM Express: NVM Express NVM Command Set Specification, July 23, 2021, Pg. 20, Section 3.2, The NVM Command Set includes the commands listed in Figure 18. This section describes the definition for each of the commands defined by this specification. Commands are submitted as described in the NVMe Base Specification; and Fig. 18: Opcodes for NVM Commands);
identifying a first completion queue entry at a physical administrative completion queue of the physical NVMe controller, the first completion queue entry corresponding to the second submission queue entry ([0075], When the translated command has been serviced, control advances to block 1116 at which the example mediator 218 translates the completion status entry (block 1116). For example, the mediator 218 accesses the completion status entry from an IOCQ or an ACQ of the shadow queue 230a, and it translates the completion status entry by converting one or more physical parameters to one or more corresponding virtual parameters for use by the guest VM 202a);
fetching the first submission queue entry from the virtual administrative submission queue ([0037], guest native NVMe drivers 214a, 214b write to (e.g., submit commands to) or read from (e.g., read completion status entries from) the guest queues 226a, 226b; [0047], virtual NVMe device 224 can read a completion status of an administrative command from the head of the ASQ/CQ0 314... the virtual NVMe device 224 writes to the head of the ASQ/CQ0 314 to notify the host native NVMe driver 222 that the virtual NVMe device 224 has read the completion status; [0056], guest native NVMe driver 214a than accesses the translated completion queue entry from the guest queues 226a; and [0072], the guest queue manager 216 ...may access an administrative command in an ASQ of the guest queue 226a);
emulating the administrative command ([0110], generating, by the virtual nonvolatile memory device, a translated command based on the first command by translating a virtual parameter of the first command to a physical parameter associated with a physical nonvolatile memory device); and
inserting a second completion queue entry into a virtual administrative completion queue of the virtual NVMe controller ([0110], submitting, by the virtual nonvolatile memory device, a completion status entry to the guest queue), the second completion queue entry including a result of emulating the administrative command ([0056], the completion queue entry may indicate to the guest native NVMe driver 214a that data requested from the NVMe device 206 is stored in the memory buffer 234a of FIG. 2 corresponding to the guest VM 202a; and [0110], submitting, by the virtual nonvolatile memory device, a completion status entry to the guest queue, the completion status entry indicative of completion of a direct memory access operation that copies data between the physical nonvolatile memory device and a guest memory buffer corresponding to the guest virtual machine).
Dong fails to specifically teach, a method, implemented at a computer system that includes a processor, comprising:... and including: the first submission queue entry ... including: a command identifier; the second submission queue entry including: the command identifier; and the first completion queue entry ... including the command identifier.
However, Esaka teaches, the first submission queue entry ... including: a command identifier ([0071], SQ entry information field indicates information (SQ entry information) representing a corresponding write command...the SQ entry information field indicates, for example,... a command ID; Examiner Note: Esaka’s SQ is a submission queue: [0060], storage area of the NVRAM 24 is allocated as a storage area of various data used for processing by the host 2,... an area of a submission queue (SQ) 243);
the second submission queue entry including: the command identifier ([0145], the command reception module 121 stores a first completion (completion indicative of successful termination) for the write command in the completion queue 244 ((9) in FIG. 6). This first completion includes the command ID of the write command); and
the first completion queue entry ... including the command identifier ([0145], the command reception module 121 stores a first completion (completion indicative of successful termination) for the write command in the completion queue 244 ((9) in FIG. 6). This first completion includes the command ID of the write command).
Esaka also teaches, the first submission queue entry ... including: ... a first opcode of an administrative command ([0071], SQ entry information field indicates information (SQ entry information) representing a corresponding write command...the SQ entry information field indicates, for example, a command operation code; and [0072], The command operation code is a code indicative of a type of a corresponding command (here, a write command). As the command operation code, for example, Opcode defined in the NVMe standard is used).
Dong and Esaka are analogous because they are both related to managing administrative commands. Dong teaches a method of processing administrative commands for virtualized devices utilizing the NVMe protocol. ([0090], apparatus and articles of manufacture disclosed herein process commands from virtual machines using techniques that improve virtualization performance associated with accessing virtualized storage and/or memory space). Esaka teaches processing NVMe commands by passing commands, including command identifiers and corresponding opcodes, between administrative queues. (Abstract, CPU stores first data in the nonvolatile memory, performs a first transmission of a write request associated with the first data to the memory system, and stores management data including information equivalent to the write request in the nonvolatile memory. In response to receiving a first response to the write request transmitted in the first transmission, the CPU adds, to the management data, information indicating that the first response has been received; [0071], the SQ entry information field indicates, for example, a command operation code...[and] a command ID; and [0072], command operation code is a code indicative of a type of a corresponding command (here, a write command). As the command operation code, for example, Opcode defined in the NVMe standard is used). It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention that Dong’s administrative queues would be modified to include the command identifier as taught by Esaka resulting in a system that efficiently processes NVMe commands in accordance with a command identifier and a corresponding opcode. Therefore, it would have been obvious to combine the teachings of Dong and Esaka.
As per claim 2, Dong teaches, further comprising:
identifying the first opcode of the administrative command ([0052], the guest queue manager 216 (FIG. 2) processes administrative commands in the guest queues 226a to determine which administrative commands need to be forwarded to the shadow queues 230a (and, thus, the physical queues 231), ...This determination is made based on the type of administrative command; and [0099], the guest queue manager is further to determine that the first command is to be handled by the physical nonvolatile memory device before translating the first command, the determination based on ...the first command being an administrative command to access a configuration of the physical nonvolatile memory device; Examiner Note: Dong uses the NVMe protocol which uses opcodes to identify commands. This is evidenced in the NVM Express reference. (NVM Express: NVM Express NVM Command Set Specification, July 23, 2021, Pg. 20, Section 3.2, The NVM Command Set includes the commands listed in Figure 18. This section describes the definition for each of the commands defined by this specification. Commands are submitted as described in the NVMe Base Specification; and Fig. 18: Opcodes for NVM Commands); and
determining, based on the first opcode, that the administrative command is to be emulated by the virtual NVMe controller ([0052], the guest queue manager 216 (FIG. 2) processes administrative commands in the guest queues 226a to determine which administrative commands need to be forwarded to the shadow queues 230a (and, thus, the physical queues 231), ...This determination is made based on the type of administrative command; [0093], a mediator to generate a translated command based on the first command by translating a virtual parameter of the first command to a physical parameter associated with a physical nonvolatile memory device; and [0099], the guest queue manager is further to determine that the first command is to be handled by the physical nonvolatile memory device before translating the first command, the determination based on ...the first command being an administrative command to access a configuration of the physical nonvolatile memory device; Examiner Note: Dong uses the NVMe protocol which uses opcodes to identify commands. This is evidenced in the NVM Express reference. (NVM Express: NVM Express NVM Command Set Specification, July 23, 2021, Pg. 20, Section 3.2, The NVM Command Set includes the commands listed in Figure 18. This section describes the definition for each of the commands defined by this specification. Commands are submitted as described in the NVMe Base Specification; and Fig. 18: Opcodes for NVM Commands).
Esaka also teaches, identifying the first opcode of the administrative command ([0071], SQ entry information field indicates information (SQ entry information) representing a corresponding write command...the SQ entry information field indicates, for example, a command operation code).
The same motivation used in the rejection of claim 1 is applicable to the instant claim.
As per claim 3, Dong teaches, further comprising creating the virtual administrative submission queue ([0068], queue creation means may be implemented by the guest queue manager 216 of FIG. 2 for creating the guest queues 226a, 226b) and the virtual administrative completion queue ([0045], administrative queues 306 of the illustrated example include a submission queue 0 (SQ0) of an administrative submission queue (ASQ) (i.e., ASQ/SQ0 312) and a completion queue (CQ0) of an administrative completion queue (ACQ) (i.e., ACQ/CQ0 314); and [0068], queue creation means may be implemented by the guest queue manager 216 of FIG. 2 for creating the guest queues 226a, 226b).
As per claim 4, Esaka teaches, wherein inserting the second submission queue entry into the physical administrative submission queue comprises inserting the second submission queue entry into a slot of the physical administrative submission queue that corresponds to the command identifier ([0134], completion queue 244 includes multiple slots to which the memory system 3 stores responses to commands, respectively).
As per claim 5, Dong teaches, wherein identifying the first submission queue entry at the virtual administrative submission queue is based on intercepting a write by the VM guest to guest memory that is mapped to the virtual administrative submission queue ([0052], the guest queue manager 216 (FIG. 2) processes administrative commands in the guest queues 226a to determine which administrative commands need to be forwarded to the shadow queues 230a (and, thus, the physical queues 231), and which administrative commands can be intercepted and handled by the virtual NVMe device 224 without being forwarded to the shadow queues 230a).
As per claim 6, Dong teaches, wherein fetching the first submission queue entry from the virtual administrative submission queue comprises:
determining, based on the command identifier of the first completion queue entry ([0047], the control registers 304 stores a tail index value of the ASQ/SQ0 312; Examiner Note: the ASQ/SQ0 312 is an administrative submission queue and control registers 304 comprise a doorbell register: [0045], The administrative queues 306 of the illustrated example include a submission queue 0 (SQ0) of an administrative submission queue (ASQ) (i.e., ASQ/SQ0 312); and [0047], SQ0 doorbell (DBL) tail (SQ0TDBL) 326 in the control registers 304 stores a tail index value of the ASQ/SQ0 312), a doorbell value ([0047], CQ0 doorbell (DBL) head (CQ0HDBL) 328) that was written by the VM guest to a doorbell register corresponding to the physical administrative submission queue ([0047], the virtual NVMe device 224 writes to the head of the ASQ/CQ0 314 to notify the host native NVMe driver 222 that the virtual NVMe device 224 has read the completion status).
fetching one or more submission queue entries from the virtual administrative submission queue based on the doorbell value ([0047], virtual NVMe device 224 can read a completion status of an administrative command from the head of the ASQ/CQ0 314 based on the CQ0HDBL 328; Examiner Note: CQ0HDBL 328 is a doorbell value: [0047], a CQ0 doorbell (DBL) head (CQ0HDBL) 328 in the control registers 304 stores a head index value of the ASQ/CQ0 314).
As per claim 9, Dong teaches, wherein the physical NVMe controller executes the placeholder command without causing a controller state or stored user data side-effect ([0052], before it forwards an administrative command, the mediator 218 performs a translation as described below in connection with FIGS. 5 and 6 to ensure the administrative command is safe (e.g., ensures that the administrative command does not interfere in the administrative queue of the physical queues 231 with a command from another guest queue of another VM such as the guest VM 202b of FIG. 2) . For example, if the administrative command is “delete I/O Queue,” the mediator 218 confirms that the I/O queue to be deleted belongs to the guest queue 226a that is sending the delete command.).
As per claim 10, Dong teaches the invention substantially as claimed including a method, implemented at a computer system that includes a processor ([0082], processor platform 1300 of the illustrated example includes one or more processor(s) 1312), comprising:
identifying a first submission queue entry at a virtual administrative submission queue ([0071], guest queue manager 216 determines whether an I/O command or an administrative command has been submitted to the guest queues 226a (FIG. 2); and [0072], the example guest queue manager 216 accesses the command in the guest queue 226a (block 1104)) of a virtual non-volatile memory express (NVMe) controller ([0071], guest queue manager 216 of the virtual NVMe device 224 (FIG. 2) determines whether a command has been submitted), the first submission queue entry having been written by a virtual machine (VM) guest ([0093], the guest queue manager to access a first command submitted to a guest queue by a native nonvolatile memory driver executing in a guest virtual machine) to the virtual administrative submission queue ([0051], the guest queues 226a... include administrative queues (e.g., ASQ, ACQ); and [0093], a first command submitted to a guest queue), and including: ... an opcode of an administrative command ([0095], the first command is at least one of an administrative command; Examiner Note: The NVMe protocol uses opcodes to describe commands as shown in the following evidentiary reference: NVM Express: NVM Express NVM Command Set Specification, July 23, 2021, Pg. 20, Section 3.2, The NVM Command Set includes the commands listed in Figure 18. This section describes the definition for each of the commands defined by this specification. Commands are submitted as described in the NVMe Base Specification; and Fig. 18: Opcodes for NVM Commands);
determining, based on the opcode, that the administrative command is to be executed by a physical NVMe controller ([0052], the guest queue manager 216 (FIG. 2) processes administrative commands in the guest queues 226a to determine which administrative commands need to be forwarded to the shadow queues 230a (and, thus, the physical queues 231), ...This determination is made based on the type of administrative command; and [0102], generate, by the virtual nonvolatile memory device, a translated command based on the first command by translating a virtual parameter of the first command to a physical parameter associated with a physical nonvolatile memory device);
inserting a second submission queue entry into a physical administrative submission queue of the physical NVMe controller ([0050], when the shadow queue manager 220 makes a change to the shadow queues 230a, the host native NVMe driver 222 (FIG. 2) propagates or synchronizes the change to the physical queues 231; [0102], submit, by the virtual nonvolatile memory device, the translated command to a shadow queue to be processed by the physical nonvolatile memory device based on the physical parameter; and [0103], the translated command is to be processed by the physical nonvolatile memory device after the translated command is synchronized from the shadow queue in the virtual machine monitor to a physical queue in the physical nonvolatile memory device), the second submission queue entry including: ...the opcode of the administrative command ([0103], the translated command is to be processed by the physical nonvolatile memory device after the translated command is synchronized from the shadow queue in the virtual machine monitor to a physical queue in the physical nonvolatile memory device; Examiner Note: NVMe uses opcodes to describe commands: NVM Express: NVM Express NVM Command Set Specification, July 23, 2021, Pg. 20, Section 3.2, The NVM Command Set includes the commands listed in Figure 18. This section describes the definition for each of the commands defined by this specification. Commands are submitted as described in the NVMe Base Specification; and Fig. 18: Opcodes for NVM Commands);
identifying a first completion queue entry at a physical administrative completion queue of the physical NVMe controller, the first completion queue entry corresponding to the second submission queue entry ([0075], When the translated command has been serviced, control advances to block 1116 at which the example mediator 218 translates the completion status entry (block 1116). For example, the mediator 218 accesses the completion status entry from an IOCQ or an ACQ of the shadow queue 230a, and it translates the completion status entry by converting one or more physical parameters to one or more corresponding virtual parameters for use by the guest VM 202a) and including: ... a result of the physical NVMe controller having executed the administrative command ([[0062], functions include a shadow completion queue create function 902 for use by the virtual NVMe device 224 to create shadow completion queues (e.g., IOCQ or ACQ) when completion status entries are generated by the NVMe device 206 (FIG. 2) in the physical queues 231 (FIGS. 2 and 4-7)); and
based on the first completion queue entry ...:
fetching the first submission queue entry from the virtual administrative submission queue ([0037], guest native NVMe drivers 214a, 214b write to (e.g., submit commands to) or read from (e.g., read completion status entries from) the guest queues 226a, 226b; [0047], virtual NVMe device 224 can read a completion status of an administrative command from the head of the ASQ/CQ0 314... the virtual NVMe device 224 writes to the head of the ASQ/CQ0 314 to notify the host native NVMe driver 222 that the virtual NVMe device 224 has read the completion status; [0056], guest native NVMe driver 214a than accesses the translated completion queue entry from the guest queues 226a; and [0072], the guest queue manager 216 ...may access an administrative command in an ASQ of the guest queue 226a); and
inserting a second completion queue entry into a virtual administrative completion queue of the virtual NVMe controller ([0110], submitting, by the virtual nonvolatile memory device, a completion status entry to the guest queue), the second completion queue entry including: the command identifier, and the result of the physical NVMe controller having executed the administrative command ([0056], the completion queue entry may indicate to the guest native NVMe driver 214a that data requested from the NVMe device 206 is stored in the memory buffer 234a of FIG. 2 corresponding to the guest VM 202a; and [0110], submitting, by the virtual nonvolatile memory device, a completion status entry to the guest queue, the completion status entry indicative of completion of a direct memory access operation that copies data between the physical nonvolatile memory device and a guest memory buffer corresponding to the guest virtual machine).
Dong fails to specifically teach, the first submission queue entry ... including: a command identifier, the second submission queue entry including: the command identifier; the first completion queue entry ... including: the command identifier; and first completion queue entry including the command identifier.
However, Esaka teaches, the first submission queue entry ... including: a command identifier ([0071], SQ entry information field indicates information (SQ entry information) representing a corresponding write command...the SQ entry information field indicates, for example,... a command ID; Examiner Note: Esaka’s SQ is a submission queue: [0060], storage area of the NVRAM 24 is allocated as a storage area of various data used for processing by the host 2,... an area of a submission queue (SQ) 243);
the second submission queue entry including: the command identifier ([0145], the command reception module 121 stores a first completion (completion indicative of successful termination) for the write command in the completion queue 244 ((9) in FIG. 6). This first completion includes the command ID of the write command); and
the first completion queue entry ... including: the command identifier ([0145], the command reception module 121 stores a first completion (completion indicative of successful termination) for the write command in the completion queue 244 ((9) in FIG. 6). This first completion includes the command ID of the write command).
The same motivation used in the rejection of claim 1 is applicable to the instant claim.
As per claim 12, this claim is similar to claim 3 and is rejected for the same reasons.
As per claim 13, this claim is similar to claim 4 and is rejected for the same reasons.
As per claim 14, his claim is similar to claim 5 and is rejected for the same reasons.
As per claim 15, this claim is similar to claim 6 and is rejected for the same reasons.
As per claim 18, Dong teaches, wherein inserting the second submission queue entry into the physical administrative submission queue comprises inserting a copy of the first submission queue entry into the physical administrative submission queue ([0052], the guest queue manager 216 (FIG. 2) processes administrative commands in the guest queues 226a to determine which administrative commands need to be forwarded to the shadow queues 230a (and, thus, the physical queues 231).
As per claim 19, Dong teaches the invention substantially as claimed including a computer system comprising:
a processing system ([0082], processor platform 1300 of the illustrated example includes one or more processor(s) 1312); and
a computer storage media that stores computer-executable instructions that are executable by the processing system ([0069], the machine readable instructions implement programs for execution by one or more processors such as the processor(s) 1312 shown in the example processor platform 1300 discussed below in connection with FIG. 13. These programs may be embodied in software stored on a non-transitory computer readable storage medium) to at least:
identify a first submission queue entry at a virtual administrative submission queue ([0071], guest queue manager 216 determines whether an I/O command or an administrative command has been submitted to the guest queues 226a (FIG. 2); and [0072], the example guest queue manager 216 accesses the command in the guest queue 226a (block 1104)) of a virtual non-volatile memory express (NVMe) controller ([0071], guest queue manager 216 of the virtual NVMe device 224 (FIG. 2) determines whether a command has been submitted), the first submission queue entry having been written by a virtual machine (VM) guest ([0093], the guest queue manager to access a first command submitted to a guest queue by a native nonvolatile memory driver executing in a guest virtual machine) to the virtual administrative submission queue([0051], the guest queues 226a... include administrative queues (e.g., ASQ, ACQ); and [0093], a first command submitted to a guest queue), and including: ... an opcode of an administrative command ([0095], the first command is at least one of an administrative command; Examiner Note: The NVMe protocol uses opcodes to describe commands as shown in the following evidentiary reference: NVM Express: NVM Express NVM Command Set Specification, July 23, 2021, Pg. 20, Section 3.2, The NVM Command Set includes the commands listed in Figure 18. This section describes the definition for each of the commands defined by this specification. Commands are submitted as described in the NVMe Base Specification; and Fig. 18: Opcodes for NVM Commands);
insert a second submission queue entry into a physical administrative submission queue of a physical NVMe controller ([0050], when the shadow queue manager 220 makes a change to the shadow queues 230a, the host native NVMe driver 222 (FIG. 2) propagates or synchronizes the change to the physical queues 231; [0102], submit, by the virtual nonvolatile memory device, the translated command to a shadow queue to be processed by the physical nonvolatile memory device based on the physical parameter; and [0103], the translated command is to be processed by the physical nonvolatile memory device after the translated command is synchronized from the shadow queue in the virtual machine monitor to a physical queue in the physical nonvolatile memory device);
identify a first completion queue entry at a physical administrative completion queue of the physical NVMe controller, the first completion queue entry corresponding to the second submission queue entry [0075], When the translated command has been serviced, control advances to block 1116 at which the example mediator 218 translates the completion status entry (block 1116). For example, the mediator 218 accesses the completion status entry from an IOCQ or an ACQ of the shadow queue 230a, and it translates the completion status entry by converting one or more physical parameters to one or more corresponding virtual parameters for use by the guest VM 202a); and
based on the first completion queue entry ...:
fetch the first submission queue entry from the virtual administrative submission queue ([0037], guest native NVMe drivers 214a, 214b write to (e.g., submit commands to) or read from (e.g., read completion status entries from) the guest queues 226a, 226b; [0047], virtual NVMe device 224 can read a completion status of an administrative command from the head of the ASQ/CQ0 314... the virtual NVMe device 224 writes to the head of the ASQ/CQ0 314 to notify the host native NVMe driver 222 that the virtual NVMe device 224 has read the completion status; [0056], guest native NVMe driver 214a than accesses the translated completion queue entry from the guest queues 226a; and [0072], the guest queue manager 216 ...may access an administrative command in an ASQ of the guest queue 226a);
determine, based on the opcode, either: that the administrative command is to be emulated by the virtual NVMe controller ([0052], the guest queue manager 216 (FIG. 2) processes administrative commands in the guest queues 226a to determine which administrative commands need to be forwarded to the shadow queues 230a (and, thus, the physical queues 231), ...This determination is made based on the type of administrative command; [0093], a mediator to generate a translated command based on the first command by translating a virtual parameter of the first command to a physical parameter associated with a physical nonvolatile memory device; and [0099], the guest queue manager is further to determine that the first command is to be handled by the physical nonvolatile memory device before translating the first command, the determination based on ...the first command being an administrative command to access a configuration of the physical nonvolatile memory device; Examiner Note: Dong uses the NVMe protocol which uses opcodes to identify commands. This is evidenced in the NVM Express reference. (NVM Express: NVM Express NVM Command Set Specification, July 23, 2021, Pg. 20, Section 3.2, The NVM Command Set includes the commands listed in Figure 18. This section describes the definition for each of the commands defined by this specification. Commands are submitted as described in the NVMe Base Specification; and Fig. 18: Opcodes for NVM Commands), or that the administrative command has been executed by the physical NVMe controller; and
insert a second completion queue entry into a virtual administrative completion queue of the virtual NVMe controller ([0110], submitting, by the virtual nonvolatile memory device, a completion status entry to the guest queue), the second completion queue entry including the command identifier and a result that as based on one of: emulating the administrative command by the virtual NVMe controller, or the physical NVMe controller having executed the administrative command ([0056], the completion queue entry may indicate to the guest native NVMe driver 214a that data requested from the NVMe device 206 is stored in the memory buffer 234a of FIG. 2 corresponding to the guest VM 202a; and [0110], submitting, by the virtual nonvolatile memory device, a completion status entry to the guest queue, the completion status entry indicative of completion of a direct memory access operation that copies data between the physical nonvolatile memory device and a guest memory buffer corresponding to the guest virtual machine).
Dong fails to specifically teach the first submission queue entry ...including: a command identifier; the second submission queue entry including the command identifier; and the first completion queue entry...including the command identifier.
However, Esaka teaches, the first submission queue entry ...including: a command identifier ([0071], SQ entry information field indicates information (SQ entry information) representing a corresponding write command...the SQ entry information field indicates, for example,... a command ID; Examiner Note: Esaka’s SQ is a submission queue: [0060], storage area of the NVRAM 24 is allocated as a storage area of various data used for processing by the host 2,... an area of a submission queue (SQ) 243);
the second submission queue entry including the command identifier [0145], the command reception module 121 stores a first completion (completion indicative of successful termination) for the write command in the completion queue 244 ((9) in FIG. 6). This first completion includes the command ID of the write command); and
the first completion queue entry...including the command identifier ([0145], the command reception module 121 stores a first completion (completion indicative of successful termination) for the write command in the completion queue 244 ((9) in FIG. 6). This first completion includes the command ID of the write command).
The same motivation used in the rejection of claim 1 is applicable to the instant claim.
As per claim 20, this claim is similar to claim 6 and is rejected for the same reasons.
Claims 7 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Dong-Esaka as applied to independent claims 1 and 10 and in further view of Tai et al. (US 11068422 B1).
As per claim 7, the combination of Dong-Esaka fails to specifically teach, wherein identifying the first completion queue entry at the physical administrative completion queue is based on polling the physical administrative completion queue.
However, Tai teaches, wherein identifying the first completion queue entry at the physical administrative completion queue is based on polling the physical administrative completion queue (Column 3, Lines 5-8, management submission queue sQ 170 and completion queue cQ 172 are used for the commands that are part of an Admin Command Set; and Column 4, Lines 7-12, interrupts between the controller and the operating system are turned off, and the operating system polls the controller for the state of the completion queue cQ before delivering an interprocessor interrupt (IPI) to the core whose completion queue cQ was polled).
The combination of Dong-Esaka and Tai are analogous because they are each related to managing administrative commands. Dong teaches a method of processing administrative commands for virtualized devices utilizing the NVMe protocol. Esaka teaches processing NVMe commands by passing commands, including command identifiers and corresponding opcodes, between administrative queues. Tai teaches a method of efficiently managing the processing of administrative commands utilizing a polling technique for completion queues in order to reduce the amount of interrupts. (Column 2, Line 66-Column 2, Line 8, the host interface 136 couples to a submission queue sQ 170 and a completion queue cQ 172 for managing communication between the host computer system 100 and the device controller 112 and for each processor core 160, 162, 164, one or more submission queues sQs 174, 178, 180, 184 and a completion queue cQ 176, 182, 186. The management submission queue sQ 170 and completion queue cQ 172 are used for the commands that are part of an Admin Command Set; and Column 4, Lines 5-10, instead of the controller performing the flow of operations of FIG. 2, the operating system performs the flow of operations. In this embodiment, interrupts between the controller and the operating system are turned off, and the operating system polls the controller for the state of the completion queue cQ). It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention that the completion queues of the combination of Dong-Esaka would be modified to include Tai’s known polling mechanism resulting in a system that efficiently processes NVMe commands while reducing system interrupts. Therefore, it would have been obvious to combine the teachings of the combination of Dong-Esaka and Tai.
As per claim 16, this claim is similar to claim 7 and is rejected for the same reasons. The same motivation used in the rejection of claim 7 is applicable to the instant claim.
Claims 8 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Dong-Esaka as applied to independent claims 1 and 10 and in further view of Benisty et al. (US 20200333975 A1).
As per claim 8, the combination of Dong-Esaka fails to specifically teach, wherein identifying the first completion queue entry at the physical administrative completion queue is based on determining that a phase tag of the first completion queue entry has been toggled.
However, Benisty teaches, wherein identifying the first completion queue entry at the physical administrative completion queue is based on determining that a phase tag of the first completion queue entry has been toggled ([0056], One or more phase bits 312 may indicate whether the one or more completion queue elements 190 constitute the most recently added valid entry).
The combination of Dong-Esaka and Benisty are analogous because they are each related to managing administrative commands. Dong teaches a method of processing administrative commands for virtualized devices utilizing the NVMe protocol. Esaka teaches processing NVMe commands by passing commands, including command identifiers and corresponding opcodes, between administrative queues. Benisty teaches a method of efficiently managing the processing of administrative commands utilizing a throttling technique for completion queues in order to reduce efficiently communicate completion queue entries. (Abstract, throttling the delivery of completion entries pertaining to the execution of commands by a nonvolatile memory (NVM) device are provided. In an illustrative example, the data storage controller selectively throttles the delivery of completion entries to a host device using uniform delivery intervals to provide for stable delivery of completion entries to the host. In some examples, the throttling is achieved by storing new completion entries in a completion queue of the host while initially setting corresponding indicator bits within the completion entries (e.g. phase tags) to cause the host to ignore the new completion entries as though the new entries were old entries). It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention that the completion queues of the combination of Dong-Esaka would be modified to include Benisty’s known throttling technique using phase tags resulting in a system that efficiently processes NVMe commands while efficiently communicating completion queue entries. Therefore, it would have been obvious to combine the teachings of the combination of Dong-Esaka and Benisty.
As per claim 17, this claim is similar to claim 8 and is rejected for the same reasons. The same motivation used in the rejection of claim 8 is applicable to the instant claim.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over the combination of Dong-Esaka as applied to independent claim 10 and in further view of Nam et al. (US 20220147476 A1).
As per claim 11, Dong teaches based on fetching the first submission queue entry from the virtual administrative submission queue: identifying the opcode of the administrative command ([0052], the guest queue manager 216 (FIG. 2) processes administrative commands in the guest queues 226a to determine which administrative commands need to be forwarded to the shadow queues 230a (and, thus, the physical queues 231), ...This determination is made based on the type of administrative command; and [0099], the guest queue manager is further to determine that the first command is to be handled by the physical nonvolatile memory device before translating the first command, the determination based on ...the first command being an administrative command to access a configuration of the physical nonvolatile memory device; Examiner Note: Dong uses the NVMe protocol which uses opcodes to identify commands. This is evidenced in the NVM Express reference. (NVM Express: NVM Express NVM Command Set Specification, July 23, 2021, Pg. 20, Section 3.2, The NVM Command Set includes the commands listed in Figure 18. This section describes the definition for each of the commands defined by this specification. Commands are submitted as described in the NVMe Base Specification; and Fig. 18: Opcodes for NVM Commands).
Esaka also teaches, based on fetching the first submission queue entry from the virtual administrative submission queue: identifying the opcode of the administrative command ([0071], SQ entry information field indicates information (SQ entry information) representing a corresponding write command...the SQ entry information field indicates, for example, a command operation code).
The combination of Dong-Esaka fails to specifically teach, further comprising, based on fetching the first submission queue entry from the virtual administrative submission queue: determining, based on the opcode, that the administrative command has been executed by the physical NVMe controller.
However, Nam teaches, further comprising, based on fetching the first submission queue entry from the virtual administrative submission queue:
determining, based on the opcode, that the administrative command has been executed by the physical NVMe controller ([0075], the memory device Device A may transmit a request including command code indicating the completion (Comp) of data copy and a message MSG, to a host device Host).
The combination of Dong-Esaka and Nam are analogous because they are each related to managing NVMe commands. Dong teaches a method of processing administrative commands for virtualized devices utilizing the NVMe protocol. Esaka teaches processing NVMe commands by passing commands, including command identifiers and corresponding opcodes, between administrative queues. Nam teaches a method of using opcode to indicate a command completion status. ([0110], controller CTRL may write data to the non-volatile memory device NVM in response to a write command or read data from the non-volatile memory device NVM in response to a read command... A buffer BUF may temporarily store (or buffer) data to be written to the non-volatile memory device NVM or data read from the non-volatile memory device NVM. In some embodiments, the buffer BUF may include DRAM. The buffer BUF may store metadata. The metadata may refer to user data or data generated by the controller CTRL to manage the non-volatile memory device NVM). It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention that the completion queues of the combination of Dong-Esaka would be modified to include Nam’s known technique for communication completion status using opcodes resulting in a system that efficiently processes NVMe commands while efficiently communicating completion status. Therefore, it would have been obvious to combine the teachings of the combination of Dong-Esaka and Nam.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MELISSA A HEADLY whose telephone number is (571)272-1972. The examiner can normally be reached Monday- Friday 9-5:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Bradley Teets can be reached at 571-272-3338. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MELISSA A HEADLY/Examiner, Art Unit 2197