DETAILED ACTION
Claims 1-20 rejected under 35 USC §103.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Glimcher, U.S. PG-Publication No. 2023/0305977 A1, in view of Bhide et al., U.S. PG-Publication No. 2022/0121503 A1, further in view of Kim, U.S. PG-Publication No. 2018/0181425 A1.
Claim 1
Glimcher discloses a computer-implemented method, comprising: hosting an emulation software process running in a processing unit of a virtual environment embedded system, the emulation software process controlling communication of a device. Glimcher discloses “embodiments for Smart-NIC software upgrading without disruption to a host application.” Glimcher, ¶ 33. In one embodiment, “applications, e.g. containers, virtual machines (VMs), or native running apps, may consume emulated NVMe/PCIe block devices,” wherein the “emulated device may be created to handle submission and completion queue pairs (QPs) as well as doorbells and interrupts.” The target implementation of an emulated device is “running as a storage performance development kit (SPDK) application in a polling mode, handling both administrative operations as well as I/O … operations.” Id. at ¶¶ 45-46. An NVMe controller is “a software-defined emulation module to enable hardware accelerated virtualization of an emulated NVMe target 354” (NVMe controller → emulation software process). The NVMe target 534 “is a singleton SPDK-based application running inside the SmartNIC and provides emulation for all the emulated NVMe devices.” Id. at ¶ 49. Figure 3 illustrates a “shared namespace feature of NVMe protocol” with an “NVM subsystem 310” comprising “a first NVMe controller 312 and a second NVMe controller 312 with each NVMe controller respectively coupled to a port for host communication.” The NVM subsystem is “integrated within a data processing unit (DPU) … which may be integrated into a SmartNIC” (DPU integrated into a SmartNIC → processing unit). Id. at ¶¶ 38-39; See Also ¶¶ 53-54 (“SmartNIC 625 may comprise a DPU 630 that incorporates an NVMe subsystem to support multiple SPDK instances”).
Glimcher discloses deploying an alternate version of the emulation software process into the processing unit. Figure 8 illustrates “a process of non-disruptive update for SmartNIC storage.” At 820, “an old … NVMe/PCIe target emulation SPDK-based software is running on a first NVMe controller, among the multiple NVMe controllers.” At 825, “a new NVMe/PCIe target emulation SPDK-based software is uploaded to the DPU in the Smart-NIC without running” (new NVMe/PCIe target emulation software → alternate version of the emulator software process). At 830, “the new NVMe/PCIe target emulation SPDK-based software is running on a second NVMe controller, among the multiple NVMe controllers.” Id. at ¶¶ 61-63; FIG. 8.
Glimcher discloses migrating context information … from the running emulation software process to the alternate emulation software process. The NVMe controllers use a shared “namespace identifier (NSID) … to provide access to a corresponding namespace,” wherein the namespace is “a collection of logical block addresses (LBAs) accessible by host software.” The shared NSID “is implemented inside an emulated NVMe/PCIe device for SmartNIC emulated storage NDU [non-disruptive upgrade].”Id. at ¶¶ 38-40. Returning to Figure 8, at 815 “a remote NS [namespace] is created in the NVMe subsystem and enables as a shared NS which may be shared between different NVMe controllers.” Id. at ¶ 61. At 835, “upon recognizing the shared NS between the first NVMe controller and the second NVMe controller, the host server starts using the multiple-path configuration comprising a fist path … through the first NVMe controller and a second path … through the second NVMe controller.” In one multiple—path configuration policy, “both the new NVMe/PCIe target emulation SPDK-based software and the old NVMe/PCIe target emulations SPDK-based software may serve I/O commands and there may be no timeout” (i.e., migrating context information from old emulation software process to the new emulation software process).
Glimcher discloses transferring … control of the communication from the emulation software process to the alternate emulation software process. At 840, “the old NVMe/PCIe target emulation SPEDK-based software is removed or killed … after verification of successful operations for the new NVMe/PCIe target emulations SPDK-based software.” At 845, “the host server recognized that the first NVMe controller is offline and therefore sends I/O commands … only to the second NVMe controller which runs the new NVMe/PCIe target emulation SPDK-based software” (i.e., transferring communication control to new emulation software process). Id. at ¶ 65. In one embodiment, “the SmartNIC storage NDU process may be handled, partially or fully, by an orchestrator module, which is in charge of one or more of scheduling and uploading the new version of NVMe/PCIe target emulation SPDK-based software, scheduling and running both the old version and new version, confirming a successful operation of the new version software, and removing the old version for NDU completion.” Id. at ¶ 67.
Glimcher does not expressly disclose building, based on the migrated context information, one or more context maps in the alternate emulation software process and transferring, after the one or more context maps have been built, control of the communication from the emulation software process to the alternate emulation software process.
Bhide discloses building, based on the migrated context information, one or more context maps in the alternate emulation software process. Bhide discloses a method for migrating a workload from a source edge device to a destination edge device. Bhide, ¶¶ 17-19. In embodiments, the edge devices are smartNICs. Id. at ¶¶ 29, 35, 53, 61. The edge devices “may need to migrate the network state to another intelligent-edge device to provide a seamless migration experience.” Workload migration is “triggered by a workload orchestrator,” a “destination edge device … is identified,” and “initiates a secured connection with the source edge device where the workload currently resides.” The “state of the workload is moved with the workload,” wherein the state includes “information related to, for example, the CPU, a memory state, memory storage” (i.e., context information). Id. at ¶ 44. The destination edge device “provides a request for the workload state/context to the source edge device and the source wedge device provides a response with the requested information.” Id. at ¶ 64. A network state “includes state information for network protocols, firewall policies and network security,” such as “the flow state, stateful firewall information for various types of applications (e.g., FTP/TFTP/RPC), connection tracking information for each flow ( e.g., sequence numbers, ack numbers, window size, and so forth) for connection oriented protocols such as TCP, flow statistics (e.g., accept/deny bytes and packet statistics information), and policy state information ( e.g., when a specific flow among a set of interfaces, one of which is associated with the workload being migrated, is admitted or denied)” (network state → context maps). Id. at ¶ 41.
Bhide discloses transferring, after the one or more context maps have been built, control of the communication from the emulation software process to the alternate emulation software process. The network state “is migrated from the source edge device before the workload is ready to run at the destination edge device.” Id. at ¶¶ 43-44. In embodiments, “the context of each networking interface associated with a workload can be migrated to the destination device in parallel.” Id. at ¶ 54. The “network state is synchronized with deltas between the source edge device 216 and the destination edge device 226 when workload is ready to resume at the destination device 226 after being shut down on the source device 216.” Id. at ¶ 57.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the method for non-disruptive upgrade of smartNIC software of Glimcher to incorporate the network state migration between smartNICs as taught by Bhide. One of ordinary skill in the art would be motivated to integrate network state migration between smartNICs into Glimcher, with a reasonable expectation of success, in order to obtain benefits of “reducing traffic disruption in workload migrations and minimizing packet loss and the number of dropped packets during workload migration.” Bhide, ¶ 9.
Glimcher-Bhide does not expressly discloses context information for controlling the communication; the one or more context maps to be used to generate new context information for controlling the communication by the alternate emulation software process; and the alternate emulation software process to control the communication using the new context information.
Kim discloses context information for controlling the communication; the one or more context maps to be used to generate new context information for controlling the communication by the alternate emulation software process; and the alternate emulation software process to control the communication using the new context information. Kim discloses a “method … for handling network I/O device virtualization in a virtual machine environment that performs memory mapping using a remapping context entry separated on a virtual machine basis.” Kim, ¶ 8. In one embodiment, the method comprises steps of “transmitting, by a virtual machine emulator, an instruction request including [a] translated address information to an extended device driver associated with the virtual machine from which the I/O request if forwarded,” and “inserting, by the extended device driver, the translated address into a transmission queue.” Id. at ¶ 10. The method “handling network I/O device virtualization may further comprise generating a remapping context entry including a context entry divided for each of the plurality of virtual machines,” wherein the translating comprises “performing … the address translation by mapping a domain from the remapping context entry based on the identifier of the virtual machine identifier” (remapping context entry → new context information; context entries → context map), and “forming a control channel between a device model in the virtual machine emulator and the extended device driver.” (virtual machine emulator → alternate emulation software process). Id. at ¶¶ 15-17. The virtual machine emulator identifies “the extended device driver 125 associated with the virtual machine based on the information on the virtual machine included in the I/O request” and forwards translated address information via the control channel. Id. at ¶ 55. Direct mapping access mapping hardware 140 has “a remapping context entry, which logically divides a portion of the context entry table on a virtual machine basis and maps virtual machine addresses to the divided spaces,” thereby enabling a plurality of VMs to “share one physical I/O device.” Id. at ¶¶ 64-66. The method “re-assigns an instruction to a network interface chip so that a plurality of guest OS s can communicate with a plurality of virtual medium access control through a virtual machine emulator and a virtual machine monitor.” Id. at ¶ 80.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify method of migrating vNICs using context information of Glimcher-Bhide to incorporate context information that controls I/O communication as taught by Kim. One of ordinary skill in the art would be motivated to integrate context information that controls I/O communication into Glimcher-Bhide, with a reasonable expectation of success, in order to provide a method “of flexibly translating addresses for a plurality of virtual machines using a single physical I/O device by assigning context entries,” that “is capable of improving performance by reducing data copying burden … as the data is directly I/O to/from the outside without being copied in a process of performing I/O operations of a virtual machine.” Kim, ¶¶ 24-26.
Claim 2
Glimcher discloses initiating the transferring of control of the communication from the emulation software process to the alternate emulation software process upon relinquishment of control by the emulation software process. At 840, “the old NVMe/PCIe target emulation SPEDK-based software is removed or killed … after verification of successful operations for the new NVMe/PCIe target emulations SPDK-based software.” At 845, “the host server recognized that the first NVMe controller is offline and therefore sends I/O commands … only to the second NVMe controller which runs the new NVMe/PCIe target emulation SPDK-based software” (i.e., transferring communication control to new emulation software process). Glimcher, ¶ 65. In one embodiment, “the SmartNIC storage NDU process may be handled, partially or fully, by an orchestrator module, which is in charge of one or more of scheduling and uploading the new version of NVMe/PCIe target emulation SPDK-based software, scheduling and running both the old version and new version, confirming a successful operation of the new version software, and removing the old version for NDU completion.” Id. at ¶ 67.
Claim 3
Bhide discloses initiating the transfer of control of the communication from the emulation software process to the alternate emulation software process upon instructions from the emulation software process. The edge devices “may need to migrate the network state to another intelligent-edge device to provide a seamless migration experience.” Workload migration is “triggered by a workload orchestrator,” a “destination edge device … is identified,” and “initiates a secured connection with the source edge device where the workload currently resides.” The “state of the workload is moved with the workload,” wherein the state includes “information related to, for example, the CPU, a memory state, memory storage” (i.e., context information). Bhide, ¶ 44. The destination edge device “provides a request for the workload state/context to the source edge device and the source edge device provides a response with the requested information.” Id. at ¶ 64.
Claim 4
Bhide discloses indicating, using the alternate emulation software process, suspension of the running emulation software process upon completion of the context map. The network state “is migrated from the source edge device before the workload is ready to run at the destination edge device.” Bhide, ¶¶ 43-44. In embodiments, “the context of each networking interface associated with a workload can be migrated to the destination device in parallel.” Id. at ¶ 54. The “network state is synchronized with deltas between the source edge device 216 and the destination edge device 226 when workload is ready to resume at the destination device 226 after being shut down on the source device 216.” Id. at ¶ 57.
Claim 5
Bhide discloses wherein the context information includes one or more backend connections and one or more I/O context. The edge devices “may need to migrate the network state to another intelligent-edge device to provide a seamless migration experience.” Workload migration is “triggered by a workload orchestrator,” a “destination edge device … is identified,” and “initiates a secured connection with the source edge device where the workload currently resides.” The “state of the workload is moved with the workload,” wherein the state includes “information related to, for example, the CPU, a memory state, memory storage” (i.e., context information). Bhide, ¶ 44. The destination edge device “provides a request for the workload state/context to the source edge device and the source wedge device provides a response with the requested information.” Id. at ¶ 64. A network state “includes state information for network protocols, firewall policies and network security,” such as “the flow state, stateful firewall information for various types of applications (e.g., FTP/TFTP/RPC), connection tracking information for each flow ( e.g., sequence numbers, ack numbers, window size, and so forth) for connection oriented protocols such as TCP, flow statistics (e.g., accept/deny bytes and packet statistics information), and policy state information ( e.g., when a specific flow among a set of interfaces, one of which is associated with the workload being migrated, is admitted or denied)” (network state → context maps). Id. at ¶ 41.
Claim 6
Glimcher discloses wherein the processing unit is a DPU or a SmartNIC. The NVM subsystem is “integrated within a data processing unit (DPU) … which may be integrated into a SmartNIC” (DPU integrated into a SmartNIC → processing unit). Glimcher, ¶¶ 38-39; See Also ¶¶ 53-54 (“SmartNIC 625 may comprise a DPU 630 that incorporates an NVMe subsystem to support multiple SPDK instances”).
Claim 7
Glimcher discloses wherein the emulation software process communicates with one or more of a block device, a file system device, a network device, a crypto device, a GPU device, or a PCI emulated device. Glimcher discloses that “the first NVMe controller 632 and the second NVMe controller 633 may be an emulation module to enable hardware-accelerated virtualization of the first emulated NVMe target 634 and the second emulated NVMe target 635 respectively.” The NVMe subsystem comprises a BDEV [block device] module 638 that supports “NVMe/PCIe protocols and/or NVMe-oF protocol to direct data flows, e.g., I/Os to either one or more local NVMe storages 650 attached to the SmartNIC.” Glimcher, ¶¶ 54-55.
Claim 8
Bhide discloses preparing the one or more context maps in an inactive state. The network state “is migrated from the source edge device before the workload is ready to run at the destination edge device.” Bhide, ¶¶ 43-44. In embodiments, “the context of each networking interface associated with a workload can be migrated to the destination device in parallel.” Id. at ¶ 54. The “network state is synchronized with deltas between the source edge device 216 and the destination edge device 226 when workload is ready to resume at the destination device 226 after being shut down on the source device 216.” Id. at ¶ 57.
Claim 9
Glimcher discloses wherein the context information includes queue data. Glimcher discloses “embodiments for Smart-NIC software upgrading without disruption to a host application.” Glimcher, ¶ 33. In one embodiment, “applications, e.g. containers, virtual machines (VMs), or native running apps, may consume emulated NVMe/PCIe block devices,” wherein the “emulated device may be created to handle submission and completion queue pairs (QPs) as well as doorbells and interrupts.” The target implementation of an emulated device is “running as a storage performance development kit (SPDK) application in a polling mode, handling both administrative operations as well as I/O … operations.” Id. at ¶¶ 45-46.
Claim 10
Bhide discloses wherein the context information includes device context of one or more devices. The edge devices “may need to migrate the network state to another intelligent-edge device to provide a seamless migration experience.” Workload migration is “triggered by a workload orchestrator,” a “destination edge device … is identified,” and “initiates a secured connection with the source edge device where the workload currently resides.” The “state of the workload is moved with the workload,” wherein the state includes “information related to, for example, the CPU, a memory state, memory storage” (i.e., context information). Bhide, ¶ 44. The destination edge device “provides a request for the workload state/context to the source edge device and the source wedge device provides a response with the requested information.” Id. at ¶ 64. A network state “includes state information for network protocols, firewall policies and network security,” such as “the flow state, stateful firewall information for various types of applications (e.g., FTP/TFTP/RPC), connection tracking information for each flow ( e.g., sequence numbers, ack numbers, window size, and so forth) for connection oriented protocols such as TCP, flow statistics (e.g., accept/deny bytes and packet statistics information), and policy state information ( e.g., when a specific flow among a set of interfaces, one of which is associated with the workload being migrated, is admitted or denied)” (network state → context maps). Id. at ¶ 41.
Claim 11
Kim discloses determining, using the alternate emulation software process, one or more algorithms and at least one resource, to control the communication using the new context information, before control is transferred. The virtual machine emulator (i.e., alternate emulation software process) identifies “the extended device driver 125 associated with the virtual machine based on the information on the virtual machine included in the I/O request” and forwards translated address information via the control channel. Id. at ¶ 55. Direct mapping access mapping hardware 140 has “a remapping context entry, which logically divides a portion of the context entry table on a virtual machine basis and maps virtual machine addresses to the divided spaces,” thereby enabling a plurality of VMs to “share one physical I/O device.” Id. at ¶¶ 64-66.
Claim 12
Glimcher discloses a system, comprising: one or more processors; and memory including instructions that, when executed by the one or more processors, cause the system to: query context information from a running emulation software process in a processing unit of a virtual environment embedded system. Glimcher discloses “embodiments for Smart-NIC software upgrading without disruption to a host application.” Glimcher, ¶ 33. In one embodiment, “applications, e.g. containers, virtual machines (VMs), or native running apps, may consume emulated NVMe/PCIe block devices,” wherein the “emulated device may be created to handle submission and completion queue pairs (QPs) as well as doorbells and interrupts.” The target implementation of an emulated device is “running as a storage performance development kit (SPDK) application in a polling mode, handling both administrative operations as well as I/O … operations.” Id. at ¶¶ 45-46. An NVMe controller is “a software-defined emulation module to enable hardware accelerated virtualization of an emulated NVMe target 354” (NVMe controller → emulation software process). The NVMe target 534 “is a singleton SPDK-based application running inside the SmartNIC and provides emulation for all the emulated NVMe devices.” Id. at ¶ 49. Figure 3 illustrates a “shared namespace feature of NVMe protocol” with an “NVM subsystem 310” comprising “a first NVMe controller 312 and a second NVMe controller 312 with each NVMe controller respectively coupled to a port for host communication.” The NVM subsystem is “integrated within a data processing unit (DPU) … which may be integrated into a SmartNIC” (DPU integrated into a SmartNIC → processing unit). Id. at ¶¶ 38-39; See Also ¶¶ 53-54 (“SmartNIC 625 may comprise a DPU 630 that incorporates an NVMe subsystem to support multiple SPDK instances”).
Glimcher discloses migrate the queried context information to an alternate emulation software process in the processing unit of the virtual environment embedded system. Figure 8 illustrates “a process of non-disruptive update for SmartNIC storage.” At 820, “an old … NVMe/PCIe target emulation SPDK-based software is running on a first NVMe controller, among the multiple NVMe controllers.” At 825, “a new NVMe/PCIe target emulation SPDK-based software is uploaded to the DPU in the Smart-NIC without running” (new NVMe/PCIe target emulation software → alternate version of the emulator software process). At 830, “the new NVMe/PCIe target emulation SPDK-based software is running on a second NVMe controller, among the multiple NVMe controllers.” Id. at ¶¶ 61-63; FIG. 8.
The NVMe controllers use a shared “namespace identifier (NSID) … to provide access to a corresponding namespace,” wherein the namespace is “a collection of logical block addresses (LBAs) accessible by host software.” The shared NSID “is implemented inside an emulated NVMe/PCIe device for SmartNIC emulated storage NDU [non-disruptive upgrade].”Id. at ¶¶ 38-40. Returning to Figure 8, at 815 “a remote NS [namespace] is created in the NVMe subsystem and enables as a shared NS which may be shared between different NVMe controllers.” Id. at ¶ 61. At 835, “upon recognizing the shared NS between the first NVMe controller and the second NVMe controller, the host server starts using the multiple-path configuration comprising a fist path … through the first NVMe controller and a second path … through the second NVMe controller.” In one multiple—path configuration policy, “both the new NVMe/PCIe target emulation SPDK-based software and the old NVMe/PCIe target emulations SPDK-based software may serve I/O commands and there may be no timeout” (i.e., migrating context information from old emulation software process to the new emulation software process).
Glimcher discloses transfer, from the emulation software process to the alternate emulation software process, control of communications with an device, the communications directed by the migrated context information. At 840, “the old NVMe/PCIe target emulation SPEDK-based software is removed or killed … after verification of successful operations for the new NVMe/PCIe target emulations SPDK-based software.” At 845, “the host server recognized that the first NVMe controller is offline and therefore sends I/O commands … only to the second NVMe controller which runs the new NVMe/PCIe target emulation SPDK-based software” (i.e., transferring communication control to new emulation software process). Id. at ¶ 65. In one embodiment, “the SmartNIC storage NDU process may be handled, partially or fully, by an orchestrator module, which is in charge of one or more of scheduling and uploading the new version of NVMe/PCIe target emulation SPDK-based software, scheduling and running both the old version and new version, confirming a successful operation of the new version software, and removing the old version for NDU completion.” Id. at ¶ 67.
Glimcher does not expressly disclose build, based on the migrated context information, one or more context maps in the alternate emulation software process.
Bhide discloses build, based on the migrated context information, one or more context maps in the alternate emulation software process. Bhide discloses a method for migrating a workload from a source edge device to a destination edge device. Bhide, ¶¶ 17-19. In embodiments, the edge devices are smartNICs. Id. at ¶¶ 29, 35, 53, 61. The edge devices “may need to migrate the network state to another intelligent-edge device to provide a seamless migration experience.” Workload migration is “triggered by a workload orchestrator,” a “destination edge device … is identified,” and “initiates a secured connection with the source edge device where the workload currently resides.” The “state of the workload is moved with the workload,” wherein the state includes “information related to, for example, the CPU, a memory state, memory storage” (i.e., context information). Id. at ¶ 44. The destination edge device “provides a request for the workload state/context to the source edge device and the source wedge device provides a response with the requested information.” Id. at ¶ 64. A network state “includes state information for network protocols, firewall policies and network security,” such as “the flow state, stateful firewall information for various types of applications (e.g., FTP/TFTP/RPC), connection tracking information for each flow ( e.g., sequence numbers, ack numbers, window size, and so forth) for connection oriented protocols such as TCP, flow statistics (e.g., accept/deny bytes and packet statistics information), and policy state information ( e.g., when a specific flow among a set of interfaces, one of which is associated with the workload being migrated, is admitted or denied)” (network state → context maps). Id. at ¶ 41.
The network state “is migrated from the source edge device before the workload is ready to run at the destination edge device.” Id. at ¶¶ 43-44. In embodiments, “the context of each networking interface associated with a workload can be migrated to the destination device in parallel.” Id. at ¶ 54. The “network state is synchronized with deltas between the source edge device 216 and the destination edge device 226 when workload is ready to resume at the destination device 226 after being shut down on the source device 216.” Id. at ¶ 57.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the method for non-disruptive upgrade of smartNIC software of Glimcher to incorporate the network state migration between smartNICs as taught by Bhide. One of ordinary skill in the art would be motivated to integrate network state migration between smartNICs into Glimcher, with a reasonable expectation of success, in order to obtain benefits of “reducing traffic disruption in workload migrations and minimizing packet loss and the number of dropped packets during workload migration.” Bhide, ¶ 9.
Glimcher-Bhide does not expressly discloses context information for controlling the communication; the one or more context maps to be used to generate new context information for controlling the communication by the alternate emulation software process; and the alternate emulation software process to control the communication using the new context information.
Kim discloses context information for controlling the communication; the one or more context maps to be used to generate new context information for controlling the communication by the alternate emulation software process; and the alternate emulation software process to control the communication using the new context information. Kim discloses a “method … for handling network I/O device virtualization in a virtual machine environment that performs memory mapping using a remapping context entry separated on a virtual machine basis.” Kim, ¶ 8. In one embodiment, the method comprises steps of “transmitting, by a virtual machine emulator, an instruction request including [a] translated address information to an extended device driver associated with the virtual machine from which the I/O request if forwarded,” and “inserting, by the extended device driver, the translated address into a transmission queue.” Id. at ¶ 10. The method “handling network I/O device virtualization may further comprise generating a remapping context entry including a context entry divided for each of the plurality of virtual machines,” wherein the translating comprises “performing … the address translation by mapping a domain from the remapping context entry based on the identifier of the virtual machine identifier” (remapping context entry → new context information; context entries → context map), and “forming a control channel between a device model in the virtual machine emulator and the extended device driver.” (virtual machine emulator → alternate emulation software process). Id. at ¶¶ 15-17. The virtual machine emulator identifies “the extended device driver 125 associated with the virtual machine based on the information on the virtual machine included in the I/O request” and forwards translated address information via the control channel. Id. at ¶ 55. Direct mapping access mapping hardware 140 has “a remapping context entry, which logically divides a portion of the context entry table on a virtual machine basis and maps virtual machine addresses to the divided spaces,” thereby enabling a plurality of VMs to “share one physical I/O device.” Id. at ¶¶ 64-66. The method “re-assigns an instruction to a network interface chip so that a plurality of guest OS s can communicate with a plurality of virtual medium access control through a virtual machine emulator and a virtual machine monitor.” Id. at ¶ 80.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify method of migrating vNICs using context information of Glimcher-Bhide to incorporate context information that controls I/O communication as taught by Kim. One of ordinary skill in the art would be motivated to integrate context information that controls I/O communication into Glimcher-Bhide, with a reasonable expectation of success, in order to provide a method “of flexibly translating addresses for a plurality of virtual machines using a single physical I/O device by assigning context entries,” that “is capable of improving performance by reducing data copying burden … as the data is directly I/O to/from the outside without being copied in a process of performing I/O operations of a virtual machine.” Kim, ¶¶ 24-26.
Claim 13
Glimcher discloses support, using the context maps, acceleration engines for the context information. Glimcher discloses that the NVMe subsystem comprises “a first NVMe controller 632 to enable hardware-accelerated virtualization of a first emulated NVMe target 634” and “a second NVMe controller 633 to enable hardware-accelerated virtualization of a second emulated NVMe target 635.” Glimcher, ¶¶ 53-54, 58.
Claim 14
Bhide discloses initiate transfer of communication control upon relinquishment of the control by the emulation software process. The network state “is migrated from the source edge device before the workload is ready to run at the destination edge device.” Bhide, ¶¶ 43-44. In embodiments, “the context of each networking interface associated with a workload can be migrated to the destination device in parallel.” Id. at ¶ 54. The “network state is synchronized with deltas between the source edge device 216 and the destination edge device 226 when workload is ready to resume at the destination device 226 after being shut down on the source device 216.” Id. at ¶ 57.
Claim 15
Bhide discloses initiate transfer of communication control upon receipt of instructions from the emulation software process. The edge devices “may need to migrate the network state to another intelligent-edge device to provide a seamless migration experience.” Workload migration is “triggered by a workload orchestrator,” a “destination edge device … is identified,” and “initiates a secured connection with the source edge device where the workload currently resides.” The “state of the workload is moved with the workload,” wherein the state includes “information related to, for example, the CPU, a memory state, memory storage” (i.e., context information). Bhide, ¶ 44. The destination edge device “provides a request for the workload state/context to the source edge device and the source edge device provides a response with the requested information.” Id. at ¶ 64.
Claim 16
Glimcher discloses one or more circuits to: query context information from a running emulation software process in a processing unit of a virtual environment embedded system. Glimcher discloses “embodiments for Smart-NIC software upgrading without disruption to a host application.” Glimcher, ¶ 33. In one embodiment, “applications, e.g. containers, virtual machines (VMs), or native running apps, may consume emulated NVMe/PCIe block devices,” wherein the “emulated device may be created to handle submission and completion queue pairs (QPs) as well as doorbells and interrupts.” The target implementation of an emulated device is “running as a storage performance development kit (SPDK) application in a polling mode, handling both administrative operations as well as I/O … operations.” Id. at ¶¶ 45-46. An NVMe controller is “a software-defined emulation module to enable hardware accelerated virtualization of an emulated NVMe target 354” (NVMe controller → emulation software process). The NVMe target 534 “is a singleton SPDK-based application running inside the SmartNIC and provides emulation for all the emulated NVMe devices.” Id. at ¶ 49. Figure 3 illustrates a “shared namespace feature of NVMe protocol” with an “NVM subsystem 310” comprising “a first NVMe controller 312 and a second NVMe controller 312 with each NVMe controller respectively coupled to a port for host communication.” The NVM subsystem is “integrated within a data processing unit (DPU) … which may be integrated into a SmartNIC” (DPU integrated into a SmartNIC → processing unit). Id. at ¶¶ 38-39; See Also ¶¶ 53-54 (“SmartNIC 625 may comprise a DPU 630 that incorporates an NVMe subsystem to support multiple SPDK instances”).
Glimcher discloses migrate the queried context information to an alternate emulation software process in the processing unit of the virtual environment embedded system. Figure 8 illustrates “a process of non-disruptive update for SmartNIC storage.” At 820, “an old … NVMe/PCIe target emulation SPDK-based software is running on a first NVMe controller, among the multiple NVMe controllers.” At 825, “a new NVMe/PCIe target emulation SPDK-based software is uploaded to the DPU in the Smart-NIC without running” (new NVMe/PCIe target emulation software → alternate version of the emulator software process). At 830, “the new NVMe/PCIe target emulation SPDK-based software is running on a second NVMe controller, among the multiple NVMe controllers.” Id. at ¶¶ 61-63; FIG. 8.
The NVMe controllers use a shared “namespace identifier (NSID) … to provide access to a corresponding namespace,” wherein the namespace is “a collection of logical block addresses (LBAs) accessible by host software.” The shared NSID “is implemented inside an emulated NVMe/PCIe device for SmartNIC emulated storage NDU [non-disruptive upgrade].”Id. at ¶¶ 38-40. Returning to Figure 8, at 815 “a remote NS [namespace] is created in the NVMe subsystem and enables as a shared NS which may be shared between different NVMe controllers.” Id. at ¶ 61. At 835, “upon recognizing the shared NS between the first NVMe controller and the second NVMe controller, the host server starts using the multiple-path configuration comprising a fist path … through the first NVMe controller and a second path … through the second NVMe controller.” In one multiple—path configuration policy, “both the new NVMe/PCIe target emulation SPDK-based software and the old NVMe/PCIe target emulations SPDK-based software may serve I/O commands and there may be no timeout” (i.e., migrating context information from old emulation software process to the new emulation software process).
Glimcher discloses transfer, from the emulation software process to the alternate emulation software process, control of communications by a device, the communications directed by the migrated context information. At 840, “the old NVMe/PCIe target emulation SPEDK-based software is removed or killed … after verification of successful operations for the new NVMe/PCIe target emulations SPDK-based software.” At 845, “the host server recognized that the first NVMe controller is offline and therefore sends I/O commands … only to the second NVMe controller which runs the new NVMe/PCIe target emulation SPDK-based software” (i.e., transferring communication control to new emulation software process). Id. at ¶ 65. In one embodiment, “the SmartNIC storage NDU process may be handled, partially or fully, by an orchestrator module, which is in charge of one or more of scheduling and uploading the new version of NVMe/PCIe target emulation SPDK-based software, scheduling and running both the old version and new version, confirming a successful operation of the new version software, and removing the old version for NDU completion.” Id. at ¶ 67.
Glimcher does not expressly disclose build, based on the migrated context information, one or more context maps in the alternate emulation software process.
Bhide discloses build, based on the migrated context information, one or more context maps in the alternate emulation software process. Bhide discloses a method for migrating a workload from a source edge device to a destination edge device. Bhide, ¶¶ 17-19. In embodiments, the edge devices are smartNICs. Id. at ¶¶ 29, 35, 53, 61. The edge devices “may need to migrate the network state to another intelligent-edge device to provide a seamless migration experience.” Workload migration is “triggered by a workload orchestrator,” a “destination edge device … is identified,” and “initiates a secured connection with the source edge device where the workload currently resides.” The “state of the workload is moved with the workload,” wherein the state includes “information related to, for example, the CPU, a memory state, memory storage” (i.e., context information). Id. at ¶ 44. The destination edge device “provides a request for the workload state/context to the source edge device and the source wedge device provides a response with the requested information.” Id. at ¶ 64. A network state “includes state information for network protocols, firewall policies and network security,” such as “the flow state, stateful firewall information for various types of applications (e.g., FTP/TFTP/RPC), connection tracking information for each flow ( e.g., sequence numbers, ack numbers, window size, and so forth) for connection oriented protocols such as TCP, flow statistics (e.g., accept/deny bytes and packet statistics information), and policy state information ( e.g., when a specific flow among a set of interfaces, one of which is associated with the workload being migrated, is admitted or denied)” (network state → context maps). Id. at ¶ 41.
The network state “is migrated from the source edge device before the workload is ready to run at the destination edge device.” Id. at ¶¶ 43-44. In embodiments, “the context of each networking interface associated with a workload can be migrated to the destination device in parallel.” Id. at ¶ 54. The “network state is synchronized with deltas between the source edge device 216 and the destination edge device 226 when workload is ready to resume at the destination device 226 after being shut down on the source device 216.” Id. at ¶ 57.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the method for non-disruptive upgrade of smartNIC software of Glimcher to incorporate the network state migration between smartNICs as taught by Bhide. One of ordinary skill in the art would be motivated to integrate network state migration between smartNICs into Glimcher, with a reasonable expectation of success, in order to obtain benefits of “reducing traffic disruption in workload migrations and minimizing packet loss and the number of dropped packets during workload migration.” Bhide, ¶ 9.
Glimcher-Bhide does not expressly discloses context information for controlling the communication; the one or more context maps to be used to generate new context information for controlling the communication by the alternate emulation software process; and the alternate emulation software process to control the communication using the new context information.
Kim discloses context information for controlling the communication; the one or more context maps to be used to generate new context information for controlling the communication by the alternate emulation software process; and the alternate emulation software process to control the communication using the new context information. Kim discloses a “method … for handling network I/O device virtualization in a virtual machine environment that performs memory mapping using a remapping context entry separated on a virtual machine basis.” Kim, ¶ 8. In one embodiment, the method comprises steps of “transmitting, by a virtual machine emulator, an instruction request including [a] translated address information to an extended device driver associated with the virtual machine from which the I/O request if forwarded,” and “inserting, by the extended device driver, the translated address into a transmission queue.” Id. at ¶ 10. The method “handling network I/O device virtualization may further comprise generating a remapping context entry including a context entry divided for each of the plurality of virtual machines,” wherein the translating comprises “performing … the address translation by mapping a domain from the remapping context entry based on the identifier of the virtual machine identifier” (remapping context entry → new context information; context entries → context map), and “forming a control channel between a device model in the virtual machine emulator and the extended device driver.” (virtual machine emulator → alternate emulation software process). Id. at ¶¶ 15-17. The virtual machine emulator identifies “the extended device driver 125 associated with the virtual machine based on the information on the virtual machine included in the I/O request” and forwards translated address information via the control channel. Id. at ¶ 55. Direct mapping access mapping hardware 140 has “a remapping context entry, which logically divides a portion of the context entry table on a virtual machine basis and maps virtual machine addresses to the divided spaces,” thereby enabling a plurality of VMs to “share one physical I/O device.” Id. at ¶¶ 64-66. The method “re-assigns an instruction to a network interface chip so that a plurality of guest OS s can communicate with a plurality of virtual medium access control through a virtual machine emulator and a virtual machine monitor.” Id. at ¶ 80.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify method of migrating vNICs using context information of Glimcher-Bhide to incorporate context information that controls I/O communication as taught by Kim. One of ordinary skill in the art would be motivated to integrate context information that controls I/O communication into Glimcher-Bhide, with a reasonable expectation of success, in order to provide a method “of flexibly translating addresses for a plurality of virtual machines using a single physical I/O device by assigning context entries,” that “is capable of improving performance by reducing data copying burden … as the data is directly I/O to/from the outside without being copied in a process of performing I/O operations of a virtual machine.” Kim, ¶¶ 24-26.
Claim 17
Glimcher discloses wherein the context information includes at least one of doorbells, MSI-X resources, PCIe memory mapped registers, IOBAR registers, transient queue, and I/O context information. Glimcher discloses “embodiments for Smart-NIC software upgrading without disruption to a host application.” Glimcher, ¶ 33. In one embodiment, “applications, e.g. containers, virtual machines (VMs), or native running apps, may consume emulated NVMe/PCIe block devices,” wherein the “emulated device may be created to handle submission and completion queue pairs (QPs) as well as doorbells and interrupts.” The target implementation of an emulated device is “running as a storage performance development kit (SPDK) application in a polling mode, handling both administrative operations as well as I/O … operations.” Id. at ¶¶ 45-46. An NVMe controller is “a software-defined emulation module to enable hardware accelerated virtualization of an emulated NVMe target 354” (NVMe controller → emulation software process). The NVMe target 534 “is a singleton SPDK-based application running inside the SmartNIC and provides emulation for all the emulated NVMe devices.” Id. at ¶ 49. Figure 3 illustrates a “shared namespace feature of NVMe protocol” with an “NVM subsystem 310” comprising “a first NVMe controller 312 and a second NVMe controller 312 with each NVMe controller respectively coupled to a port for host communication.” The NVM subsystem is “integrated within a data processing unit (DPU) … which may be integrated into a SmartNIC” (DPU integrated into a SmartNIC → processing unit). Id. at ¶¶ 38-39; See Also ¶¶ 53-54 (“SmartNIC 625 may comprise a DPU 630 that incorporates an NVMe subsystem to support multiple SPDK instances”).
Claim 18
Glimcher discloses wherein the emulation software process and the alternate emulation software process are configured to control communications by the device. Glimcher discloses “embodiments for Smart-NIC software upgrading without disruption to a host application.” Glimcher, ¶ 33. In one embodiment, “applications, e.g. containers, virtual machines (VMs), or native running apps, may consume emulated NVMe/PCIe block devices,” wherein the “emulated device may be created to handle submission and completion queue pairs (QPs) as well as doorbells and interrupts.” The target implementation of an emulated device is “running as a storage performance development kit (SPDK) application in a polling mode, handling both administrative operations as well as I/O … operations.” Id. at ¶¶ 45-46. An NVMe controller is “a software-defined emulation module to enable hardware accelerated virtualization of an emulated NVMe target 354” (NVMe controller → emulation software process). The NVMe target 534 “is a singleton SPDK-based application running inside the SmartNIC and provides emulation for all the emulated NVMe devices.” Id. at ¶ 49. Figure 3 illustrates a “shared namespace feature of NVMe protocol” with an “NVM subsystem 310” comprising “a first NVMe controller 312 and a second NVMe controller 312 with each NVMe controller respectively coupled to a port for host communication.” The NVM subsystem is “integrated within a data processing unit (DPU) … which may be integrated into a SmartNIC” (DPU integrated into a SmartNIC → processing unit). Id. at ¶¶ 38-39; See Also ¶¶ 53-54 (“SmartNIC 625 may comprise a DPU 630 that incorporates an NVMe subsystem to support multiple SPDK instances”).
Claim 19
Bhide discloses wherein the queried context information is migrated in a stateless manner. The network state “is migrated from the source edge device before the workload is ready to run at the destination edge device.” Bhide, ¶¶ 43-44. In embodiments, “the context of each networking interface associated with a workload can be migrated to the destination device in parallel.” Id. at ¶ 54. The “network state is synchronized with deltas between the source edge device 216 and the destination edge device 226 when workload is ready to resume at the destination device 226 after being shut down on the source device 216.” Id. at ¶ 57. The network state “is incremental synchronized (e.g., via a delta of the network state since the last update) between the source edge device and destination edge device on completion of the initial synchronization” (incremental synchronization → stateless migration). Id. at ¶ 65.
Claim 20
Glimcher discloses wherein the circuits are further to control, using the alternate emulation software process, communications by one or more additional devices. Glimcher discloses “embodiments for Smart-NIC software upgrading without disruption to a host application.” Glimcher, ¶ 33. In one embodiment, “applications, e.g. containers, virtual machines (VMs), or native running apps, may consume emulated NVMe/PCIe block devices,” wherein the “emulated device may be created to handle submission and completion queue pairs (QPs) as well as doorbells and interrupts.” The target implementation of an emulated device is “running as a storage performance development kit (SPDK) application in a polling mode, handling both administrative operations as well as I/O … operations.” Id. at ¶¶ 45-46. An NVMe controller is “a software-defined emulation module to enable hardware accelerated virtualization of an emulated NVMe target 354” (NVMe controller → emulation software process). The NVMe target 534 “is a singleton SPDK-based application running inside the SmartNIC and provides emulation for all the emulated NVMe devices.” Id. at ¶ 49. Figure 3 illustrates a “shared namespace feature of NVMe protocol” with an “NVM subsystem 310” comprising “a first NVMe controller 312 and a second NVMe controller 312 with each NVMe controller respectively coupled to a port for host communication.” The NVM subsystem is “integrated within a data processing unit (DPU) … which may be integrated into a SmartNIC” (DPU integrated into a SmartNIC → processing unit). Id. at ¶¶ 38-39; See Also ¶¶ 53-54 (“SmartNIC 625 may comprise a DPU 630 that incorporates an NVMe subsystem to support multiple SPDK instances”).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to FRANK D MILLS whose telephone number is (571)270-3194. The examiner can normally be reached M-F 10-6 ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, KEVIN YOUNG can be reached at (571)270-3180. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/FRANK D MILLS/Primary Examiner, Art Unit 2194 June 23, 2026
Prosecution Insights