Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Claims 1-20 are presented for examination.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 3, 8,10,11,13, 19 and 20 are rejected under 35 U.S.C. 102(a)(1) as anticipated by Chirammal (US 2018/0375936 A1)
As per claim 1, Chirammal teaches A resource management system, comprising: a plurality of worker nodes, wherein each of one or more candidate worker nodes of the plurality of worker nodes includes both computing resources and storage resources; (Chirammal [0026] In an example, physical hosts ll0A-B [worker nodes] may run one or more isolated guests, for example, VMs 112 and 116, storage containers 150 and 155 [storage resources], and service containers 160 [compute resources] and 165. In an example, any of storage and/or service containers 150, 155, 160, and 165 may be a container using any form of operating system level virtualization)
a master node communicatively connected to the plurality of worker nodes, wherein the master node includes a first scheduler and a second scheduler (Chirammal [0003] A storage controller and a container scheduler execute in conjunction with one or more processors communicatively coupled with the first memory. The container scheduler instantiates a first storage container on the first host and a second storage container on the second host. The storage controller configures the first storage container as a first storage node of a distributed file system and the second storage container as a second storage node of the distributed file system).
the first scheduler is configured to allocate at least part of the computing resources of the one or more candidate worker nodes for a scheduling task, (Chirammal [0030] In an example, container scheduler 145 may be a component responsible for assigning compute tasks executed in containers to various host nodes (e.g., VMs 112 and 116, physical hosts 110A-B). In an example, container scheduler 145 and/or storage controller 140 may be included in a container orchestrator (e.g., Kubernetes®)).
the second scheduler is configured to schedule at least part of the storage resources of the one or more candidate worker nodes for the scheduling task. (Chirammal [0041] In example system 400, container scheduler 145 receives a request to launch an additional service container for an online game (block 410). In the example, container scheduler 145 may query storage controller 140 for physical locations of storage nodes with available storage (block 412). Storage controller 140 may then respond with a list of host nodes hosting storage containers with available storage (block 414). In the example, storage controller 140 may further provide node, rack, and/or zone identifiers for the available storage containers [0044] In example system 500, container scheduler 145 receives a request to launch yet another additional service container for the online game (block 510). In an example, container scheduler 145 again queries storage controller 140 for locations of storage nodes with available storage (block 512). In response, storage controller 140 sends a list of host nodes hosting storage containers with available storage (block 514).).
As per claim 3, Chirammal teaches wherein to schedule at least part of the storage resources of the one or more worker nodes for the scheduling task, the second scheduler is configured to:
determine whether the implementation of the scheduling task needs storage resources; (Chirammal [0026] In an example, physical hosts ll0A-B [worker nodes] may run one or more isolated guests, for example, VMs 112 and 116, storage containers 150 and 155 [storage resources], and service containers 160 [compute resources] and 165. In an example, any of storage and/or service containers 150, 155, 160, and 165 may be a container using any form of operating system level virtualization)
in response to determining that the implementation of the scheduling task needs storage resources, determine the one or more candidate worker nodes from the plurality of worker nodes; determine a target worker node from the one or more candidate worker nodes for the scheduling task; (Chirammal [0041] In example system 400, container scheduler 145 receives a request to launch an additional service container for an online game (block 410). In the example, container scheduler 145 may query storage controller 140 for physical locations of storage nodes with available storage (block 412). Storage controller 140 may then respond with a list of host nodes hosting storage containers with available storage (block 414). In the example, storage controller 140 may further provide node, rack, and/or zone identifiers for the available storage containers [0044] In example system 500, container scheduler 145 receives a request to launch yet another additional service container for the online game (block 510). In an example, container scheduler 145 again queries storage controller 140 for locations of storage nodes with available storage (block 512). In response, storage controller 140 sends a list of host nodes hosting storage containers with available storage (block 514))
schedule at least part of the storage resources of the target worker node for the scheduling task. (Chirammal [0041] In example system 400, container scheduler 145 receives a request to launch an additional service container for an online game (block 410). In the example, container scheduler 145 may query storage controller 140 for physical locations of storage nodes with available storage (block 412). Storage controller 140 may then respond with a list of host nodes hosting storage containers with available storage (block 414). In the example, storage controller 140 may further provide node, rack, and/or zone identifiers for the available storage containers [0044] In example system 500, container scheduler 145 receives a request to launch yet another additional service container for the online game (block 510). In an example, container scheduler 145 again queries storage controller 140 for locations of storage nodes with available storage (block 512). In response, storage controller 140 sends a list of host nodes hosting storage containers with available storage (block 514) [0061] Storage controller 1040 receives persistent volume claim (“PVC”) 1035 associated with service container 1082. Storage controller 1040 creates persistent storage volume 1054 in storage node 1052 based on persistent volume claim 1035. The persistent storage volume 1054 is mapped to service container 1082, where content 1070A of persistent storage volume 1054 is replicated to storage node 1057 (e.g., as content 1070B)).
As per claim 8, Chirammal teaches wherein the storage resource includes at least one of a hard disk drive (HHD) storage resource or a solid state drive (SSD) storage resource. (Chirammal [0017] In a typical example, such persistent storage may store data in devices such as hard drive disks (“HDD”), solid state drives (“SSD”), and/or persistent memory (e.g., Non-Volatile Dual In-line Memory Module (“NVDIMM”))).
As per claim 10, Chirammal teaches wherein the plurality of worker nodes and the master node form a Kubernetes cluster. (Chirammal [0030] In an example, container scheduler 145 may be a component responsible for assigning compute tasks executed in containers to various host nodes (e.g., VMs 112 and 116, physical hosts 110A-B). In an example, container scheduler 145 and/or storage controller 140 may be included in a container orchestrator (e.g., Kubernetes®). [0049] In an example, orchestrator 740 may be a comprehensive containerization management suite (e.g., Red Hat® OpenShift®, Kubernetes®) including storage controller 742 and container scheduler 745. In an example, orchestrator 740 may allow for the discovery of container deployment solutions to efficiently locate service containers (e.g., service containers 761-763) in close physical and/or network proximity to persistent storage volumes associated with each service container (e.g., persistent storages 755-757). In the example, a comprehensive service such as an e-commerce website may include several standalone components. In an example, service cluster 760 may be a group of associated service containers (e.g., service containers 761-763) that combine to deliver a more comprehensive service. For example, service container 761 may provide a product search service, service container 762 may provide a shopping cart service, and service container 763 may provide a payment service. In an example, service containers 761-763 may map associated persistent storages 755-757 on storage node 752 located on a shared physical host 710A. In an example, service cluster 760 may be deployed to VM 714 rather than VM 712 due to a shortage of compute resources (e.g., CPU, memory, disk, persistent memory, network bandwidth, etc.) on VM 712.)
As to claims 11 and 20, they are rejected based on the same reason as claim 1.
As to claim 13, it is rejected based on the same reason as claim 3.
As to claim 19, it is rejected based on the same reason as claim 10.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 2 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Chirammal (US 2018/0375936 A1) in view of Chinnam (US 10,754,696 B1).
As per claim 2, Chirammal does not teach receive a storage planning instruction input by a user; generate, based on the storage planning instruction, storage planning information relating to the storage resources in the candidate worker node; and store the storage planning information in an annotation of the candidate worker node.
However, Chiannam teaches receive a storage planning instruction input by a user; generate, based on the storage planning instruction, storage planning information relating to the storage resources in the candidate worker node; and store the storage planning information in an annotation of the candidate worker node. (Chiannam [col 6, lines 18-38] (23) FIG. 2B is a flow chart that illustrates an overall process of load balancing a cluster system, in some embodiments. The process starts by determining that a resource threshold for a node (“source” node or “first” node) has been reached (met or exceeded) by data requests from a client, step 222. The threshold can be either a storage capacity, CPU cycle, interface line rate, or any other appropriate threshold. The threshold values act as triggers to determine whether or not the load balancer will initiate a transfer of data, file, or IP addresses from the source node to a target node. In an embodiment, the threshold values for CPU usage and storage capacity are set to default values upon system configuration and initialization, but are dynamic during runtime. That is, they can be modified by the user through inputs that tune the performance of the nodes, or, or though user policies that set performance limits or constraints on node resources. The thresholds may be expressed as an absolute value (e.g., 1 TB, 2 TB, 4 TB of storage space, etc.), or as a percentage utilization of a maximum resource (e.g., 70% max CPU cycles, etc.). [col 8, lines 9-28] As shown in FIG. 3, each VCF node 304 samples on a periodic basis (e.g., every minute) its CPU, storage, stream, and other usage and stores this data in its own database 310. The VCM node 302 collects this data and aggregates it into a single database 320. The load balancer 306 queries this database 320 to determine whether or not to initiate a file migration from a node that exhibits overuse based on defined storage/CPU thresholds. The file migration workflow may require an additional workflow to increase target node capacity or spawn new nodes. Sampling of node statistics may be done on any appropriate time scale, such as minutes or several minutes, and in general, data migration is done on a substantially longer time scale, such once daily or once every several hours. The act of data migration itself can be disruptive and consume system resources, so initiating data migrations based user defined policies and thresholds (trigger conditions) can be tailored based on system needs and constraints through user tunable parameters, such as threshold values, and performance/storage settings).
It would have been obvious to a person in the ordinary skill in the art before the effective filing date of the claimed invention to combine Chinnam with the system of Chirammal to generate storage planning information. One having ordinary skill in the art would have been motivated to use Chinnam into the system of Chirammal for the purpose of load balancing backup appliances in a cluster system (Chinnam [col 1, lines 6-8])
As to claim 12, it is rejected based on the same reason as claim 2.
Claims 4 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Chirammal (US 2018/0375936 A1) in view of Tsubaki US 2022/0083425 Al).
As per claim 4, Chirammal teaches wherein to determine a target worker node from the one or more candidate worker nodes for the scheduling task, the second scheduler is configured to: obtain a persistent volume claim (PVC) corresponding to the scheduling task; (Chirammal [0047] Storage controller 640 receives persistent volume claim (“PVC”) 635 associated with service container 662. Storage controller 640 creates persistent storage volume 654 in storage node 652 based on persistent volume claim 635. The persistent storage volume 654 is mapped to service container 662, where content 670A of persistent storage volume 654 is replicated to storage node 657 (e.g., as content 670B))
select, from the one or more candidate worker nodes, at least one candidate worker node whose storage planning information in its annotation satisfies a condition defined in the PVC; (Chirammal [0041] In example system 400, container scheduler 145 receives a request to launch an additional service container for an online game (block 410). In the example, container scheduler 145 may query storage controller 140 for physical locations of storage nodes with available storage (block 412). Storage controller 140 may then respond with a list of host nodes hosting storage containers with available storage (block 414). In the example, storage controller 140 may further provide node, rack, and/or zone identifiers for the available storage containers
determine the target worker node from the at least one selected candidate worker node. (Chirammal [0061] Storage controller 1040 receives persistent volume claim (“PVC”) 1035 associated with service container 1082. Storage controller 1040 creates persistent storage volume 1054 in storage node 1052 based on persistent volume claim 1035. The persistent storage volume 1054 is mapped to service container 1082, where content 1070A of persistent storage volume 1054 is replicated to storage node 1057 (e.g., as content 1070B)).
Chirammal does not teach for each of the one or more candidate worker nodes, obtain an annotation of the candidate worker node.
However, Tsubaki teaches for each of the one or more candidate worker nodes, obtain an annotation of the candidate worker node; (Tsubaki [0050] In addition, the data acquisition unit 110 acquires node information from a management device deployed at each node 10 or a management device that centrally manages the nodes 10 via the network, or from an input of the administrator, and stores the acquired node information in the storage unit 120. Here, the node information includes identification information about the node 10, and the various information about the storage of the node 10. The various information about the storage includes static information such as the storage capacity of the entire node 10 at which the storage is deployed, and the writing speed of the storage. In some examples, the storage information may include dynamically changing information such as the remaining storage capacity).
It would have been obvious to a person in the ordinary skill in the art before the effective filing date of the claimed invention to combine Tsubaki with the system of Chirammal to obtain an annotation of the candidate worker node. One having ordinary skill in the art would have been motivated to use Tsubaki into the system of Chirammal for the purpose of implementing a backup system that stores data stored in storage deployment. (Tsubaki paragraph 01)
As to claim 14, it is rejected based on the same reason as claim 4.
Claim 5 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Chirammal (US 2018/0375936 A1) in view of Wang (US 2019/0384515 A1).
As per claim 5, Chriammal does not teach generate a scheduling record recording that the at least part of the storage resources of the target worker node is scheduled for the scheduling task; and persistently store the scheduling record in the storage device.
However, Wang teaches generate a scheduling record recording that the at least part of the storage resources of the target worker node is scheduled for the scheduling task; and persistently store the scheduling record in the storage device. (Wang [0008] In some embodiments, the method further includes determining a priority of the resource quota checking task based at least in part on a number of processes for performing the resource quota checking task; and recording information and the priority of the resource quota checking task in a task list of the storage system, so as to schedule the resource quota checking task according to the priority).
It would have been obvious to a person in the ordinary skill in the art before the effective filing date of the claimed invention to combine Wang with the system of Chirammal to use storage for scheduling tasks. One having ordinary skill in the art would have been motivated to use Wang into the system of Chirammal for the purpose of performing the resource quota checking task with a first number of processes. (Wang paragraph 05)
As to claim 15, it is rejected based on the same reason as claim 5.
Claim 6 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Chirammal (US 2018/0375936 A1) in view of Wang (US 2019/0384515 A1) in further view of Datla (US 2014/0109097 Al).
As per claim 6, Wang teaches wherein each of the one or more candidate worker nodes comprises a container storage interface (CSI) configured to:
determine, based on the scheduling record, whether the candidate worker node corresponding to the CSI is the target worker node; (Wang [0008] In some embodiments, the method further includes determining a priority of the resource quota checking task based at least in part on a number of processes for performing the resource quota checking task; and recording information and the priority of the resource quota checking task in a task list of the storage system, so as to schedule the resource quota checking task according to the priority).
Wang does not teach in response to determining that the candidate worker node corresponding to the CSI is the target worker node, establish a persistent volume (PV) for the scheduling task on the storage resources of the candidate worker node corresponding to the CIS.
However, Datla teaches in response to determining that the candidate worker node corresponding to the CSI is the target worker node, establish a persistent volume (PV) for the scheduling task on the storage resources of the candidate worker node corresponding to the CIS. (Datla see Fig 3 Block 310 (Store the task definitions in a task library) and [0009] FIG. 6 is an example script of task identifiers of task definitions to create volumes on the storage component of the CI from FIG. 1.[0042] Also, scripts may be generated based on the techniques described herein to configure all of the components 110-116 of CI 106 (FIG. 1). For example, assume the task definitions include: a compute component task definition including a compute component command and a corresponding compute task argument to assign a server blade among a pool of server blades on compute component 114; a storage task definition including a storage component command and a corresponding storage task argument to create a storage volume on storage component 110; and a network task definition including a network component command and a corresponding network task argument to establish a network pool and address range on network component 112. Then a script may be generated to call the compute, storage, and network task definitions to configure the storage, network, and compute components, respectively)
It would have been obvious to a person in the ordinary skill in the art before the effective filing date of the claimed invention to combine Datla with the system of Chirammal and Wang to establish a persistent volume. One having ordinary skill in the art would have been motivated to use Datla into the system of Chirammal and Wang for the purpose of implementing an automated configuration of converged infrastructures. (Datla paragraph 02)
As to claim 16, it is rejected based on the same reason as claim 6.
Claim 7 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Chirammal (US 2018/0375936 A1) in view of Wang (US 2019/0384515 A1) in further view of Datla (US 2014/0109097 Al) in further view of Pakatci (US 2023/0020268 A1).
As per claim 7, Chirammal and Wang and Datla do not teach in response to determining that the PV has been established, remove the scheduling record from the storage device.
However, Pakatci teaches in response to determining that the PV has been established, remove the scheduling record from the storage device (Pakatci [0301] The example method depicted in FIG. 10 also includes performing (1020) the potential change to the execution environment of the storage system (1018). In the example method depicted in FIG. 10, the manner in which performing (1020) the potential change to the execution environment of the storage system (1018) is carried out may be dependent on the nature of the potential change. In an example where the potential change to the execution environment of the storage system (1018) includes adding a workload to the storage system (1018), performing (1020) the potential change to the execution environment of the storage system (1018) may be carried out, for example, by migrating data associated with the workload from a source storage system to the target storage system (1018), updating hosts that support the execution of an application associated with the workload to issue I/O requests to the target storage system (1018), and so on. Alternatively, in an example where the potential change to the execution environment of the storage system (1018) includes removing a workload from the storage system (1018), performing (1020) the potential change to the execution environment of the storage system (1018) may be carried out, for example, by deleting all unique data that is associated with the workload from the storage system (1018), updating host information if the workload is being migrated to another storage system, and so on)
It would have been obvious to a person in the ordinary skill in the art before the effective filing date of the claimed invention to combine Pakatci with the system of Chirammal and Wang and Datla to remove the scheduling record from the storage device. One having ordinary skill in the art would have been motivated to use Pakatci into the system of Chirammal and Wang and Datla for the purpose of providing data to computing devices 164A-B, monitoring and reporting of storage device utilization and performance (Pakatci paragraph 39)
As to claim 17, it is rejected based on the same reason as claim 7.
Claims 9 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Chirammal (US 2018/0375936 A1) in view of Guo (US 2017/0153918 A1)
As per claim 9, Chirammal does not teach wherein the second scheduler is embedded in the first scheduler using a plug-in.
However, Guo teaches wherein the second scheduler is embedded in the first scheduler using a plug-in. (Guo [0031] To increase utilization of the resources 150, the system 100 can be configured for the concurrent operation of workloads on the same machine, and to schedule and/or allocate resources and workloads across the system 100. To manage different resources 150, distributed resource managers can implement separate schedulers for different types of resources 150. For example, the system 100 can have a scheduler for managing central processing units (CPUs), a scheduler for memory, and a scheduler for storage. However, separate schedulers may fail to recognize relationships between different types of resources, and may require custom coding or plug-ins to handle non-standard resource hardware or non-standard resource requests).
It would have been obvious to a person in the ordinary skill in the art before the effective filing date of the claimed invention to combine Guo with the system of Chirammal to use embedded schedulers. One having ordinary skill in the art would have been motivated to use Guo into the system of Chirammal for the purpose of managing a resource in a distributed resource management system. (Guo paragraph 04)
As to claim 18, it is rejected based on the same reason as claim 9.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20240394398 A1 – discloses a memory storing instructions and a processor communicatively coupled to the memory and configured to execute the instructions to determine a host to execute a serverless function, the host included in a set of hosts of a serverless function platform configured to execute serverless functions, the serverless function defined to use a dataset of a storage system that is external to the serverless function platform; configure the storage system to provide access from the host to the dataset; and direct the host to provide access from the serverless function to the dataset and to execute the serverless function.
US 20220385647 A1 – discloses storage management system of a container system performing, for a worker node added to a cluster of the container system based on a first authentication of the worker node, a second authentication for the worker node, and determining, based on the second authentication, whether the worker node is authorized to perform one or more operations on a storage system associated with the cluster.
US 20220197567 A1 – discloses executing in a hosted cluster that is hosted by an infrastructure cluster (IC), receives a request to dynamically provision a hosted cluster (HC) persistent volume object that is coupled to a physical storage. The HC storage provisioner causes an IC control plane executing on the IC to generate IC volume metadata that is backed by a storage volume on the physical storage. The HC storage provisioner determines that the IC volume metadata has been generated. The HC storage provisioner creates HC volume metadata on the hosted cluster that is linked to the IC volume metadata, the HC volume metadata comprising an HC persistent volume object that represents a persistent volume for use by the hosted cluster that is backed, via the IC volume metadata, by the physical storage.
US 20210311759 A1 – discloses a host cluster including hosts executing a virtualization layer on hardware platforms thereof, the virtualization layer configured to support execution of virtual machines (VMs), the VMs including a pod VM, the pod VM including a container engine configured to support execution of containers in the pod VM, the pod VM including a first virtual disk attached thereto; and an orchestration control plane integrated with the virtualization layer, the orchestration control plane including a master server in communication with a pod VM controller, the pod VM controller configured to execute in the virtualization layer external to the VMs and cooperate with a pod VM agent in the pod VM, the pod VM agent generating root directories for the containers in the pod VM, each of the root directories comprising a union a read/write ephemeral layer stored on the first virtual disk and a read-only layer.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MEHRAN KAMRAN whose telephone number is (571)272-3401. The examiner can normally be reached on 9-5.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, April Blair can be reached on (571)270-1014. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MEHRAN KAMRAN/
Primary Examiner, Art Unit 2196
Prosecution Insights