DETAILED ACTION
1. Claims 1-20 are pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
2. Claims 1-2, 4-15, and 17-19 are rejected under 35 U.S.C. 101 because the claimed invention recites a judicial
exception, is directed to that judicial exception, an abstract idea, as it has not been integrated into
practical application and the claims do not recite significantly more than the judicial exception.
Step 1: Claims 1-11 are directed to methods, claims 12-20 are directed to machines and fall within the statutory categories of processes and machines. Therefore, “Are the claims to a process, machine, manufacture, or composition of matter?
Yes.
In order to evaluate the Step 2A inquiry “Is the claim directed to a law of nature, a natural
phenomenon or an abstract idea?” we must determine, at Step 2A Prong 1, whether the claim recites a law of nature, a natural phenomenon or an abstract idea and further whether the claim recites
additional elements that integrate the judicial exception into a practical application.
Step 2A Prong 1:
Claims 1, 12, and 18: The limitations of determining that the private cloud has insufficient available resources to host the pending workload by comparing a resource request of the pending workload to available resources of the private cloud; identifying a preemptible workload hosted by the private cloud based on infrastructure group metadata tags associated with already deployed workloads on the private cloud; determining, based on resources used by the preemptible workload, that the private cloud has sufficient available resources to host the pending workload when including the resources used by the preemptible workload; preempting the preemptible workload, as drafted, is a process that, but for the recitation of generic computing components, under the broadest reasonable interpretation, covers performance of the limitation in the mind. For example, determining that the private cloud has insufficient available resources to host the pending workload by comparing a resource request of the pending workload to available resources of the private cloud may be done by writing down the available resources of the private cloud using pen and paper and the required resources of a pending workload with pen and paper. Then, the comparison may be done by comparing the written resources to determine whether the resources of the cloud lacks the required amount to fulfill the request. Further, this may also be done via mathematical equations. See MPEP § 2106.04(a)(II)(B), “In addition, there are instances where a formula or equation is written in text format that should also be considered as falling within this grouping. For example, the phrase ‘determining a ratio of A to B’ is merely using a textual replacement for the particular equation (ratio = A/B). Additionally, the phrase ‘calculating the force of the object by multiplying its mass by its acceleration’ is using a textual replacement for the particular equation (F= ma)”. For example, suppose the resources used by the preemptible workload is represented by X, the available resources of the private cloud is represented by Y, and the required resources of the pending workload is represented by Z. Then, the limitation may simply be a comparison using the inequality X + Y
<
Z to determine there are insufficient resources.
Moreover, identifying a preemptible workload hosted by the private cloud based on infrastructure group metadata tags associated with already deployed workloads on the private cloud may be done by identifying tags associated with workloads based on the tag group using pen and paper. For example, a user may specify that only workloads with a tag of East-US is preemptible. Further, a user may use pen and paper to maintain a list of all deployed workloads on the cloud while adding the assigned tags to said workloads using pen and paper. Subsequently, said user may make a mental determination on which workloads are preemptible by identifying the workloads with tags associated with the tag of East-US in the example above using pen and paper.
Further, the limitation of determining, based on resources used by the preemptible workload, that the private cloud has sufficient available resources to host the pending workload when including the resources used by the preemptible workload may be done via mathematical equations and is considered an abstract idea. See at least MPEP § 2106.04(a)(II)(B), “In addition, there are instances where a formula or equation is written in text format that should also be considered as falling within this grouping. For example, the phrase ‘determining a ratio of A to B’ is merely using a textual replacement for the particular equation (ratio = A/B). Additionally, the phrase ‘calculating the force of the object by multiplying its mass by its acceleration’ is using a textual replacement for the particular equation (F= ma)”. For example, suppose the resources used by the preemptible workload is represented by X ,the available resources of the private cloud is represented by Y, and the required resources of the pending workload is represented by Z. Then, the limitation may simply be a comparison using the inequality X + Y
≥
Z to determine there are sufficient resources.
The limitation of preempting the preemptible workload may also be done in the mind. For example, one may decide to stop working on one task to work on another task with a higher priority. A task may be considered a workload and the decision to pause the work on a task corresponds to preempting a preemptible workload.
Therefore, Yes, claims 1, 12, and 18 recite judicial exceptions.
The claims have been identified to recite judicial exceptions, Step 2A Prong 2 will evaluate whether the claims are directed to the judicial exception.
Step 2A Prong 2:
Claims 1, 12, and 18: The judicial exception is not integrated into a practical application. In particular the claims recite the following additional elements — fetching, by a workload scheduler of a private cloud, a request to deploy a pending workload in the private cloud from a pending workload queue of the private cloud, is merely insignificant extra-solution data gathering activity (see MPEP § 2106.05(g)) and does not integrate a judicial exception into practical application. Examiner notes that these elements held to be merely insignificant extra-solution data gathering will be addressed below in Step 2B as further being Well-Understood, Routine, and Conventional (WURC).
Further, the additional limitation of preempting the preemptible workload on the private cloud is merely a recitation of field of use/technological environment and deploying the pending workload on the private cloud is considered insignificant extra-solution activity as deploying applications/workloads on the cloud is an example of insignificant application. See MPEP § 2106.05(g), “Below are examples of activities that the courts have found to be insignificant extra-solution activity: … Insignificant application: i. Cutting hair after first determining the hair style, In re Brown, 645 Fed. App'x 1014, 1016-1017 (Fed. Cir. 2016) (non-precedential); and ii. Printing or downloading generated menus, Ameranth, 842 F.3d at 1241-42, 120 USPQ2d at 1854-55.”.
Therefore, “Do the claims recite additional elements that integrate the judicial exception into a
practical application?” No, these additional elements do not integrate the abstract idea into a practical
application and they do not impose any meaningful limits on practicing the abstract idea. The claims are
directed to an abstract idea.
After having evaluated the inquiries set forth in Steps 2A Prongs 1 and 2, it has been concluded that claims 1, 12, and 18, not only recite a judicial exception but that the claims are directed to the judicial exception as the judicial exception has not been integrated into practical application.
Step 2B:
Claims 1, 12, and 18: The claims do not include additional elements, alone or in combination,
that are sufficient to amount to significantly more than the judicial exception. As discussed above with
respect to integration of the abstract idea into a practical application, the additional elements amount
to no more than insignificant extra-solution data gathering and transmission which do not amount to
significantly more than the abstract idea. Further, the insignificant extra-solution data gathering activity
is also WURC, see at least MPEP § 2106.05(d)(II) “The courts have recognized the following computer
functions as well‐understood, routine, and conventional functions when they are claimed in a merely
generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity. i. Receiving
or transmitting data over a network” where fetching a request, by a workload scheduler of a private cloud, as claimed, is receiving data over a network.
Having concluded analysis within the provided framework, Claims 1, 12, and 18 do not recite patent eligible subject matter under 35 U.S.C. 101.
Regarding claim 2, it recites the additional limitations of configuring, by the workload scheduler, a first preemption policy based on an infrastructure group priority of the pending workload; and identifying the preemptible workload based on the first preemption policy, which under the broadest reasonable interpretation, but for the recitation of generic computing components, covers the performance of the limitations in the mind. For example a first preemption policy based on an infrastructure group priority of the pending workload may simply be a policy to pause all tasks assigned a specific tag where each tag has a priority. Then, a list of preemptible workloads with assigned tags may be written down on paper and identifying the preemptible workloads based on the first preemptible policy may simply be reading through the list and identifying all workloads with a tag assigned a low priority. Therefore, claim 2 does not recite patent eligible subject matter under 35 U.S.C. 101.
Regarding claim 3, it recites the additional limitation of configuring, by the workload scheduler, a second preemption policy based on an infrastructure group priority of the pending workload and one or more of the threshold workload age, the threshold workload utilization, or the power status choice; and identifying the preemptible workload based on the second preemption policy which adds significantly more to the abstract idea. Therefore claim 3 recites patent eligible subject matter under 35 U.S.C. 101.
Regarding claim 4, it recites the additional limitation of wherein the preempting comprises temporarily disabling a host node executing the preemptible workload from allocating resources to new workload deployments, which is considered WURC. For example, this may be a simple application of resource locking. Therefore, claim 4 does not recite patent eligible subject matter under 35 U.S.C. 101.
Regarding claim 5, it recites the additional limitation of wherein the preempting comprises withdrawing resources allocated to the preemptible workload to make the resources available for allocation to the pending workload, which is considered WURC as withdrawing resources is a routine function done in many computer applications (E.g. database transactions, releasing unused resources, etc.). Therefore, claim 5 does not recite patent eligible subject matter under 35 U.S.C. 101.
Regarding claim 6, it recites the additional limitation of wherein the preempting further comprises enabling the host node to allocate the resources to the pending workload after the resources are withdrawn from the preemptible workload, which is considered WURC as this may simply be unlocking previously locked resources. Therefore, claim 6 does not recite patent eligible subject matter under 35 U.S.C. 101.
Regarding claim 7, it recites the additional limitation of wherein the preempting further comprises creating, by the workload scheduler, a backup of the preemptible workload on a backup repository, which is considered WURC as data backups are a common and routine function in many cloud applications for disaster recovery. Therefore, claim 7 does not recite patent eligible subject matter under 35 U.S.C. 101.
Regarding claim 8, it recites the additional limitation of receiving, by the workload scheduler, the request for workload deployment, which is merely insignificant extra solution data gathering activity. Further, the limitation of updating, by the workload scheduler, the pending workload queue with the request is considered WURC. See MPEP § 2106.05(d)(II) “iii. Electronic recordkeeping, Alice Corp. Pty. Ltd. v. CLS Bank Int'l, 573 U.S. 208, 225, 110 USPQ2d 1984 (2014) (creating and maintaining "shadow accounts"); Ultramercial, 772 F.3d at 716, 112 USPQ2d at 1755 (updating an activity log)”. Therefore, claim 8 does not recite patent eligible subject matter under 35 U.S.C. 101.
Regarding claim 9, it recites the additional limitation of maintaining, by the workload scheduler, the request to deploy the pending workload in the pending workload queue responsive to determining that the private cloud does not comprise the preemptible workload, which is considered WURC as this is simply a conditional statement to make no changes to a queue if a condition is not met. Such conditional changes to a data structure like a queue is a common and routine function. Therefore, claim 9 does not recite patent eligible subject matter under 35 U.S.C. 101.
Regarding claim 10, it recites the additional limitation of updating, by the workload scheduler, a running workload repository to store a new entry indicating the pending workload as an already-deployed workload after the pending workload is deployed in the private cloud, which is considered WURC as this is simply adding an entry to an existing queue and is considered a common function. Therefore, claim 10 does not recite patent eligible subject matter under 35 U.S.C. 101.
Regarding claim 11, it recites the additional limitation of wherein the workload scheduler is accessible to a tenant of the private cloud via a private cloud management platform is merely a recitation of field of use. Therefore, claim 11 does not recite patent eligible subject matter under 35 U.S.C. 101.
Regarding claim 13, it recites the additional limitation to identify preemptible workload based on a first preemption policy and infrastructure group priorities defined for an infrastructure group metadata tag assigned to the pending workload and the infrastructure group metadata tags assigned to already-deployed workloads, wherein the first preemption policy defines workload selection criteria based on the infrastructure group priorities, under the broadest reasonable interpretation, but for the recitation of generic computing components, covers performance of the limitations in the mind. For example, identifying preemptible workload based on a first preemption policy and infrastructure group priorities defined for an infrastructure group metadata tag assigned to the pending workload may simply be reading off a paper of a list of pending workloads and identifying workloads assigned with a tag indicating preemption. For example, the policy may simply be all workloads assigned with a tag of X has the lowest priority and is therefore preemptible. Moreover, the additional limitation of wherein the first preemption policy defines workload selection criteria based on the infrastructure group priorities, under the broadest reasonable interpretation, covers performance of the limitations in the mind. For example, the policy may simply be to choose the tag associated with the lowest priority. Then, one may simply go through a list of workloads on a piece of paper, where their assigned tags are also written on said paper, and identify the workloads with the tags assigned the lowest priority for preemption. Therefore, claim 13 does not recite patent eligible subject matter under 35 U.S.C. 101.
Regarding claim 14, it recites the additional limitation of wherein the pending workload and the already-deployed workloads on the private cloud comprise one or more of virtual machines, containers, executable applications, pods, or combinations thereof, which is merely a recitation of field of use/technological environment. Therefore, it does not recite patent eligible subject matter under 35 U.S.C. 101.
Regarding claim 15, it recites the additional limitations to configure the first preemption policy based on a priority criterion corresponding to the infrastructure group priorities, wherein the priority criterion comprises one or more priority thresholds and a condition to select the already-deployed workloads based on the one or more priority thresholds, which under the broadest reasonable interpretation, covers performance of the limitations in the mind. For example, configuring the first preemption policy based on a priority criterion corresponding to the infrastructure group priorities may simply be to choose a group with the lowest priority which may be done in the mind. Further, this being based on the one or more priority thresholds may be done via mathematical equations such as inequalities where the condition to select the already-deployed workloads based on the one or more priority thresholds is based on said inequality. Therefore, claim 15 does not recite patent eligible subject matter under 35 U.S.C. 101.
Regarding claim 16, it recites the additional limitation to configure a second preemption policy based on an infrastructure group priority of the pending workload and one or more of the threshold workload age, the threshold workload utilization, or the power status choice; and identify the preemptible workload based on the second preemption policy, which adds significantly more to the abstract idea. Therefore claim 16 recites patent eligible subject matter under 35 U.S.C. 101.
Regarding claim 17, it recites the additional limitation of wherein the workload scheduler is subscribed for use by a tenant of the private cloud on a pay-per-use basis, which is merely a recitation of field of use. Therefore, claim 17 does not recite patent eligible subject matter under 35 U.S.C. 101.
Regarding claim 19, it contains similar limitations as claim 13 above. Therefore, claim 19 does not recite patent eligible subject matter under 35 U.S.C. 101. under the same rationale.
Regarding claim 20, it contains similar limitations as claim 3 above. Therefore, claim 20 recites patent eligible subject matter under 35 U.S.C. 101. Under the same rationale.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
3. Claims 1-2, 4-6, 8-9, 11-13, 15, and 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over Li et al. (US 11,150,951 B2) in further view of Dimitrov et al. (US 20200034206 A1).
4. Regarding claim 1, Li et al. teaches a method (Abstract: “A computer-implemented method… for releasable resource-based preemptive scheduling”), comprising:
Fetching, by a workload scheduler of a private cloud (Col. 2, lines 52-54: “This cloud model may include at least five characteristics, at least three service models, and at least four deployment models.”; Col. 3, lines 47-50: “Deployment Models are as follows: Private cloud: the cloud infrastructure is operated solely for an organization. It may be managed by the organization or a third party and may exist on-premises or off-premises.”; Col. 3, lines 66-67 and Col. 4, lines 1-3: “A cloud computing environment is service oriented with a focus on statelessness, low coupling, modularity, and semantic interoperability. At the heart of cloud computing is an infrastructure that includes a network of interconnected nodes.”; Col. 5, lines 46-53: “cloud computing environment 50 includes one or more cloud computing nodes 10 with which local computing devices used by cloud consumers, such as, for example, personal digital assistant (PDA) or cellular telephone 54A, desktop computer 54B, laptop computer 54C, and/or automobile computer system 54N may communicate (This means the nodes may be hosts). Nodes 10 may communicate with one another. “; Col. 8, lines 23-32: “A scheduler 410 is a machine (or machines) that processes requests for resources from batch jobs and distributes or dispatches the jobs to the host(s) 420 on which the jobs will run. A host 420 is communicatively coupled to the scheduler 410 (Scheduler 410 corresponds to the workload scheduler. The workload scheduler is part of a private cloud as a private cloud is included as a deployment model). A host 420 may be one of various kinds of machines including, but not limited to, computer workstations, servers, clients, etc. A plurality of hosts in the system may be the same with or different from each other and may be local to or remote from each other.”),
a request to deploy a pending workload in the private cloud from a pending workload queue of the private cloud (Col. 8, lines 23-26: “A scheduler 410 (Workload scheduler) is a machine (or machines) that processes requests for resources from batch jobs and distributes or dispatches the jobs to the host(s) 420 on which the jobs will run.”; Col. 8, lines 43-49: “The scheduler 410 may maintain a scheduling queue (Pending workload queue) to receive batch jobs. Batch jobs get scheduled to run in an order according to various known principles such as first-in-first-out (FIFO), highest priority first or shortest execution time first. Each pending job 430 in the scheduling queue has a resource requirement indicating how many resources it will need. For a job 440 currently running on the host 420, the scheduler may acquire information about how many resources are releasable from the job. The information may be obtained directly by the scheduler, or optionally, be collected and transmitted to the scheduler by a monitor 450 in the host 420”; Col. 9, lines 11-14: “As described above, step 510 may be performed at a scheduler. A pending workload (denoted by job i) may be received by the scheduling system in a scheduling queue at the scheduler”; Col. 10, lines 38-39: “At step 520, the pending workload is dispatched (Dispatching the workload involves fetching a request to dispatch (i.e. deploy) said pending workload first. It is from the workload scheduler since the workload scheduler is responsible for dispatching (deploying) the jobs/workloads.) so that it may use at least part of the releasable resources from the one or more currently running workloads to run.”);
determining, based on resources used by the preemptible workload, that the private cloud has insufficient available resources to host the pending workload by comparing a resource request of the pending workload to available resources of the private cloud (Col. 5, lines 52-54: “Nodes 10 may communicate with one another. They may be grouped (not shown) physically or virtually, in one or more networks, such as Private (The group of hosts may be part of one or more private clouds)”; Col. 9, lines 48-66: “Based on the required resources Rsv.sub.i of job i and the releasable resources Rls.sub.j from each job xj, one or more currently running jobs (denoted by job xk (1≤k≤n) to be preempted by job i are selected from job x1, job x2, . . . , job xm such that a sum of the quantities of releasable resources from job x1, job x2, . . . , job xn can satisfy the required resources of job i. As can be learned from Equation 1, the releasable sources from the one or more currently running jobs are calculated by subtracting locked resources of the one or more currently running workloads from total resources allocated to the one or more currently running workloads. Locked resources of the one or more currently running workloads may already include the constant quantities of resources of all of the one or more currently running workloads. In some cases, host x may already have some idle resources thereon (denoted by Avail.sub.x) that can be directly allocated to a job dispatched to it. Thus, the one or more currently pending jobs to be preempted by job i are determined according to the following equation.”. See image below; Col. 10, lines 30-39: “In some embodiments, the one or more currently running workloads determined to be preempted by the pending workload reside on a same host. In a scheduling system comprising a plurality of hosts, the scheduler will traverse each of the plurality of hosts until a host with one or more currently running workloads determined at step 510 is found (The determination uses equation 2 shown below. Determining a suitable host during traversal also includes determining if a private cloud has insufficient resources to host the requested workload.). At step 520, the pending workload is dispatched so that it may use at least part of the releasable resources from the one or more currently running workloads to run.“);
PNG
media_image1.png
376
1031
media_image1.png
Greyscale
Identifying a preemptible workload (Col. 9, lines 4-10: “FIG. 5 depicts a flowchart of a releasable resource-based preemptive scheduling method according to an embodiment of the present disclosure. At step 510, one or more currently running workloads are determined to be preempted by a pending workload, wherein releasable resources from the one or more currently running workloads meet demanded resource of the pending workload.”; Col. 9, lines 48-51: “Based on the required resources Rsv.sub.i of job i and the releasable resources Rls.sub.j from each job xj, one or more currently running jobs (denoted by job xk (1≤k≤n) to be preempted by job i are selected from job x1, job x2 (Identifying a preemptible workload)”; Col. 10, lines 42-44: “At step 610, the one or more currently running workloads as determined at step 510 are suspended to release the releasable resources.“);
Determining, based on resources used by the preemptible workload, that the private cloud has sufficient available resources to host the pending workload when including the resources used by the preemptible workload (Col. 9, lines 52-54: “such that a sum of the quantities of releasable resources (Sum of the resources of preemptible workloads) from job x1, job x2, . . . , job xn can satisfy the required resources of job i.”; Col. 10, lines 30-39: “In some embodiments, the one or more currently running workloads determined to be preempted by the pending workload reside on a same host. In a scheduling system comprising a plurality of hosts, the scheduler will traverse each of the plurality of hosts until a host with one or more currently running workloads determined at step 510 is found”; See image below. If RSVi (Resources required by the pending workload) is less than or equal to the available resources plus the sum of the releasable resources, it means the private cloud has sufficient resources to host the pending workload.);
PNG
media_image2.png
331
1031
media_image2.png
Greyscale
deploying the pending workload on the private cloud (Col. 10, lines 52-56: “at step 620, the pending workload is started (Deploying the pending workload) on the host or hosts (On the private cloud. The hosts are part of said private cloud) using at least part of the released resources. As such, the pending workload can operate normally without influencing other workloads that will still be running on the host or hosts.”).
However, Li et al. does not explicitly teach identifying a preemptible workload hosted by the private cloud to be based on infrastructure group metadata tags associated with already deployed workloads on the private cloud.
But Dimitrov et al. teaches using infrastructure group metadata tags associated with already deployed workloads to provide information about resources associated with said workloads ([0118]: “A tag is a label (Infrastructure group metadata tag) including metadata that can categorize or limit the associated resource according to the properties and/or other information in the tag. The tag can specify setting or restriction such as a group (e.g., a business or technology group, etc.), a zone, requirement, other option, etc.“; [0121]: “To address this limitation on the administrator's ability to affect host 602 selection and/or other aspect of blueprint 126 provisioning, one or more tags can be applied to resources, such as endpoints 608, 610, etc., defined in the blueprint 126 to place limits, criteria, and/or other constraints on the configuration and deployment of such resources. For example, a tag placed on a resource can define one or more requirements related to placement of the resource. The placement links a user business group (e.g., development, finance, etc.) and a placement zone for the resource. The business group is used to associate a set of services and resources with a set of users.”; [0122]: “Placement can be leveraged by the host 602 to configure a container 114a and associated network, for example. Placement settings (e.g., a tag, etc.) can be used to link deployment of resource(s) associated with the placement to particular host(s) 602, container 114a definition(s), etc. (Infrastructure group metadata tag is associated with already deployed workloads). For example, a particular host 602 and deployment policy can be specified in a placement configuration for a business group and/or resource. Tagging the resource can tie the resource to the placement configuration, for example.”; [0129]: “ When a blueprint 126 and/or system configuration of resources is defined, tag(s) to be applied to a business group, resource, etc., can be specified to constrain users of one or more business groups with respect to configuration, deployment, use, etc., of one or more resources (Resources include workloads as a resource in the cloud is considered any object managed in the cloud such as VMs or containers)”) on a private cloud ([0031]: “Certain examples provide multi-cloud management systems that manage a combination of public and private clouds (e.g., a hybrid cloud environment) running a variety of computing processes from traditional processes to virtual machines to container (e.g., cloud native) workloads.”).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the invention of Li et al. with the teachings of Dimitrov et al. such that the identifying a preemptible workload hosted by the private cloud is based on infrastructure group metadata tags associated with already deployed workloads on the private cloud. This would be achieved by using the tags to determine underutilized/underused resources that can be “recovered” where such resources are considered preemptible to improve the efficiency of running cloud infrastructure as taught by Dimitrov et al. ([0062]: “The resource manager 144 of the illustrated example facilitates recovery of cloud computing resources of the cloud provider 110 that are no longer being activity utilized. Automated reclamation may include identification, verification and/or reclamation of unused, underutilized, etc. resources to improve the efficiency of the running cloud infrastructure.”; [0118]: “A tag is a label including metadata that can categorize or limit the associated resource (This metadata may be used to determine if a resource is underutilized or unused) according to the properties and/or other information in the tag.“).
5. Regarding claim 2, Li et al. modified by Dimitrov et al. teaches the method of claim 1, further comprising:
Li et al. teaches configuring, by the workload scheduler, a first preemption policy of the based on a priority of the pending workload (Col. 8, lines 43-49: “The scheduler 410 (Workload scheduler) may maintain a scheduling queue to receive batch jobs. Batch jobs get scheduled to run in an order according to various known principles such as first-in-first-out (FIFO), highest priority first or shortest execution time first. Each pending job 430 in the scheduling queue has a resource requirement indicating how many resources it will need.”; Col. 9, lines 1-3: “The scheduler 410 may further take the priorities of the jobs into consideration in making preemption decisions”; Col. 10, lines 1-8: “In some embodiments, the preemptive scheduling method based on releasable resources of currently running workloads to be preempted may be implemented in further consideration of priorities (First preemption policy) of the currently running workloads and the pending workload. Specifically, the one or more currently running workloads to be preempted may have lower priorities than that of the preemptive pending workload”); and
Identifying the preemptible workload based on the first preemption policy (Col. 10, lines 1-8: “In some embodiments, the preemptive scheduling method based on releasable resources of currently running workloads to be preempted may be implemented in further consideration of priorities of the currently running workloads and the pending workload. Specifically, the one or more currently running workloads to be preempted may have lower priorities than that of the preemptive pending workload”; Col. 10, lines 33-36: “the scheduler will traverse each of the plurality of hosts until a host with one or more currently running workloads determined at step 510 is found.”).
However, Li et al. does not explicitly teach the priority of the pending workload to be an infrastructure group priority.
But Dimitrov et al. teaches using an infrastructure group priority ([0121]: “a tag placed on a resource can define one or more requirements related to placement of the resource. The placement links a user business group (e.g., development, finance, etc.) and a placement zone for the resource.”; [0122]: “based on the user's business group, one or more placements can be selected. Each placement defines requirements for the placement and a placement zone. For example, placements and placement settings can be used to limit and reserve resources used by a business group. The placement can also set a priority to reserve an amount of processor and/or memory resources (and/or other resource(s)) to be allocated and/or otherwise made available to the placement.”; [0123]: “an associated container 114a is provisioned, placements are filtered based on business group, available resources, and priority, for example.”; [0125]-[0128]: “For example, a placement can be represented as follows:
Placement 1: group: dev, zone: us-east, requirements: option:no-backup group: finance, zone: us-east, requirements: fastSSD, encrypted“. The priorities associated with groups such as dev or finance correspond to an infrastructure group priority), and
Li et al. teaches placing groups of pending workloads (i.e. batch jobs) in the scheduler (Col. 8, lines 43-47: “The scheduler 410 may maintain a scheduling queue to receive batch jobs. Batch jobs get scheduled to run in an order according to various known principles such as first-in-first-out (FIFO), highest priority first or shortest execution time first”).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the invention of Li et al. with the teachings of Dimitrov et al. such that the priority used for preemption is based on an infrastructure group priority. This would be achieved by assigning the tags of Dimitrov et al. to the batch jobs in Li et al. in order to separate the batch jobs by their group placement such as dev or finance as described in Dimitrov et al. above. Doing so allows a user to reserve and limit the number of resources allowed based on the priority of a workload such that more resources may remain for higher priority workloads as taught in Dimitrov et al. ([0122]: “placements and placement settings can be used to limit and reserve resources used by a business group. The placement can also set a priority to reserve an amount of processor and/or memory resources (and/or other resource(s)) to be allocated and/or otherwise made available to the placement. Placement can be leveraged by the host 602 to configure a container 114a and associated network, for example. Placement settings (e.g., a tag, etc.) can be used to link deployment of resource(s) associated with the placement to particular host(s) 602, container 114a definition(s), etc. For example, a particular host 602 and deployment policy can be specified in a placement configuration for a business group and/or resource. Tagging the resource can tie the resource to the placement configuration, for example.”). The priorities corresponding to these groups corresponds to the infrastructure group priority.
6. Regarding claim 4, Li et al. modified by Dimitrov et al. teaches the method of claim 1,
Li et al. teaches wherein the preempting comprises temporarily disabling a host node executing the preemptible workload from allocating resources to new workload deployments (Col. 7, lines 9-19: “Furthermore, in the stopping-and-resuming way, a workload may have a portion of its reserved resources not releasable to be used by a preemptive workload. The portion of resources is referred to herein as locked resources. For example, locked memory may be used in large quantities of workloads to improve their efficiency or security, or to reduce the overheads. The locked memory of these workloads will not be swapped out in preemption (Locking resources correspond to disabling a host node from allocating resources to new workload deployments). For real-time applications, frequent memory swapping out will delay their execution. Thus, it is beneficial to set a portion of the reserved resources as being locked.”).
7. Regarding claim 5, Li et al. modified by Dimitrov et al. teaches the method of claim 4, wherein the preempting comprises
Li et al. teaches withdrawing resources allocated to the preemptible workload to make the resources available for allocation to the pending workload (Col. 10, lines 42-56: “At step 610, the one or more currently running workloads as determined at step 510 are suspended to release the releasable resources. In response to determining, according to step 510, that one or more currently running workloads are appropriate to be preempted by a pending workload, the scheduler may signal the corresponding host or hosts on which the currently running workloads are running to stop processes of the currently running workloads. In such a way, resources previously reserved on the host or hosts for the currently running workloads are released (Releasing resources for the pending workload corresponds to withdrawing the resources. Releasable resources corresponds to the resources allocated to the preemptible workload.). Next, at step 620, the pending workload is started on the host or hosts using at least part of the released resources. As such, the pending workload can operate normally without influencing other workloads that will still be running on the host or hosts.”).
8. Regarding claim 6, Li et al. modified by Dimitrov et al. teaches the method of claim 5, wherein the preempting further comprises
Li et al. teaches enabling the host node to allocate the resources to the pending workload after the resources are withdrawn from the preemptible workload (Col. 10, lines 47-54: “the scheduler may signal the corresponding host or hosts on which the currently running workloads are running to stop processes of the currently running workloads. In such a way, resources previously reserved on the host or hosts for the currently running workloads are released. Next, at step 620, the pending workload is started on the host or hosts using at least part of the released resources (The released resources corresponds to the withdrawn resources, and using the released resources means the withdrawn resources were allocated to the pending workload. Therefore, the host was enabled to allocate resources to the pending workload after the resources were withdrawn from the preemptible workload)”).
9. Regarding claim 8, Li et al. modified by Dimitrov et al. teaches the method of claim 1, further comprising:
Li et al. teaches receiving, by the workload scheduler, the request for workload deployment (Col. 8, lines 23-26: “A scheduler 410 is a machine (or machines) that processes requests for resources from batch jobs and distributes or dispatches the jobs to the host(s) 420 on which the jobs will run”; Col. 10, lines 37-39: “At step 520, the pending workload is dispatched (Dispatching the pending workload involves receiving a request to deploy said workload first. It is received by the workload scheduler as the workload scheduler is responsible for dispatching and processing requests.) so that it may use at least part of the releasable resources from the one or more currently running workloads to run.”); and
Updating, by the workload scheduler, the pending workload queue with the request (Col. 8, lines 43-47: “The scheduler 410 may maintain a scheduling queue to receive batch jobs. Batch jobs get scheduled to run in an order according to various known principles such as first-in-first-out (FIFO), highest priority first or shortest execution time first.“; Col. 9, lines 11-14: “As described above, step 510 may be performed at a scheduler. A pending workload (denoted by job i) may be received by the scheduling system in a scheduling queue at the scheduler (Adding the pending workload to the scheduling queue corresponds to updating the pending workload queue with the request)”).
10. Regarding claim 9, Li et al. modified by Dimitrov et al. teaches the method of claim 1,
Li et al. teaches further comprising maintaining, by the workload scheduler, the request to deploy the pending workload in the pending workload queue responsive to determining that the private cloud does not comprise the preemptible workload (Col. 8, lines 60-65: “Based on the resource requirement of a pending job 430 and information about releasable resources from the currently running jobs 440, the scheduler 410 can determine accurately which ones of the currently running jobs 440 are to be stopped and to release resources for the pending job 430.”; Col. 9, lines 48-54: “Based on the required resources Rsv.sub.i of job i and the releasable resources Rls.sub.j from each job xj, one or more currently running jobs (denoted by job xk (1≤k≤n) to be preempted by job i are selected from job x1, job x2, . . . , job xm such that a sum of the quantities of releasable resources from job x1, job x2, . . . , job xn can satisfy the required resources of job i.”; Col. 10, lines 36-39: “At step 520, the pending workload is dispatched so that it may use at least part of the releasable resources from the one or more currently running workloads to run”; Col. 10, lines 32-36: “In a scheduling system comprising a plurality of hosts, the scheduler will traverse each of the plurality of hosts until a host with one or more currently running workloads determined at step 510 is found.”; (This means that if a suitable host is not found due to having no preemptible workloads, the pending workload will not be dispatched and the request to deploy the pending workload remains in the scheduling queue. Determining the private cloud does not contain any hosts with preemptible workloads corresponds to determining the private cloud does not comprise the preemptible workload.)”).
11. Regarding claim 12, it is a media/product type claim with similar limitations as claim 1 above. Moreover, Li et al. discloses the additional limitations of a workload scheduler (Col. 8, lines 23-26: “A scheduler 410 (Workload scheduler) is a machine (or machines) that processes requests for resources from batch jobs and distributes or dispatches the jobs to the host(s) 420 on which the jobs will run”), a machine-readable storage medium storing executable instructions (Col. 1, lines 57-60: “The computer system comprises a processor and a computer-readable memory coupled to the processor, the memory comprising instructions that when executed by the processor”), and a pending workload queue comprising a request to deploy a pending workload (Col. 8, lines 43-49: “The scheduler 410 may maintain a scheduling queue (Pending workload queue) to receive batch jobs. Batch jobs get scheduled to run in an order according to various known principles such as first-in-first-out (FIFO), highest priority first or shortest execution time first. Each pending job 430 (Each pending job in the scheduler corresponds to a request to deploy the pending workload.) in the scheduling queue has a resource requirement indicating how many resources it will need.”). Therefore, it is rejected under the same rationale.
12. Regarding claim 13, Li et al. modified by Dimitrov et al. teaches the workload scheduler of claim 12,
Li et al. teaches wherein the processing resource is configured to execute one or more instructions (Col. 1, lines 64-67 and Col. 2, line 1: “The computer program product comprises a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor”) to identify preemptible workload based on a first preemption policy and group priorities (Col. 8, lines 43-49: “The scheduler 410 (Workload scheduler) may maintain a scheduling queue to receive batch jobs. Batch jobs get scheduled to run in an order according to various known principles such as first-in-first-out (FIFO), highest priority first (The priorities of bath jobs correspond to a group priority where a batch of jobs correspond to a group) or shortest execution time first. Each pending job 430 in the scheduling queue has a resource requirement indicating how many resources it will need.”; Col. 9, lines 1-3: “The scheduler 410 may further take the priorities of the jobs into consideration in making preemption decisions”; Col. 10, lines 1-8: “In some embodiments, the preemptive scheduling method based on releasable resources of currently running workloads to be preempted may be implemented in further consideration of priorities (First preemption policy) of the currently running workloads and the pending workload. Specifically, the one or more currently running workloads to be preempted may have lower priorities than that of the preemptive pending workload”),
Wherein the first preemption policy defines workload selection criteria based on group priorities (Col. 8, lines 43-47: “The scheduler 410 may maintain a scheduling queue to receive batch jobs. Batch jobs get scheduled to run in an order according to various known principles such as first-in-first-out (FIFO), highest priority first (Priorities of a batch of jobs is a group priority) or shortest execution time first.“; Col. 9, lines 1-3: “The scheduler 410 may further take the priorities of the jobs into consideration in making preemption decisions.”).
However, Li et al. does not explicitly teach the infrastructure group priorities are defined for an infrastructure group metadata tag assigned to the pending workload and the group priorities to be infrastructure group priorities.
But Dimitrov et al. teaches assigning an infrastructure group metadata tag to workloads to determine priorities ([0118]: “In certain examples, provisioned resources (Workload), such as endpoints 608, 610, and/or other items specified in a blueprint 126, can be tagged to affect placement of resources in the multi-cloud system 600. A tag is a label including metadata that can categorize or limit the associated resource according to the properties and/or other information in the tag. The tag can specify setting or restriction such as a group (e.g., a business or technology group, etc.), a zone, requirement, other option, etc.; [0121]: “one or more tags can be applied to resources, such as endpoints 608, 610, etc., defined in the blueprint 126 to place limits, criteria, and/or other constraints on the configuration and deployment of such resources. For example, a tag placed on a resource can define one or more requirements related to placement of the resource. The placement links a user business group (e.g., development, finance, etc.) and a placement zone for the resource.”; [0122]: “For example, based on the user's business group, one or more placements can be selected. Each placement defines requirements for the placement and a placement zone. For example, placements and placement settings can be used to limit and reserve resources used by a business group. The placement can also set a priority to reserve an amount of processor and/or memory resources (and/or other resource(s)) to be allocated and/or otherwise made available to the placement.”).
Moreover, Dimitrov et al. teaches using an infrastructure group priority ([0121]: “a tag placed on a resource can define one or more requirements related to placement of the resource. The placement links a user business group (e.g., development, finance, etc.) and a placement zone for the resource.”; [0122]: “based on the user's business group, one or more placements can be selected. Each placement defines requirements for the placement and a placement zone. For example, placements and placement settings can be used to limit and reserve resources used by a business group. The placement can also set a priority to reserve an amount of processor and/or memory resources (and/or other resource(s)) to be allocated and/or otherwise made available to the placement.”; [0123]: “hen an associated container 114a is provisioned, placements are filtered based on business group, available resources, and priority, for example.”; [0125]-[0128]: “For example, a placement can be represented as follows: Placement 1: group: dev, zone: us-east, requirements: option:no-backup group: finance, zone: us-east, requirements: fastSSD, encrypted“. The priorities associated with groups such as dev or finance correspond to an infrastructure group priority), and
Li et al. teaches placing groups of pending workloads (i.e. batch jobs) in the scheduler (Col. 8, lines 43-47: “The scheduler 410 may maintain a scheduling queue to receive batch jobs. Batch jobs get scheduled to run in an order according to various known principles such as first-in-first-out (FIFO), highest priority first or shortest execution time first”).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the invention of Li et al. with the teachings of Dimitrov et al. such that the priority used for preemption is based on an infrastructure group priority. This would be achieved by assigning the tags of Dimitrov et al. to the batch jobs in Li et al. that can separate the batch jobs by their group placement such as dev for finance as described in Dimitrov et al. above. Doing so allows a user to reserve and limit the number of resources allowed based on the priority of a workload such that more resources may remain for higher priority workloads as taught in Dimitrov et al. ([0122]: “placements and placement settings can be used to limit and reserve resources used by a business group. The placement can also set a priority to reserve an amount of processor and/or memory resources (and/or other resource(s)) to be allocated and/or otherwise made available to the placement. Placement can be leveraged by the host 602 to configure a container 114a and associated network, for example. Placement settings (e.g., a tag, etc.) can be used to link deployment of resource(s) associated with the placement to particular host(s) 602, container 114a definition(s), etc. For example, a particular host 602 and deployment policy can be specified in a placement configuration for a business group and/or resource. Tagging the resource can tie the resource to the placement configuration, for example.”). The priorities corresponding to these groups will correspond to the infrastructure group priority.
It would have also been obvious to one of ordinary skill in the art, that the invention of Li et al. modified by Dimitrov et al. that the infrastructure group priorities may be defined for an infrastructure group metadata tag assigned to the pending workload to set a priority for an amount of processor and/or memory resources to reserve for said pending workload as taught by Dimitrov et al. above. This reservation may then be used to determine the number of resources to lock from preemption when said pending workload becomes a running workload as previously taught by Li et al. above(Col. 7, lines 9-16: “Furthermore, in the stopping-and-resuming way, a workload may have a portion of its reserved resources not releasable to be used by a preemptive workload. The portion of resources is referred to herein as locked resources. For example, locked memory may be used in large quantities of workloads to improve their efficiency or security, or to reduce the overheads. The locked memory of these workloads will not be swapped out in preemption”).
13. Regarding claim 15, Li et al. modified by Dimitrov et al. teaches the workload scheduler of claim 13, wherein the processing resource is configured to execute one or more of the instructions to configure the first preemption policy based on
Li et al. teaches a priority criterion corresponding to the infrastructure group priorities (Col. 8, lines 43-47: “The scheduler 410 may maintain a scheduling queue to receive batch jobs. Batch jobs get scheduled to run in an order according to various known principles such as first-in-first-out (FIFO), highest priority first or shortest execution time first”),
Wherein the priority criterion comprises one or more priority thresholds (Col. 10, lines 5-16: “Specifically, the one or more currently running workloads to be preempted may have lower priorities than that of the preemptive pending workload. Still referring to the case that job x1, job x2, . . . , job xn are determined to be preempted by job i, the priorities of job x1, job x2, . . . , job xn (denoted by Pri.sub.x1, Pri.sub.x2, . . . , Pri.sub.xn) may be lower than the priority of job i (denoted by Pri.sub.i), which is represented in the following equation. Pri.sub.i>MAX(Pri.sub.x1,Pri.sub.x2, . . . ,Pri.sub.xn) (This corresponds to a priority threshold)“. See equation 3) and a condition to select the already-deployed workloads based on the one or more priority thresholds (Col. 8, lines 43-47: “The scheduler 410 may maintain a scheduling queue to receive batch jobs. Batch jobs get scheduled to run in an order according to various known principles such as first-in-first-out (FIFO), highest priority first”; Col. 10 lines 5-8: “Specifically, the one or more currently running workloads to be preempted may have lower priorities than that of the preemptive pending workload. (Condition to select the already-deployed workload based on one or more priority thresholds)”).
14. Regarding claim 18, it is a media/product type claim with similar limitations as claim 1 above. Moreover, Dimitrov et al. discloses the additional limitations of a non-transitory machine-readable medium storing instructions executable by a processing resource ([0194]: “Example 8 provides a non-transitory computer-readable storage medium comprising computer readable instructions that, when executed, cause at least one processor to at least implement a cloud management platform.”). Therefore, it is rejected under the same rationale.
15. Regarding claim 19, it is a media/product type claim with similar limitations as claim 13 above. Therefore, it is rejected under the same rationale.
16. Claims 3, 16, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Li et al. in further view of Dimitrov et al., as applied to claims 1 and 12, in further view of CHEN et al. (US 20170083367 A1).
17. Regarding claim 3, Li et al. modified by Dimitrov et al. teaches the method of claim 1,
Li et al. teaches using a preemption policy based on a group priority of the pending workload (Col. 8, lines 43-49: “The scheduler 410 (Workload scheduler) may maintain a scheduling queue to receive batch jobs. Batch jobs get scheduled to run in an order according to various known principles such as first-in-first-out (FIFO), highest priority first (The priority of batch jobs corresponds to a group priority. Jobs in the scheduler or jobs being scheduled corresponds to a pending workload.) or shortest execution time first. Each pending job 430 in the scheduling queue has a resource requirement indicating how many resources it will need.”; Col. 9, lines 1-3: “The scheduler 410 may further take the priorities of the jobs into consideration in making preemption decisions”; Col. 10, lines 1-8: “In some embodiments, the preemptive scheduling method based on releasable resources of currently running workloads to be preempted may be implemented in further consideration of priorities (Preemption policy) of the currently running workloads and the pending workload. Specifically, the one or more currently running workloads to be preempted may have lower priorities than that of the preemptive pending workload”) and
Dimitrov et al. teaches the priority to be an infrastructure group priority ([0121]: “a tag placed on a resource can define one or more requirements related to placement of the resource. The placement links a user business group (e.g., development, finance, etc.) and a placement zone for the resource.”; [0122]: “based on the user's business group, one or more placements can be selected. Each placement defines requirements for the placement and a placement zone. For example, placements and placement settings can be used to limit and reserve resources used by a business group. The placement can also set a priority to reserve an amount of processor and/or memory resources (and/or other resource(s)) to be allocated and/or otherwise made available to the placement.”; [0123]: “hen an associated container 114a is provisioned, placements are filtered based on business group, available resources, and priority, for example.”; [0125]-[0128]: “For example, a placement can be represented as follows: Placement 1: group: dev, zone: us-east, requirements: option:no-backup group: finance, zone: us-east, requirements: fastSSD, encrypted“. The priorities associated with groups such as dev or finance correspond to an infrastructure group priority).
Moreover, Dimitrov et al. teaches a power status choice ([0099]: “the user device 614 seeks to manage (e.g., enumerate/discover, provision/destroy, power on/off (A resource being on or off corresponds to the power status. E.g. on corresponds to running and off means unused/inactive), etc.) resources such as VMs, containers, etc.,”).
However, Li et al. modified by Dimitrov et al. do not explicitly teach receiving, by the workload scheduler, one or more of a threshold workload age, a threshold workload utilization, or a power status choice; configuring, by the workload scheduler, one or more of the threshold workload age, the threshold workload utilization, or the power status choice; and identifying the preemptible workload based on the second preemption policy.
But CHEN et al. teaches one or more of a threshold workload age ([0095]: “Another example selection criterion may be an age of the workload. If a workload has been operating for a longer period of time, it may be more detrimental to terminate such a workload as it may be expensive to have to restart and re-run the workload for that period of time. Therefore, in some examples, older workloads may be ranked lower than younger workloads for enforcement action.”), a threshold workload utilization ([0062]-[0063]: “At 504, the processor(s) can be configured to determine whether one or more utilization conditions are met, and if any workload's resource utilization exceeds the workload's resource allocation limit (Threshold workload utilization). In some embodiments, the utilization conditions can be met when utilization of a resource exceeds a defined utilization limit.“);
A preemption policy ([0067]: “In some examples, this condition may be set in recognition that since an overusing workload will take a certain amount of time to terminate and free its allocation, if a resource which is being consumed and may be exhausted soon, the system 10 should start a termination or other enforcement action far enough before that time to complete the enforcement action before complete resource exhaustion occurs”; [0072]-[0073]: “Enforcement actions (Preemption policy) include, for example, terminating, suspending, blocking, and/or checkpointing the workload. Terminating a workload can, in some examples, include killing any associated process, freeing any consumed resources and/or informing a scheduler or results manager. In some examples, the enforcement action may free resources, stop or slow the consumption of a resource, backup the state of a workload which is to be terminated or may be terminated soon, etc.”)
based on one or more of the threshold workload age ([0095]: “Another example selection criterion may be an age of the workload (Threshold workload age). If a workload has been operating for a longer period of time, it may be more detrimental to terminate such a workload as it may be expensive to have to restart and re-run the workload for that period of time. Therefore, in some examples, older workloads may be ranked lower than younger workloads for enforcement action.”), the threshold workload utilization ([0004]: “The at least one processor is configured for: monitoring utilization of the resource being used by at least one workload; and performing an enforcement action on a particular workload of the at least one workload when a utilization condition is met, and when the particular workload has a current resource utilization exceeding its associated resource allocation limit.”).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to further modify the invention of Li et al. modified by Dimitrov et al. with the teachings of CHEN et al. such that the workload scheduler may receive a threshold workload age to ensure the workload scheduler identifies a workload that has been running past the threshold workload age has a lower priority for being preempted. This is because releasing resources of such a workload runs the risk of forcing a workload to restart which may be more detrimental if said workload has been running for a long period of time as taught by CHEN et al. ([0095]: “Another example selection criterion may be an age of the workload. If a workload has been operating for a longer period of time, it may be more detrimental to terminate such a workload as it may be expensive to have to restart and re-run the workload for that period of time. Therefore, in some examples, older workloads may be ranked lower than younger workloads for enforcement action.”). Further it would have also been obvious to include the teachings of CHEN et al. such that the workload scheduler may receive a threshold workload utilization to identify preemptible workloads based on whether a workload’s resource utilization is under a limit due to unused resources and to further reallocate said unused resources to a pending workload as taught by Dimitrov et al. . ([0062]: “The resource manager 144 of the illustrated example facilitates recovery of cloud computing resources of the cloud provider 110 that are no longer being activity utilized. Automated reclamation may include identification, verification and/or reclamation of unused, underutilized, etc. resources to improve the efficiency of the running cloud infrastructure.”).
Further, it would have been obvious to include the power status choice such as whether a resource is powered on or off in the invention of Dimitrov et al. ([0099]: “the user device 614 seeks to manage (e.g., enumerate/discover, provision/destroy, power on/off…) resources such as VMs, containers…”) to allow for the suspension or termination of a workload when said workload’s utilization exceeds an allocation limit as taught by CHEN et al. ([0072]: “Enforcement actions include, for example, terminating, suspending, blocking, and/or checkpointing the workload.”; [0087]: “ It multiple workloads have resource utilizations exceeding their respective resource allocation limits, in some embodiments, at 608, the processor(s) can be configured to select one or more of the workloads on which to perform enforcement action(s).”).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the invention of Li et al. modified by Dimitrov et al. and CHEN et al. to configure, by the workload scheduler, a second preemption policy based on an infrastructure group priority of the pending workload using one or more of the threshold workload age, the threshold workload utilization, or a power status choice. A threshold workload age would be used to configure the priority of preemption between workloads where younger workloads have a higher priority for preemption as taught by CHEN et al. ([0095]: “in some examples, older workloads may be ranked lower than younger workloads for enforcement action”), and the threshold workload utilization would be used to identify preemptible workloads based on whether a workload’s resource utilization is under a limit due to unused resources and to further reallocate said unused resources to a pending workload as taught by Dimitrov et al. . ([0062]: “The resource manager 144 of the illustrated example facilitates recovery of cloud computing resources of the cloud provider 110 that are no longer being activity utilized. Automated reclamation may include identification, verification and/or reclamation of unused, underutilized, etc. resources to improve the efficiency of the running cloud infrastructure.”). Moreover, it may also be the case that a workload exceeds a threshold workload utilization and was terminated as a result of an enforcement action as taught by CHEN et al. above. Subsequently, it would have been obvious to one of ordinary skill in the art to use this as a method to release resources of a preempted workload as taught by Li et al. (Col. 6, lines 46-59: “As described above, there are two ways for a workload scheduling system to release resources of a preempted workload: by terminating and re-queuing or re-running the workload, or by stopping the workload and resuming it later. Terminating a workload means the system will completely kill the process of the workload. Before killing the process, the system may save the state of the process so that when the process is later re-scheduled to run, it can start from the last saved point. The killed workload may be placed back in a scheduling queue or be moved to another host, and may be re-run from the very beginning or from the last saved point. This first way of resource releasing is suitable for a workload having no dependency on the running environment or just running for a short time.”).
The power status choice would be included to allow a user to power on/off resources during preemption as taught by Dimitrov et al. ([0099]: “In certain examples, the user device 614 seeks to manage (e.g., enumerate/discover, provision/destroy, power on/off, etc.) resources such as VMs, containers, etc.,”).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the invention of Li et al. modified by Dimitrov et al. and CHEN et al. to identify a preemptible workload based on the second preemption policy to avoid preempting old workloads or to release unused or underutilized resources for the pending workload as previously mentioned above.
18. Regarding claim 16, it is a media/product type claim with similar limitations as claim 3 above. Therefore, it is rejected under the same rationale.
19. Regarding claim 20, it is a media/product type claim with similar limitations as claim 3 above. Therefore, it is rejected under the same rationale.
20. Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Li et al. modified by Dimitrov et al., as applied to claim 6, in further view of Veeam (https://web.archive.org/web/20210926145845/https://vccbook.io/1.introduction/).
21. Regarding claim 7, Li et al. modified by Dimitrov et al. teaches the method of claim 6,
Li et al. teaches wherein the preempting further comprises creating, by the workload scheduler, a backup of the preemptible workload (Col. 6, lines 46-54: “As described above, there are two ways for a workload scheduling system to release resources of a preempted workload: by terminating and re-queuing or re-running the workload, or by stopping the workload and resuming it later... Before killing the process, the system may save the state of the process (Creating a backup of the preemptible workload) so that when the process is later re-scheduled to run, it can start from the last saved point.”).
However, Li et al. modified by Dimitrov et al. do not explicitly teach storing the backup in a backup repository.
But Veeam teaches using a backup repository to store backup jobs (1. Introduction: “In 2014, with the release of Veeam® Backup & Replication™ v8, Veeam introduced a new technology — named Veeam Cloud Connect — developed specifically for service providers to create and serve remote backup repositories… Service providers who are part of the Veeam Cloud & Service Provider (VCSP) program can use Veeam Cloud Connect to offer customers Backup as a Service (BaaS)… Every Veeam Backup & Replication v8, v9 or v9.5 customer can then consume these services from their service provider of choice to send backups off site or to replicate (v9.x and newer users only) virtual machines (VMs).”; Integration: “Customers do not have to learn new tools or processes to consume the resources exposed via Veeam Cloud Connect; they can simply configure backup copy jobs (toward a cloud repository) or replication jobs (toward a cloud host) as before. The ease of use and complete integration makes Veeam Cloud Connect an easy-to-onboard and easy-to-consume solution for service providers.”).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to further modify the invention of Li et al. modified by Dimitrov et al. with the teachings of Veeam to use a backup repository to store a backup of the preemptible workload by storing the saved state of the preemptible workload as taught by Li et al. above in the backup repository in Veeam. One would be motivated to store the backup in a backup repository for increased security of the workload as taught by Veeam (Security: “A service exposed via public internet connection and shared between multiple tenants cannot ignore security. Veeam Cloud Connect offers different levels of security: At source: By leveraging the encryption capability first introduced in Veeam Backup & Replication v8, data is immediately encrypted by Veeam components on the customer side… In flight: The connection between a tenant and the cloud gateway(s) is encrypted using SSL certificates (technically, it’s TLS 1.2). This way, no man-in-the-middle attack will happen unnoticed, and even unencrypted data can traverse the public internet securely… At rest: Backup files are stored in an encrypted format at the service provider using customer keys. There is no possibility for the service provider to read the content of a customer’s backups if the customer doesn’t share the passwords with the provider.”).
22. Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Li et al. modified by Dimitrov et al., as applied to claim 1, in further view of Moreira Martins et al. (US 20220100578 A1).
23. Regarding claim 10, Li et al. modified by Dimitrov et al. teaches the method of claim 1.
However, Li et al. modified by Dimitrov et al. do not explicitly teach further comprising updating, by the workload scheduler, a running workload repository to store a new entry indicating the pending workload as an already-deployed workload after the pending workload is deployed in the private cloud.
But Moreira Martins et al. teaches adding running workloads to a running workload repository to monitor runtime statistics ([0022]: “Resident 116 may monitor and track mainframe resources to report back to workload control module 114 on performance and usage statistics, as well as workloads running in mainframe platform 110. Optionally, resident 116 can also communicate with other residents in other logical partitions to consolidate data being sent over to workload control module 114.”; [0025]: “data repository 106 (Running workload repository) may include and save data from… application workload 104. Data repository 106 may include performance and cost data. For example, the performance and cost data may include usage, cost, latency, and throughput data. The performance and cost data may be process priority, processing index, data requirement, connectivity, system affinity, server health, management, and business data. For example, performance and cost data may be information including usage, the number of servers, response-time, cost, connectivity, and throughput of mainframe platform 110 and distributed computing platform 112. The performance and cost data may be collected from data system management and resource management data of mainframe platform 110... Workload control module 114 may update historical data associated with application workload 104 in data repository 106. Workload control module 114 may process and update data repository 106 to update decision-making information data.”; [0028]: “ Workload control module 114 may determine whether any historical data related to application workload 104 exists in data repository 106 (Data repository 106 stores data from running workloads as historical data)”).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to further modify the invention of Li et al. modified by Dimitrov et al. with the teachings of Moreira Martins et al. to include a running workload repository in the workload scheduling system of Li et al. to monitor runtime statistics to balance performance and cost in real time as taught by Moreira Martins et al. ([0031]: “workload control module 114 is configured to allocate the plurality of work units of application workload 104 to run on mainframe platform 110 and distributed computing platform 112 respectively to balance performance and cost in real time. For example, workload control module 114 may decompose the plurality of work units and can send some work units based on a cost model to be processed on distributed computing platform 112 to avoid exceeding the mainframe costs based on the consumption.”; [0029]: “the performance and cost data can be from data repository 106. The collected data can be further saved and updated in data repository 106. The performance and cost data may include usage, cost, latency, and throughput data. The performance and cost data may process priority, processing index, data requirement, connectivity, system affinity, server health, management, and business data.“).
For a similar reason, it would have also been obvious to one of ordinary skill in the art to apply the teachings of Moreira Martins et al. to the invention of Li et al. modified by Dimitrov et al. to update, by the workload scheduler, a running workload repository to store a new entry indicating the pending workload as an already-deployed workload after the pending workload is deployed in the private cloud by monitoring performance metrics of the deployed workload and storing said metric in a data repository such as the data repository 106 in the invention of Moreira Martins et al., where storing a new entry is considered adding an entry to store the performance metrics of a running workload in said data repository.
24. Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Li et al. in further view of Dimitrov et al., as applied to claim 1, in further view of Verma et al. (https://www.workloadautomation-community.com/blogs/connect-aws-batch-with-workload-automation).
25. Regarding claim 11, Li et al. modified by Dimitrov et al. teaches the method of claim 1,
Li et al. modified by Dimitrov et al. does not explicitly teach wherein the workload scheduler is accessible to a tenant of the private cloud via a private cloud management platform.
But Verma et al. teaches a workload scheduler that is accessible to a tenant of the private cloud via a private cloud management platform (Paragraph 1: “AWS Batch (This corresponds to the Workload Scheduler. It is accessible to a tenant of the private cloud via a private cloud management platform through the UI shown below.) enables developers, scientists, and engineers to run hundreds of thousands of batch computing jobs easily and efficiently on AWS. AWS Batch dynamically provisions the optimal quantity and type of compute resources (e.g., CPU or memory optimized instances) based on the volume and specific resource requirements of the batch jobs submitted. With AWS Batch, there is no need to install and manage batch computing software or server clusters that you use to run your jobs, allowing you to focus on analysing results and solving problems. AWS Batch plans, schedules, and executes your batch computing workloads across the full range of AWS compute services and features.”. See image below).
PNG
media_image3.png
704
1279
media_image3.png
Greyscale
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to further modify the invention of Li et al. modified by Dimitrov et al., with the teachings of Verma et al. to enable the workload scheduler accessible to a tenant of the private cloud via a private cloud management platform to provide a user with more control over their deployed workloads such as early termination or to determine the statuses of deployed workload as taught by Verma et al. above.
26. Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Li et al. modified by Dimitrov et al., as applied to claim 13, in further view of Medium (https://medium.com/geekculture/what-are-pods-in-kubernetes-861beb65e138).
27. Regarding claim 14, Li et al. modified by Dimitrov et al. teaches the workload scheduler of claim 13,
Dimitrov et al. teaches wherein the pending workload and the already-deployed workloads on the private cloud comprise one or more of virtual machines (Abstract: “The hosts are to manage requests and allocate resources through one or more virtual machines.”), containers ([0023]: “In certain examples, a VM can host a container and/or a container can be implemented for virtualization in place of the VM. Containers (e.g., Docker®, Rocket™ Linux® containers (LXC), etc.) can be used in computing environments to run applications, programs, utilities, and/or any other software in isolation. Containers can be used to achieve improved resource management (e.g., resources used by containerized components are isolated for use only by those components that are part of the same container) and/or for security purposes (e.g., restricting access to containerized files or components). In addition, containers can also be used to achieve lightweight, reproducible application deployment”), executable applications ([0095]: “Using the example platform 600, one or more external devices 614 can deploy and manage resources from clouds and/or hypervisors that already have accounts with the platform 600. Applications can be deployed (Executed) as a set of VMs with applicable software installed on them. For example, cloud providers can deploy applications as a set of VMs configured and installed with software via the multi-cloud management platform 600”), or combinations thereof ([0023]: “a VM can host a container and/or a container can be implemented for virtualization in place of the VM (Combination of virtual machines and containers)”).
However, Li et al. modified by Dimitrov et al. do not explicitly disclose the private cloud comprises pods.
However, Medium teaches using pods for running containers inside a Kubernetes cluster (What are Pods?: “Pods are Kubernetes Objects that are the basic unit for running our containers inside our Kubernetes cluster. In fact, Pods are the smallest object of the Kubernetes Model. Kubernetes uses pods to run an instance of our application and a single pod represents a single instance of that application. We can scale out our application horizontally by adding more Pod replicas.“).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to further modify the invention of Li et al. modified by Dimitrov et al. with the teachings of Medium, to include pods in the pending workload and the already-deployed workloads on the private cloud to run workloads in the containers of Li et al. modified by Dimitrov et al., as taught by Medium (What are Pods?: “Pods are Kubernetes Objects that are the basic unit for running our containers inside our Kubernetes cluster. In fact, Pods are the smallest object of the Kubernetes Model. Kubernetes uses pods to run an instance of our application and a single pod represents a single instance of that application.”).
28. Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over Li et al. in further view of Dimitrov et al., as applied to claim 12, in further view of Kiah (https://fastspring.com/blog/subscription-vs-pay-per-use-saas/).
29. Regarding claim 17, Li et al. modified by Dimitrov et al. teaches the workload scheduler of claim 12.
However Li et al. modified by Dimitrov et al. does not explicitly teach wherein the workload scheduler is subscribed for use by a tenant of the private cloud on a pay-per-use basis.
But Kiah teaches using a pay-per-use basis form of billing customers (What Is Pay-Per-Use Billing?: “With pay-per-use billing the customer makes a single purchase at a fixed price, and may or may not conduct future business with your company. Many SaaS startups launch with a fixed charge pay-per-use model, because of its simplicity. It has a straightforward revenue formula and customers like the predictability of a one-time charge.”).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to further modify the invention of Li et al. modified by Dimitrov et al., with the teachings of Kiah such that the workload scheduler is subscribed for use by a tenant of the private cloud on a pay-per-use basis to improve affordability and flexibility of billing as taught by Kiah (Pay-Per-Use: When and Why?: “Usage-based pricing models, like pay-per-use, certainly have their place. Many customers appreciate the affordability and flexibility pay-per-use provides, particularly when they don’t have a need to use the software more than a few times per month. A service that’s got a relatively limited lifetime should be available to customers in a format that allows them to pay for the service only when they need to use it.“).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to EDWARD J LI whose telephone number is (571)272-7695. The examiner can normally be reached Monday-Friday 9:00-5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kevin Young can be reached on (571) 270-3180. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/EDWARD JIANDE LI/Examiner, Art Unit 2194
/KEVIN L YOUNG/Supervisory Patent Examiner, Art Unit 2194
Prosecution Insights