DETAILED ACTION
This communication is in response to the application filed on November 25, 2024, in which claims 1-20 are pending in the application. Claims 1, 19, and 20 are in independent form.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statements (IDS) submitted on April 4, 2025, July 24, 2025, and August 7, 2026 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.
Drawings
The drawings are objected to because FIG. 5 inaccurately depicts step 523. The box for step 523 recites "Release resources occupied by a first quantity of the allocated service requests in the migrating node", while [0106] recites "Step 523: Release resources occupied by the M allocated service requests in the migrating node". The figure should recite "the M allocated service requests" to agree with the description and with claim 1.
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Specification
The disclosure is objected to because of the following informalities:
[0108] recites "Referring to FIG. 1 and FIG. 6, the apparatus can be applied to a scheduler." FIG. 1 is a schematic diagram illustrating resource allocation and shows neither a scheduler nor the apparatus. The scheduler is shown in FIG. 2. The passage should recite "Referring to FIG. 2 and FIG. 6."
Appropriate correction is required.
Claim Objections
Claims 5, 10, 11, 14, 17, and 18 are objected to because of the following informalities:
Claims 5 and 14 recite "each M allocated service request". Claim 1 recites M as a positive integer, so "each M allocated service request" is not grammatical. The claims should recite "each of the M allocated service requests", which is the form the specification uses at [0089] and [0119].
Claim 10 recites "The computer-implemented method of claim 8, comprising" and then recites a step, with no colon after "comprising". The claim should recite "comprising:”.
Claim 11 recites "wherein, based on quantities of occupied resources, sorting, for each migratable node based on quantities of occupied resources, allocated service requests corresponding to the migratable node, comprises:". The phrase "based on quantities of occupied resources" is recited twice. The leading occurrence should be deleted so that the restated step matches the sorting step of claim 8.
Claims 17 and 18 recite "comprising:" and then recite "a resource comprises at least one of a hardware resource or a virtual resource" and "the node is a physical machine or a virtual machine". Neither recitation is a step of the method. The claims should recite "wherein" in place of "comprising”.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 2 recites the term “large-scale” which is a relative term which renders the claim indefinite. The term “large-scale” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. For examination purposes, "large-scale resources required by future service requests" is interpreted as any forecast quantity of resources for a service request that has not yet arrived, consistent with paragraphs
Claim 6 recites that "M satisfies a condition that a sum of a total quantity of resources occupied by the M allocated service requests and a quantity of current remaining resources in the node is not less than the quantity of reserved resources". Claim 6 recites "the node" earlier as the node under determination and claim 1 recites M only for the migrating node. Those are different nodes wherever more than one node is marked migratable, so it is unclear which of the two the condition is measured on. For examination purposes, "a quantity of current remaining resources in the node" is interpreted as the quantity in the migrating node, consistent with paragraphs [0086] and [0087] of the specification, paragraph [0087] stating that "M is actually equal to the value of Ni corresponding to the migrating node".
Claim 8 recites "wherein selecting one migrating node from migratable nodes when determining that there are two or more nodes in which quantities of current remaining resources are less than the quantity of reserved resources, comprises". Claim 1 recites that step with no condition on the number of nodes, so claim 8 restates a step that claim 1 does not recite. Claim 8 then recites that the step "comprises" only a sorting of allocated service requests, which is not an act of selecting a node, so it is also unclear whether a migrating node is selected at all and how the sorting selects it. For examination purposes, claim 8 is interpreted as requiring the selecting step of claim 1 to include the recited sorting, performed for each migratable node, consistent with paragraphs [0078] and [0079] of the specification. Claims 9-13, which are dependent on claim 8, are similarly rejected.
Claim 10 recites "a value of Ni". There is insufficient antecedent basis for this limitation in the claim. Claim 10 depends on claim 8, and Ni is recited only in claim 9, which claim 10 does not depend on. Amending claim 10 to depend on claim 9 would resolve the ambiguity. For examination purposes, "a value of Ni of each migratable node" is interpreted as the value recited in claim 9, consistent with paragraphs [0080] and [0083] of the specification.
Claim 18 recites "the node is a physical machine or a virtual machine". Claim 1 recites "one node", "the node" for each determined node, "a migratable node", "one migrating node", and "at least one other node", and it is unclear which of them "the node" in claim 18 refers to. For examination purposes, "the node" is interpreted as each node recited in claim 1, consistent with paragraph [0043] of the specification.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 3, 8-13, and 16-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lang et al. (US 9,350,800 B2) (hereinafter Lang) in view of Han (CN 111722908 A) (hereinafter Han).
The examiner notes that Han is in Chinese and that all citations to Han are to the English translation of record.
As per claim 1, Lang primarily teaches the invention as claimed including:
A computer-implemented method for resource use (col. 3:27-30, an offline software tool and method to create Service Level Objective (SLO) capacity in a Database as a Service (DaaS) cluster environment; col. 10:4-10, the methods may be practiced by a computer system including one or more processors), comprising:
determining a quantity of reserved resources that need to be reserved in one node (col. 8:47-50, the method 400 includes determining an amount of server resources needed for an additional deployment reservation request for a new deployment or increasing reservation of resources of an existing deployment (act 402); col. 8:59-61, the method 400 further includes determining a server that currently does not have capacity to service the additional deployment reservation request (act 404); The examiner notes that act 402 determines the amount of resources a deployment reservation requires, and that Lang measures that amount against the capacity of a single server rather than against the capacity of the cluster, so the amount determined is one that needs to be reserved in one node);
determining nodes in which quantities of current remaining resources are less than the quantity of reserved resources (col. 8:59-64, the method 400 further includes determining a server that currently does not have capacity to service the additional deployment reservation request (act 404). As illustrated, above, this may be accomplished by finding a server for which replicas could be moved to free up space for the additional deployment request; The examiner notes that a server which currently does not have capacity to service the request is a server whose remaining resources are less than the resources the request requires, which is the comparison the claim recites);
determining, for each determined node …, whether the node is capable of satisfying the quantity of reserved resources, and if yes, marking the node as a migratable node (col. 8:65-col. 9:3, the method 400 further includes determining how resources on the server can be freed up by moving other replicas of other deployments on the server to other servers to allow the server to service the additional deployment reservation request (act 406); col. 5:9-11, one or more replica sets are selected from the selected server (i.e. s1) such that moving the replica-set out would create sufficient space to host the SLO; col. 6:60-62, the execution will return back to this source server selection level and it will explore the space for the remaining five candidates; col. 8:62-64, finding a server for which replicas could be moved to free up space for the additional deployment request; The examiner notes that Lang decides whether a server can host the request by how much space would be freed from moving a replica-set off it which is the determination of whether the node is capable of satisfying the quantity of reserved resources. Lang returns to the source server selection level and works through the remaining candidates, so the determination is made for each such server, and then identifies the server that passes as one for which replicas could be moved, which is the marking of the node as a migratable node);
selecting one migrating node from migratable nodes (col. 4:61-65, all the source server candidates (i.e. s1, s2, and s3 for this example) are sorted in descending order of free SLO capacity. Thus, the servers are sorted in the order shown, namely s1, s2 to s3. The server with the most capacity is selected. Thus, in this example, s1 is selected);
migrating, to at least one other node, M allocated service requests corresponding to the migrating node (col. 8:66-67, moving other replicas of other deployments on the server to other servers; col. 5:12-14, this may be accomplished by moving any one of replicas 206, 208 or 210, such that there would be four available cores for the new deployment 204; col. 6:67-col. 7:2, two out of three "medium" replicas should be moved out to create "xlarge" space on server 304-1; The examiner notes that Lang moves the replicas off the server and onto other servers. In one instance Lang moves a single replica off the server, any one of replicas 206, 208 or 210, and four cores then become available for the new deployment 204. In another instance Lang moves two of the three medium replicas off server 304-1, and xlarge space is then created there. The number of replicas moved is the M of the claim); and
releasing resources occupied by the M allocated service requests in the migrating node, wherein M satisfies a condition that after the resources occupied by the M allocated service requests are released, a quantity of remaining resources in the migrating node is not less than the quantity of reserved resources, wherein M is a positive integer not less than 1 (col. 9:3-5, for example, FIG. 1 illustrates that moving any one of replicas 206, 208 or 210 would cause four cores to be available for the new deployment 204; col. 6:65-col. 7:2, a replica-set is selected from the server 304-1 such that moving the replica-set out would create sufficient space to host the SLO ("xlarge" in the present example.) Two out of three "medium" replicas should be moved out to create "xlarge" space on server 304-1; col. 4:50-54, a new deployment reservation 204 needing four cores (e.g. an xlarge single replica deployment) is to be deployed to a cluster 202 having three servers, s1, s2 and s3, each having total capacities of 8 cores; col. 4:54-59, s1 has three single core (medium) replicas and a double core (large) replica, leaving three cores available for new or expanded deployments. The server s2 has a double core replica and a quadruple core (xlarge) reservation, leaving two cores available for further reservations. The third server s3 has all 8 cores reserved by an eight core reservation; The examiner notes that Lang selects the replicas to move off a server by whether enough space to host the request would be created by moving them out, which is the condition the claim places on M. In Lang's example a new deployment needs four cores on a cluster of three servers that hold eight cores each. Server s1 holds three cores free, and its other five are taken by three single core replicas and one double core replica. Moving one of those single core replicas off s1 gives up the one core it held, and s1 then has the four cores the new deployment requires. The quantity of resources remaining in the node after the release is therefore not less than the quantity of reserved resources).
Lang discloses determining, for each determined node based on how much space would be freed from moving replicas off it (col. 5:9-11, one or more replica sets are selected from the selected server (i.e. s1) such that moving the replica-set out would create sufficient space to host the SLO).
The quantity Lang arrives at is the space that moving replicas off the server would free up, which act 406 determines for that server (col. 8:65-col. 9:3, the method 400 further includes determining how resources on the server can be freed up by moving other replicas of other deployments on the server to other servers to allow the server to service the additional deployment reservation request (act 406)).
Determining how much space would be freed from moving replicas off it is not determining, for each determined node based on the calculated total quantity of resources.
Lang therefore does not explicitly teach:
determining, for each determined node based on the calculated total quantity of resources
In addition, Lang does not explicitly teach:
calculating, for each determined node and as a calculated total quantity of resources, a total quantity of resources occupied by allocated service requests in the node;
However, Han teaches:
calculating, for each determined node and as a calculated total quantity of resources, a total quantity of resources occupied by allocated service requests in the node ([0052], acquiring resources occupied by each created virtual machine on the node; [0054], the sorted virtual machines are combined according to preset rules to obtain multiple virtual machine combinations, and it is determined in turn whether the remaining resources after migration of each virtual machine combination meet the requested resources; [0055], the virtual machines running on each NUMA node whose resources are less than the requested resources are sorted from low to high; [0056], if the numbers of the virtual machines sorted from low to high are 1, 2, and 3, the resulting combination of virtual machines can be 1, 2, 3, 1 and 2, 1 and 3, 1, 2 and 3, 2 and 3; The examiner notes that Han obtains the resources that each workload on a node occupies and then forms combinations of those workloads. At [0056] the combination holding every such workload is virtual machines 1, 2 and 3 together, and that combination is the total quantity of resources occupied by the workloads on that node);
determining, for each determined node based on the calculated total quantity of resources ([0054], the sorted virtual machines are combined according to preset rules to obtain multiple virtual machine combinations, and it is determined in turn whether the remaining resources after migration of each virtual machine combination meet the requested resources; The examiner notes that Han determines in turn whether the resources left on a node meet the requested resources after a virtual machine combination is migrated out. In Lang as modified by Han, Lang determines whether a server can be freed enough for the request based on the total Han calculates for that server).
Lang and Han are both concerned with placing a request that exceeds the free resources of every node by migrating the workloads already running on a node so that the node can accommodate it and are therefore combinable.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Lang to include a total of the resources occupied by the workloads on each server that cannot currently serve the request, and using that total to decide whether that server can be freed enough to take it, as taught by Han, in order to determine whether the requested resources can be assembled on a single server ([0050]).
Motivation would shorten the search for a server that can be freed because the resources held by each workload are counted before any move is evaluated, and a server is passed over when its workloads do not hold enough to cover the request, as taught by Han ([0035] and [0050]).
As per claim 3, Lang in view of Han discloses the claimed invention as detailed above for claim 1, and further teaches wherein determining a quantity of reserved resources that need to be reserved in one node, comprises:
determining, based on sizes of resources required by a currently received service request, the quantity of reserved resources that need to be reserved in one node. (Lang col. 4:9-12, an end user specifies the SLO requirement. For example, the user may specify the SLO size and how many replicas of each deployment are needed; col. 8:47-50, determining an amount of server resources needed for an additional deployment reservation request for a new deployment or increasing reservation of resources of an existing deployment (act 402); The examiner notes that the SLO size specified by the user for the deployment is the size of resources the request requires, and that act 402 determines the amount to reserve from it)
As per claim 8, Lang in view of Han discloses the claimed invention as detailed above for claim 1, and further teaches wherein selecting one migrating node from migratable nodes when determining that there are two or more nodes in which quantities of current remaining resources are less than the quantity of reserved resources, comprises:
sorting, for each migratable node based on quantities of occupied resources, allocated service requests corresponding to the migratable node. (Han [0053], sorting the multiple virtual machines that occupy less resources than the requested resources in the created multiple virtual machines from low to high; [0055], the virtual machines running on each NUMA node whose resources are less than the requested resources are sorted from low to high; [0050], there may be a situation where no NUMA node can satisfy the resources requested by VMn after traversing all the remaining resources of NUMA nodes; The examiner notes that Han addresses the case in which no node has enough remaining resources. That is the case of two or more nodes whose current remaining resources are less than the quantity of reserved resources. Han sorts the workloads on each such node by the quantity of resources they occupy)
As per claim 9, Lang in view of Han discloses the claimed invention as detailed above for claims 1 and 8, and further teaches selecting top Ni allocated service requests for an ith migratable node, wherein Ni is an integer not less than 1, and Ni satisfies a condition that after resources occupied by the top Ni allocated service requests are released, a quantity of remaining resources in the ith migratable node is not less than the quantity of reserved resources (Han [0056], the preset rule may first combine the virtual machine with the lowest resource consumption with other virtual machines in sequence; [0056], if the numbers of the virtual machines sorted from low to high are 1, 2, and 3, the resulting combination of virtual machines can be 1, 2, 3, 1 and 2, 1 and 3, 1, 2 and 3, 2 and 3; [0054], it is determined in turn whether the remaining resources after migration of each virtual machine combination meet the requested resources; The examiner notes that Han builds its combinations from the lowest consumer upward. The combinations 1, then 1 and 2, then 1, 2 and 3 are therefore the top Ni workloads of the sorted list for successive values of Ni. Each combination is kept only if the resources left on the node meet the request after it is migrated out).
As per claim 10, Lang in view of Han discloses the claimed invention as detailed above for claims 1 and 8, and further teaches selecting one migratable node as the migrating node based on a value of Ni of each migratable node (Lang col. 9:37-39, alternative embodiments may rank servers by lowest cost to move replicas rather than most free capacity, or some other ranking; col. 9:51-54, cost may be determined by sizes of replicas (e.g. size of data or cores or some other resource calculation). Alternatively or additionally, cost may be determined by least number of replicas to move; The examiner notes that Lang ranks the source server candidates on a cost carried by each server and then takes the server standing at the top of that ranking, and that cost may be the least number of replicas that must be moved off the server. The examiner further notes that in the combination that least number is the count Han reaches for the node, which is the value of Ni detailed above for claim 9, so ranking on it and taking the top server selects the migrating node on the value of Ni of each migratable node).
As per claim 11, Lang in view of Han discloses the claimed invention as detailed above for claims 1, 8, and 10, and further teaches wherein, based on quantities of occupied resources, sorting, for each migratable node based on quantities of occupied resources, allocated service requests corresponding to the migratable node, comprises:
sorting the allocated service requests corresponding to the migratable node in ascending order of the quantities of occupied resources (Han [0053], sorting the multiple virtual machines that occupy less resources than the requested resources in the created multiple virtual machines from low to high; [0055], sorted from low to high).
As per claim 12, Lang in view of Han discloses the claimed invention as detailed above for claims 1, 8, and 10, and further teaches wherein selecting one migratable node as the migrating node based on a value of Ni of each migratable node, comprises:
sorting the migratable nodes in ascending order of values of Ni (Lang col. 9:38, rank servers by lowest cost to move replicas; col. 9:53-54, cost may be determined by least number of replicas to move; The examiner notes that ranking the servers upward from the lowest number of replicas that must be moved is a sort of those nodes in ascending order of the value of Ni).
As per claim 13, Lang in view of Han discloses the claimed invention as detailed above for claims 1, 8, 10, and 12, and further teaches selecting a current top migratable node as the migrating node (Lang col. 4:64-65, the server with the most capacity is selected. Thus, in this example, s1 is selected; col. 8:4-5, server 304-6 is selected first for the first branch as it is the least loaded server).
As per claim 16, Lang in view of Han discloses the claimed invention as detailed above for claim 1, and further teaches wherein the computer-implemented method is applied to cluster resource scheduling and each node is a node in a cluster (Lang col. 4:52-53, is to be deployed to a cluster 202 having three servers, s1, s2 and s3; col. 3:28-30, create Service Level Objective (SLO) capacity in a Database as a Service (DaaS) cluster environment).
As per claim 17, Lang in view of Han discloses the claimed invention as detailed above for claim 1, and further teaches a resource comprises at least one of a hardware resource or a virtual resource (Lang col. 1:47-49, customers will be able to subscribe to specific Service-Level-Objectives (SLOs) that provide exclusive reservations on resources like CPU cores and worker threads).
As per claim 18, Lang in view of Han discloses the claimed invention as detailed above for claim 1, and further teaches the node is a physical machine or a virtual machine (Lang col. 4:53-54, three servers, s1, s2 and s3, each having total capacities of 8 cores; col. 1:61-62, an xxlarge SLO customer occupies an entire server (continuing the running example above); The examiner notes that each of Lang's servers is a physical machine, being a server holding eight cores that a single customer can occupy in its entirety).
As per claim 19, it has similar limitations as claim 1 and is therefore rejected using the same rationale. Lang further teaches A non-transitory, computer-readable medium storing one or more instructions executable by a computer system to perform one or more operations for resource use, comprising: (col. 10:4-10, the methods may be practiced by a computer system including one or more processors and computer readable media such as computer memory. In particular, the computer memory may store computer executable instructions that when executed by one or more processors cause various functions to be performed, such as the acts recited in the embodiments)
As per claim 20, it has similar limitations as claim 1 and is therefore rejected using the same rationale. Lang further teaches A computer-implemented system for resource use, comprising: one or more computers (col. 10:4-10, the methods may be practiced by a computer system including one or more processors); and
one or more computer memory devices interoperably coupled with the one or more computers and having tangible, non-transitory, machine-readable media storing one or more instructions that, when executed by the one or more computers, perform one or more operations, comprising: (col. 10:4-10, the computer memory may store computer executable instructions that when executed by one or more processors cause various functions to be performed, such as the acts recited in the embodiments)
Claim(s) 2 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lang in view of Han in view of Zhou et al. (US 2021/0389894 A1) (hereinafter Zhou).
As per claim 2, Lang in view of Han discloses the claimed invention as detailed above for claim 1 but does not explicitly teach wherein determining a quantity of reserved resources that need to be reserved in one node, comprises:
estimating, as a requirement, large-scale resources required by future service requests;
determining, based on the requirement, the quantity of reserved resources that need to be reserved in one node; and
correspondingly, after the releasing resources occupied by the M allocated service requests in the migrating node: adding current remaining resources of the migrating node to a resource cache pool; and
when a service request is received, allocating remaining resources in one node from the resource cache pool for the service request.
However, Zhou teaches:
estimating, as a requirement, large-scale resources required by future service requests ([0014], by determining expansion failure metrics that include an expansion failure prediction for a set of deployments on a node cluster, the cluster defragmentation management system can determine whether expansion failures are expected to happen even where no expansion failures (or a very limited number of expansion failures) have recently taken place on the node cluster; [0077], the failure prediction may include an indication (e.g., a value or category) associated with an estimated likelihood that the node cluster will have one or more expansion failures for an existing set of deployments over a predetermined period of time (e.g., within 1-2 days, within a week); The examiner notes that an expansion failure is the failure of a request for further resources by a deployment already on the cluster, so Zhou's prediction of the expansion failures expected over the coming period is an estimate of the resources that future requests will call for);
determining, based on the requirement, the quantity of reserved resources that need to be reserved in one node ([0044], the resource management system 110 may mandate or have a setting that mandates a minimum number of empty nodes 224 on the node cluster 216 to ensure that the node cluster 216 be capable of supporting expansions; [0053], the defragmentation engine 218 continues performing defragmentation actions 310 until a target number of healthy nodes are available on the node cluster; The examiner notes that Zhou requires a number of the cluster's nodes to be held with no virtual machine deployed on them, and that the number is set from the expansions Zhou has predicted. A node held that way has all of its resources kept free for a deployment that has not yet been placed, so the quantity of resources reserved in that one node is the whole of that node's capacity); and
correspondingly, after the releasing resources occupied by the M allocated service requests in the migrating node: adding current remaining resources of the migrating node to a resource cache pool ([0053], the defragmentation engine 218 can live-migrate virtual machines to consolidate workloads on fragmented nodes to increase a number of empty nodes on the node cluster; [0044], the node cluster 216 may also include empty nodes 224 having no virtual machines deployed thereon; The examiner notes that a node whose workloads have been migrated off it joins the set of empty nodes the cluster holds, and that this set of empty nodes is the pool from which the cluster later draws); and
when a service request is received, allocating remaining resources in one node from the resource cache pool for the service request ([0045], where a fragmented node includes fewer empty cores than is needed to host a virtual machine, the virtual machine would need to be deployed to a different node, such as an empty node or another fragmented node having enough empty cores; [0044], the empty nodes 224 may be used as a target destination for any virtual machine on the node cluster 216; The examiner notes that a workload too large for any fragmented node is placed on one of the empty nodes that the defragmentation has produced).
Lang, Han, and Zhou are all concerned with gathering a node's scattered free resources into usable capacity by migrating the workloads already running on it and are therefore combinable.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Lang in view of Han to include a prediction of the expansions the cluster's existing deployments will call for over the coming period, and a target number of its nodes to be emptied and held against that prediction, as taught by Zhou, in order to keep the cluster capable of supporting the expansions it is asked for ([0044]).
Motivation would increase the effective capacity of the cluster because the workloads on its fragmented nodes are consolidated until whole nodes stand empty, and a deployment that no fragmented node can hold is then placed on one of them, as taught by Zhou ([0016] and [0053]).
Claim(s) 4, 5, 14, and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lang in view of Han in view of Chen et al. (US 9,513,962 B2) (hereinafter Chen).
As per claim 4, Lang in view of Han discloses the claimed invention as detailed above for claims 1 and 3 but does not explicitly teach:
correspondingly, after releasing resources occupied by the M allocated service requests in the migrating node:
allocating current remaining resources in the migrating node for the currently received service request.
However, Chen teaches:
correspondingly, after releasing resources occupied by the M allocated service requests in the migrating node: allocating current remaining resources in the migrating node for the currently received service request (col. 11:13-18, when a decision is made by a grid scheduler 1001 to preempt a lower priority workload (e.g., Job 1) to make way for a higher priority workload (e.g., Job 3), a usual preemption mechanism is used to reserve Job 1's resources for Job 3; col. 11:46-51, once the migration is complete (indicated in FIG. 7 by arrow 701), Job 1's allocation is again resized (but this time decreased) to encompass only the allocation of the dummy job, releasing its initial allocation. At this point the higher priority job (i.e., Job 3) is dispatched (e.g., to Host 1 (or a multiple of hosts)); The examiner notes that Chen moves the lower priority job off its host and releases the allocation that job held there. Chen then dispatches the higher priority job onto that host. The resources given up by the moved job are the ones the pending job is placed on, which is the allocating of the node's remaining resources for the request).
Lang, Han, and Chen are all concerned with freeing the resources of a node for a request it cannot currently serve by moving the workloads already running on that node and are therefore combinable.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Lang in view of Han to include an allocation of the migrating server's remaining resources to the currently received request, as taught by Chen, in order to reserve the freed resources for the request the move was made for (col. 11:13-18).
Motivation would improve the reliability of the placement because the moved workload keeps running on the server's resources until the move finishes, and the request is dispatched onto them only after that workload's allocation is released, as taught by Chen (col. 11:31-35 and col. 11:46-51).
As per claim 5, Lang in view of Han discloses the claimed invention as detailed above for claim 1 but does not explicitly teach:
wherein each M allocated service request is an allocated service request whose service attribute allows migration.
However, Chen teaches:
wherein each M allocated service request is an allocated service request whose service attribute allows migration. (col. 10:6-13, a preemptable workload uses resources that can be reassigned. The grid scheduler may implement a workload analyzer, which analyzes each computer program representing a grid workload or job and categorizes it as preemptable or non-preemptable. In step 802, it is determined whether the job to be preempted is eligible for live migration; col. 9:1-3, embodiments of the present invention presuppose that the workload in the grid can be live migrated, for example, by using virtual machines as the workload's container)
Lang, Han, and Chen are all concerned with freeing the resources of a node for a request it cannot currently serve by moving the workloads already running on that node and are therefore combinable.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Lang in view of Han to include an eligibility for live migration determined for each workload on a candidate server, as taught by Chen, in order to separate the workloads whose resources can be reassigned from those whose resources cannot (col. 10:6-13).
Motivation would shorten the time it takes to bring a node to the reserved quantity, because each workload that is chosen for release is one already found able to leave the node without stopping, as taught by Chen (col. 10:6-13).
As per claim 14, Lang in view of Han discloses the claimed invention as detailed above for claim 1, and further teaches wherein migrating, to at least one other node, M allocated service requests corresponding to the migrating node, comprises:
for each M allocated service request of the M allocated service requests: determining, based on a quantity of resources occupied by each M allocated service request, whether each M allocated service request is migratable to a target node other than the migrating node; and (Lang col. 5:23-26, for each replica to be moved, all the target server candidates are identified, observing upgrade domain and/or fault domain, and SLO capacity constraints; col. 5:38-40, the first target server in the ranked set, evaluated from least capacity to most capacity, with enough SLO capacity can be selected; The examiner notes that Lang tests each replica against the candidate servers other than the one it is being moved off, and that the test is whether the candidate has capacity enough to hold that replica)
Lang in view of Han does not explicitly teach:
if yes: reserving, in the target node, a quantity of resources required by each M allocated service request; and migrating each M allocated service request to the target node.
However, Chen teaches:
if yes: reserving, in the target node, a quantity of resources required by each M allocated service request; and migrating each M allocated service request to the target node (col. 10:34-38, a fake workload that has identical resource requirements as the low-priority workload is submitted to the grid scheduler 1001. The system then schedules this "dummy" workload normally in step 902 and its result is the desired location; col. 11:36-39, the grid scheduler 1001 holds the allocation of the idle slot in Host 2 (at the highest possible priority) so that no other jobs are scheduled to that slot in the interim while the move of Job 1 takes place; col. 8:61-67, if space is found to move a job, resources need to be allocated at both its source host and its destination host while the operation takes place in order to ensure that no other workload is scheduled at the source site (which is to be used by the pending higher priority job) or the target site (which is to be used by the existing lower priority job being moved, or migrated); col. 11:40-42, the grid scheduler 1001 then triggers (i.e., initiates) the migration (step 904 in FIG. 9) by requesting it of the live migration controller 1002; The examiner notes that Chen sizes a "dummy" workload to match the workload to be moved and places it on the destination. Chen then holds that allocation there, which reserves in the target node the quantity of resources the moved workload requires, and then starts the move).
Lang, Han, and Chen are all concerned with freeing the resources of a node for a request it cannot currently serve by moving the workloads already running on that node and are therefore combinable.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Lang in view of Han to include a reservation in the target server of the resources each moved workload requires, and a move of that workload onto them, as taught by Chen, in order to settle where each workload will land before it leaves its own server (col. 10:34-38).
Motivation would raise the number of workloads the cluster can move in one pass because the space chosen for each workload is held from the moment it is chosen, and no second workload is sent to space already promised to another, as taught by Chen (col. 8:61-67 and col. 11:36-39).
As per claim 15, Lang in view of Han in view of Chen discloses the claimed invention as detailed above for claims 1, 3, and 4, and further teaches wherein releasing resources occupied by the M allocated service requests in the migrating node, comprises: marking each released resource as a reserved resource for the currently received service request (Chen col. 11:13-18, when a decision is made by a grid scheduler 1001 to preempt a lower priority workload (e.g., Job 1) to make way for a higher priority workload (e.g., Job 3), a usual preemption mechanism is used to reserve Job 1's resources for Job 3; col. 11:31-35, the grid scheduler 1001 holds the allocation of the slot Job 1 is currently running on (the source), so that the grid scheduler 1001 does not detect these resources as released and allow the higher priority Job 3 to run prematurely; col. 8:62-66, resources need to be allocated at both its source host and its destination host while the operation takes place in order to ensure that no other workload is scheduled at the source site (which is to be used by the pending higher priority job); col. 11:46-51, once the migration is complete (indicated in FIG. 7 by arrow 701), Job 1's allocation is again resized (but this time decreased) to encompass only the allocation of the dummy job, releasing its initial allocation. At this point the higher priority job (i.e., Job 3) is dispatched (e.g., to Host 1 (or a multiple of hosts)); The examiner notes that Chen reserves the resources of the moved workload for the pending request as soon as the decision to move is taken, and setting them aside for that request is the marking the claim recites. Chen then holds that allocation on the source host so that no other workload is scheduled there while the move runs, and the reservation is therefore still on those resources at the point they are released. That makes each resource released on the source host a resource reserved for the currently received request, and Chen dispatches that request onto them once the move completes).
Claim(s) 6 and 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lang in view of Han in view of Li et al. (US 2020/0159587 A1) (hereinafter Li).
As per claim 6, Lang in view of Han discloses the claimed invention as detailed above for claim 1 but does not explicitly teach:
wherein: determining, based on the calculated total quantity of resources, whether the node is capable of satisfying the quantity of reserved resources comprises: determining whether a sum of the calculated total quantity of resources and a quantity of current remaining resources in the node is not less than the quantity of reserved resources; and
if yes, determining that the quantity of reserved resources is satisfied, wherein correspondingly, M satisfies a condition that a sum of a total quantity of resources occupied by the M allocated service requests and a quantity of current remaining resources in the node is not less than the quantity of reserved resources.
However, Li teaches:
wherein: determining, based on the calculated total quantity of resources, whether the node is capable of satisfying the quantity of reserved resources comprises: determining whether a sum of the calculated total quantity of resources and a quantity of current remaining resources in the node is not less than the quantity of reserved resources ([0063], Based on the required resources Rsvi of job i and the releasable resources Rlsj from each job xj; [0063], in some cases, host x may already have some idle resources thereon (denoted by Availx) that can be directly allocated to a job dispatched to it. Thus, the one or more currently pending jobs to be preempted by job i are determined according to the following equation; [0063], Equation 2, Availx < Rsvi ≤ Availx + Σk=1n Rlsxk; [0074], since the host does not have enough idle memory for job 4, job 4 has to take memory from the running jobs on the host; The examiner notes that Equation 2 puts the resources the pending job requires above the idle resources the host already holds and no higher than those idle resources added to the sum of what the selected running workloads can release. Li works that case through with job 4, where the host does not hold enough idle memory by itself and job 4 then takes the rest from the jobs already running there. The right side of Equation 2 is the node's current remaining resources added to the resources its workloads hold, and Equation 2 measures the resources the request needs against that sum, which is the comparison the claim recites); and
if yes, determining that the quantity of reserved resources is satisfied, wherein correspondingly, M satisfies a condition that a sum of a total quantity of resources occupied by the M allocated service requests and a quantity of current remaining resources in the node is not less than the quantity of reserved resources ([0063], Thus, the one or more currently pending jobs to be preempted by job i are determined according to the following equation; [0063], Equation 2, Availx < Rsvi ≤ Availx + Σk=1n Rlsxk; [0076], a sum of releasable memory of job 1 (1.8 GB) plus releasable memory of job 2 (1.9 GB) plus idle memory (1 GB) can satisfy the required memory of job 4 (3 GB); [0058], the monitor 450 may know how many resources in total are reserved for a job, the percentage or quantity of locked resources in the total resources, and/or the percentage or quantity of releasable resources in the total resources; The examiner notes that Equation 2 fixes which running workloads are selected, because Li takes the set whose releasable resources added to the host's idle resources cover the requirement. In the worked case Li selects job 1 and job 2, and the memory those two release together with the host's idle memory covers what job 4 requires. Those two workloads are the M allocated service requests of the claim. Li sums the releasable portion of what each workload holds rather than the whole of it, and a set whose releasable portion meets the requirement is a set whose occupied resources meet it).
Lang, Han, and Li are all concerned with placing a pending request on a host whose free resources alone cannot hold it by recovering resources from the workloads already running there and are therefore combinable.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Lang in view of Han to include a sum of a server's free resources and the resources its workloads hold that is weighed against the quantity the request reserves, as taught by Li, in order to select which of the workloads running there are to be preempted for the request ([0054]).
Motivation would improve the utilization of the resources a server already holds idle because the server's idle resources go toward the request before anything is taken from the workloads running there, as taught by Li ([0055] and [0063]).
As per claim 7, Lang in view of Han discloses the claimed invention as detailed above for claim 1 but does not explicitly teach:
determining, based on the calculated total quantity of resources, whether the node is capable of satisfying the quantity of reserved resources comprises: determining whether the calculated total quantity of resources is not less than the quantity of reserved resources; and
if yes, determining that the quantity of reserved resources is satisfied, wherein correspondingly, M satisfies a condition that a total quantity of resources occupied by the M allocated service requests is not less than the quantity of reserved resources.
However, Li teaches:
wherein: determining, based on the calculated total quantity of resources, whether the node is capable of satisfying the quantity of reserved resources comprises: determining whether the calculated total quantity of resources is not less than the quantity of reserved resources ([0063], to be preempted by job i are selected from job x1, job x2, . . . , job xm such that a sum of the quantities of releasable resources from job x1, job x2, . . . , job xn can satisfy the required resources of job i; [0063], in some cases, host x may already have some idle resources thereon; [0059], at step 510, one or more currently running workloads are determined to be preempted by a pending workload, wherein releasable resources from the one or more currently running workloads meet demanded resource of the pending workload; [0066], in a scheduling system comprising a plurality of hosts, the scheduler will traverse each of the plurality of hosts until a host with one or more currently running workloads determined at step 510 is found; The examiner notes that Li states the selection criterion as a sum of releasable resources meeting the requirement and brings the host's idle resources in only as a further case, so at step 510 Li compares the resources of the running workloads alone against the pending request and takes no account of any resources the host already has free. That comparison is the determination of whether the node is capable of satisfying the quantity of reserved resources, and Li makes it from the calculated total on its own. Li passes over a host until it finds one whose running workloads satisfy the test); and
if yes, determining that the quantity of reserved resources is satisfied, wherein correspondingly, M satisfies a condition that a total quantity of resources occupied by the M allocated service requests is not less than the quantity of reserved resources ([0063], a sum of the quantities of releasable resources from job x1, job x2, . . . , job xn can satisfy the required resources of job i; [0058], the monitor 450 may know how many resources in total are reserved for a job, the percentage or quantity of locked resources in the total resources, and/or the percentage or quantity of releasable resources in the total resources; The examiner notes that the workloads Li selects are those whose releasable resources added together satisfy the pending request, which is the condition the claim places on M. Li sums the releasable portion of what each workload holds rather than the whole of it, and a set whose releasable portion meets the requirement is a set whose occupied resources meet it).
Lang, Han, and Li are all concerned with placing a pending request on a host whose free resources alone cannot hold it by recovering resources from the workloads already running there and are therefore combinable.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Lang in view of Han to include a test that compares the request against the resources that the workloads running on each server would give back, as taught by Li, in order to find a server that can be freed before any workload is moved ([0066]).
Motivation would increase the stability of the server the request is placed on because that server admits the request only when its workloads can actually give up enough to cover it, and none of them is driven to an out-of-resource failure when they do, as taught by Li ([0053] and [0055]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Cao et al. (US 2021/0200573 A1) discloses that the destination server pre-reserves a virtual machine resource for the first virtual machine ([0501]), which relates to the claimed reserving, in the target node, a quantity of resources required by each M allocated service request.
Liu (CN 109995871 A) discloses select a Pod on the sorted Node node, according to the amount of resources required to create the Pod, search for the migrated Node node that meets the second preset condition in the second Node node set (abstract), which relates to the claimed migrating, to at least one other node, M allocated service requests corresponding to the migrating node.
Wu (CN 114598665 A) discloses filtering other nodes except the at least one target node in the cluster to obtain a candidate node list meeting the service requirement of the service list (abstract), which relates to the claimed determining, based on the requirement, the quantity of reserved resources that need to be reserved in one node.
Gaurav et al. (US 2016/0378563 A1) discloses identifying the hosts on which VMs and containers can be consolidated based on resource availability (abstract), which relates to the claimed determining, for each determined node, whether the node is capable of satisfying the quantity of reserved resources, and if yes, marking the node as a migratable node.
Examiner has cited particular columns/paragraphs/sections and line numbers in the references applied and not relied upon to the claims above for the convenience of the applicant. Although the specified citations are representative of the teachings of the art and are applied to specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested from the applicant in preparing responses, to fully consider the references in entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the Examiner.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to THANH NGO whose telephone number is (571)270-3019. The examiner can normally be reached M-F 9am to 6pm ET, first F of bi-week off.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pierre Vital can be reached at (571)272-4215. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/T.N./ Examiner, Art Unit 2198
/PIERRE VITAL/ Supervisory Patent Examiner, Art Unit 2198
Prosecution Insights