DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 07/09/2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Objections
Claims 6 and 10-18 are objected to because of the following informalities:
Claim 6 recites “wherein the plurality of nodes comprises a plurality of processing nodes each comprising processing resource”. The Examiner suggests “each comprising one or more processing resources” for clarity.
Claim 10 recites “wherein the optimizing of the objective function is performed based least in part on...”. The Examiner suggests “performed based at least in part on...” for clarity.
Claim 12 recites “comprising value for the weight factor”. There is a missing article before “value”. The Examiner suggests adding “a” before “value”.
Claim 13 recites “performing action selection operation to select action from the plurality of possible actions”. There are missing articles before “action selection operation” and “action”. The Examiner suggests adding “an” before “action selection operation” and “an” before “action”.
Any claim not explicitly mentioned above is objected to due to dependency on an objected claim.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 9 and 18 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 9, the claim recites: “migrating one or some of the workload elements from one or more of the active processing nodes to another one or more of the active nodes to reduce the number of active processing nodes; and/or migrating one or some of the workload elements from one or more of the active memory nodes to another one or more of the active nodes to reduce the number of active memory nodes.” The use of “and/or” renders the claim indefinite because it introduces ambiguity as to the metes and bounds of the claim. It is unclear whether the claim requires only migrating one or some of the workload elements from one or more of the active processing nodes to another one or more of the active nodes to reduce the number of active processing nodes, only migrating one or some of the workload elements from one or more of the active memory nodes to another one or more of the active nodes to reduce the number of active memory nodes, or both migrating one or some of the workload elements from one or more of the active processing nodes to another one or more of the active nodes to reduce the number of active processing nodes and migrating one or some of the workload elements from one or more of the active memory nodes to another one or more of the active nodes to reduce the number of active memory nodes, and whether all of these combinations are encompassed or if there is an intended hierarchy. The use of “and/or” lacks clarity and may lead to multiple interpretations. For the purposes of examination, the Examiner interprets “and/or” as “or”.
Regarding claim 18, the pertinent portions of the claim recites: “wherein if the action application operation does not migrate a workload element from a current resource pool to another resource pool, then the learning operation further comprises, in each epoch: removing, from the list, all actions associated with a workload element to which action has been applied; removing, from the list, all actions associated with migrating workload elements belonging to the same workload as the workload element to one or more resource pools different from that of the workload element.”
The workload element in the wherein clause appears to refer to one to which action is not applied. The workload element in the first removing step refers to one in which action is affirmatively applied. It is unclear as to whether “the workload element” in the second removing step is referring to the workload element to which action is affirmatively applied or a workload element to which action is not applied, causing a person of ordinary skill in the art to be unable to ascertain the metes and bounds of the claim. For purposes of examination, the Examiner assumes the workload element in the second removing step refers to one in which action is affirmatively applied.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2 and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Zad Tootaghaj (US 20220414817 A1) hereafter Zad in view of Sen et al. (US 20150192980 A1) hereafter Sen, further in view of Sundararajan et al. (US 20160226789 A1) hereafter Sundararajan.
Regarding claim 1, Zad teaches:
optimizing an objective function for workload consolidation based at least in part on the obtained information, the objective function being established for optimizing operational cost and workload migration (Paragraph 29; “4) The weights the system administrator chooses for the objective functions ϵ.sub.1, ϵ.sub.2 (where ϵ.sub.1 represents the operational cost and ϵ.sub.2 represents the migration cost); 5) The required number of virtual GPUs R.sub.i ∀i; for each job; and 6) The total number N of physical GPUs in the system.”, explicitly disclosing an objective function for optimizing the migration cost, corresponding to workload migration, and operational cost. Paragraph 23 discloses “This allocation determination takes into consideration the existing job F 202 and the previously allocated L vGPUs. This may result in some GPUs being unused and powered off. This may result in some vGPUs being unused.”. Potentially resulting in some vGPUs being unused implies that workloads may be consolidated across resources, corresponding to the workload consolidation aspect of the claim.);
minimizing migration cost (Paragraph 22; “In a first example invocation of GPU scheduler 108, the GPU scheduler optimally allocates job F 202 to L different vGPUs from the set of vGPUs 116, 118, . . . 120 such that the migration cost and operational cost for computing system 102 are minimized”, explicitly disclosing minimizing the migration cost.);
consolidating the workloads (Paragraph 23; “This allocation determination takes into consideration the existing job F 202 and the previously allocated L vGPUs. This may result in some GPUs being unused and powered off. This may result in some vGPUs being unused.”.).
Zad does not teach obtaining information associated with workloads in the data center; energy efficiency; obtaining, based at least in part on optimizing the objective function, a plurality of Pareto optimal solutions, each Pareto optimal solution respectively representing an optimal energy efficiency; a data center; based at least in part on at least one of the plurality of Pareto optimal solutions.
However, Sen teaches:
obtaining information associated with workloads in the data center (Paragraph 54; “As shown in FIG. 2, this process involves repeated execution of three states 60 during execution by core 12 of the workload program 20, the states 60 including: a training state 62, a state selection state 64, and a configuration state 66. During the training state 62, the predictors 50 monitor their respective computational resources to evaluate the relative trade-offs in performance and power under the historical operating environment of executing a workload program 20.”, where historical data about workloads requires that information associated with workloads are obtained. Paragraph 3 explicitly considers data centers, “Electrical power consumption is a significant constraint in electronic computer design and use. These constraints result both from a need to conserve power (to save power costs for data centers and to prolong the operating life of battery-operated devices)”.);
obtaining, based at least in part on optimizing the output, a plurality of Pareto optimal solutions, each Pareto optimal solution respectively representing an optimal energy efficiency (Paragraph 54; “In the state selection state 64, Pareto optimal combination states (representing particular combinations of operating states of each computational resource) are determined by the coordinator 56 and output to a user or control program”, in which a plurality of Pareto optimal combination states, corresponding to solutions, are obtained. Paragraph 7 further discloses “In one embodiment only Pareto optimal settings (with respect to energy consumption and performance) are developed to simplify adjustment of the computer system during runtime.”, corresponding to the optimal energy efficiency aspect of the claim.);
based at least in part on at least one of the plurality of Pareto optimal solutions (Paragraph 54; “Pareto optimal combination states (representing particular combinations of operating states of each computational resource) are determined by the coordinator 56 and output to a user or control program, and in state 66 a desired state configuration selected from among the output Pareto optimal combination states is transmitted to the computational resources through the state signals 30a. 36a, 30b, 36b, 40, and 42.”).
Zad and Sen are considered to be analogous to the claimed invention because they are in the same field of rebalancing load in distributed systems. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zad to incorporate the teachings of Sen and obtain information associated with workloads in the data center, and obtain, based on optimizing the output, a plurality of Pareto optimal solutions representative of an optimal energy efficiency and perform actions based on at least one of the Pareto optimal solution. A person of ordinary skill in the art before the effective filing date of the claimed invention would have recognized that evaluating multiple Pareto optimal solutions based on workload characteristics is a known method for enabling the selection of resource allocations that appropriately balance competing objectives, yielding the predictable result of operating the data center more efficiently under varying workload conditions.
Zad in view of Sen does not teach minimizing a number of migrations; an objective function.
However, Sundararajan teaches:
minimizing a number of migrations (Paragraph 111; “based on the fitness function, the best candidate solution is the one that minimizes the standard deviation in CPU and memory residual capacity among the hosts (i.e., equally distributes load among the hosts) and minimizes the number of migrations needed to achieve the mapping in the candidate solution.”);
an objective function (Paragraph 62; “At block 146, a fitness function is applied to each candidate solution. The fitness function scores each candidate solution based on distribution of resource usage among the hosts for each candidate solution and the number of migrations (migration count) from the current mapping that each candidate solution would need to achieve the mappings in each candidate solution.”, the fitness function corresponding to a species within the genus of objective functions.).
Zad, Sen, and Sundararajan are considered to be analogous to the claimed invention because they are in the same field of rebalancing load in distributed systems. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zad in view of Sen to incorporate the teachings of Sundararajan and minimize a number of migrations, and utilize an objective function. A person of ordinary skill in the art would have recognized minimizing workload migrations as a recognized objective that reduces migration overhead, and would have further been motivated to include migration count as a term in the objective function such that the optimization simultaneously considers both migration cost and other known optimization metrics, yielding the predictable result of selecting workload placements that balance competing resource management objectives while improving overall system efficiency.
Regarding claim 20, Zad in view of Sen, further in view of Sundararajan teaches the method of claim 1. Zad teaches:
A system comprising:
one or more processors (Paragraph 38; “these steps may be performed by hardware components or may be embodied in machine-executable instructions, which may be used to cause a processor programmed with the instructions to perform the steps”);
and memory storing a computer program configured to be executed by the one or more processors, the computer program comprising instructions for performing or facilitating performing of the computer-implemented method of claim 1 (Paragraph 39; “Embodiments described herein may be provided as a computer program product, which may include a tangible machine-readable storage medium embodying thereon instructions, which may be used to program a computer (or other electronic devices) to perform a process.”, the storage medium corresponding to a memory. Zad, Sen, and Sundararajan teach the method of claim 1 as discussed above.).
Regarding claim 2, Zad in view of Sen, further in view of Sundararajan teach the method of claim 1. Sen teaches:
wherein the output optimizes the energy efficiency by maximizing the energy efficiency associated with the workloads (Paragraph 17; “It is thus a feature of at least one embodiment of the invention to provide a mechanism for setting multiple operating states of different computational resources to obtain a desired energy consumption/performance outcome by selecting the desired energy consumption/performance outcome.”, in which it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have identified a desired energy consumption/performance outcome that maximized energy efficiency associated with the workloads.).
Sundararajan teaches:
the objective function (Paragraph 62; “At block 146, a fitness function is applied to each candidate solution. The fitness function scores each candidate solution based on distribution of resource usage among the hosts for each candidate solution and the number of migrations (migration count) from the current mapping that each candidate solution would need to achieve the mappings in each candidate solution.”, the fitness function corresponding to a species within the genus of objective functions.);
and minimizing the number of migrations (Paragraph 111; “based on the fitness function, the best candidate solution is the one that minimizes the standard deviation in CPU and memory residual capacity among the hosts (i.e., equally distributes load among the hosts) and minimizes the number of migrations needed to achieve the mapping in the candidate solution.”).
Regarding claim 19, Zad in view of Sen, further in view of Sundararajan teaches the method of claim 1. Zad teaches:
wherein the consolidating of the workloads is based at least in part on the obtained information (Paragraph 21; “once GPU scheduler 108 formulates a solution to the problem of optimal GPU allocation into an integer linear programming optimization problem based on input variables, the GPU scheduler sends the formulation to solver 122. Solver 122 determines an optimal solution for the formulation and returns a set of output data (described below) to the GPU scheduler. The output data is used by the GPU scheduler to implement the optimal allocation of jobs to GPUs in computing system 100 (e.g., possibly migrating existing jobs and/or allocating new jobs).”. Paragraph 20 further discloses “This may result in migrating one or more existing jobs from one physical GPU to another physical GPU. In another embodiment, whenever an existing job is complete, GPU scheduler 108 determines a new optimal allocation of jobs to vGPUs, taking into consideration the requirements of the completed job and allocation of existing jobs to vGPUs. This may also result in migrating one or more jobs from one physical GPU to another physical GPU.”, which contemplates migration of multiple jobs from one GPU to another for processing, corresponding to workload consolidation. Paragraph 22 further discusses the utilization of distributed GPUs for processing.).
Sen teaches:
wherein the computer-implemented method further comprises selecting one of the plurality of Pareto optimal solutions (Paragraph 54; “During the training state 62, the predictors 50 monitor their respective computational resources to evaluate the relative trade-offs in performance and power under the historical operating environment of executing a workload program 20. In the state selection state 64, Pareto optimal combination states (representing particular combinations of operating states of each computational resource) are determined by the coordinator 56 and output to a user or control program, and in state 66 a desired state configuration selected from among the output Pareto optimal combination states is transmitted to the computational resources through the state signals 30a. 36a, 30b, 36b, 40, and 42.”);
and the Pareto optimal solution (Paragraph 54; “Pareto optimal combination states (representing particular combinations of operating states of each computational resource) are determined by the coordinator 56 and output to a user or control program”).
Claims 3 and 5-7 are rejected under 35 U.S.C. 103 as being unpatentable over Zad in view of Sen, further in view of Sundararajan, further in view of Yu et al. (US 20260161458 A1) hereafter Yu.
Regarding claim 3, Zad in view of Sen, further in view of Sundararajan teach the method of claim 2. Zad teaches:
wherein the data center comprises a plurality of nodes each respectively comprising one or more types of computing resource (Paragraphs 15-16; “Computing system 100 may include one or more servers, storage devices, communications networks, network fabrics, interconnects, network interface cards, switches, routers, etc. In an implementation, computing system 100 is situated in a data center and coupled to other computing systems.”, disclosing the different types of computing resources. Paragraph 15 discusses the nodes, “A “node” or “processing node” generally refers to a computing element. The nodes of a distributed system may be computer systems (e.g., clients, servers or peers) in virtual or physical form, one or more components of a computer system, computing elements, compute engines, hardware devices, software entities or processes, or a combination thereof.”).
Sen teaches:
the data center (Paragraph 3; “Electrical power consumption is a significant constraint in electronic computer design and use. These constraints result both from a need to conserve power (to save power costs for data centers and to prolong the operating life of battery-operated devices)”.).
Sundararajan teaches:
wherein the objective function is established to optimize the energy efficiency and the workload migration by minimizing the minimizing the number of migrations associated with the workloads (Paragraph 62; “At block 146, a fitness function is applied to each candidate solution. The fitness function scores each candidate solution based on distribution of resource usage among the hosts for each candidate solution and the number of migrations (migration count) from the current mapping that each candidate solution would need to achieve the mappings in each candidate solution.”, in which Paragraph 111 further discusses minimizing the number of migrations, “based on the fitness function, the best candidate solution is the one that minimizes the standard deviation in CPU and memory residual capacity among the hosts (i.e., equally distributes load among the hosts) and minimizes the number of migrations needed to achieve the mapping in the candidate solution.”);
Zad in view of Sen, further in view of Sundararajan does not teach the plurality of nodes comprises active nodes.
However, Yu teaches:
the plurality of nodes comprises active nodes (Paragraph 119; “target processing node can appropriately increase or decrease the number of processing units for running the script based on the number of requests received corresponding to the same script, so as to minimize the power consumption of the target processing node while ensuring data processing efficiency.”, the processing units corresponding to nodes, those set for running the node corresponding to active nodes.);
wherein the workloads are distributed in the active nodes (Paragraph 119; “target processing node can appropriately increase or decrease the number of processing units for running the script based on the number of requests received corresponding to the same script, so as to minimize the power consumption of the target processing node while ensuring data processing efficiency.”, where if there are multiple processing units running the same script simultaneously, the task is split across multiple processing units, thereby making the workload distributed among the active nodes.).
Zad, Sen, Sundararajan, and Yu are considered to be analogous to the claimed invention because they are in the same field of rebalancing node in distributed systems. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zad in view of Sen, further in view of Sundararajan to incorporate the teachings of Yu and have the plurality of nodes comprise active nodes in which workloads are distributed across. A person of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to utilize active nodes in the workload management system to ensure that placement and optimization decisions are based on nodes currently available to execute workloads, yielding the predictable result of improved resource utilization and operational efficiency. Further, a person of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to minimize the number of active nodes while satisfying workload requirements in order to achieve the recognized objective of improving energy efficiency, yielding the predictable result of reduced energy consumption through workload consolidation and reduced operation of unnecessary computing nodes.
Regarding claim 5, Zad in view of Sen, further in view of Sundararajan, further in view of Yu teach the method of claim 3. Sundararajan teaches:
wherein the number of migrations associated with the workloads is the number of migrations of the workloads (Paragraph 82; “The exemplary fitness function is calculated by adding the standard deviation of the hosts being considered and mig_count, which denotes the number of migrations needed to achieve the mapping of the candidate solution”.);
wherein consolidating the workloads comprises migrating one or some of the workloads (Paragraph 62; “At block 146, a fitness function is applied to each candidate solution. The fitness function scores each candidate solution based on distribution of resource usage among the hosts for each candidate solution and the number of migrations (migration count) from the current mapping that each candidate solution would need to achieve the mappings in each candidate solution.”).
Yu teaches:
active nodes (Paragraph 119; “target processing node can appropriately increase or decrease the number of processing units for running the script based on the number of requests received corresponding to the same script, so as to minimize the power consumption of the target processing node while ensuring data processing efficiency.”).
It would have been obvious to a person of ordinary skill in the art to have the optimization step reduce the number of active nodes in light of Yu explicitly disclosing the goal of minimizing power consumption and increasing/decreasing the number of processing units, alongside the fitness function of Sundararajan utilized to optimize the workload mapping solution, yielding the predictable result of optimized resource usage.
Regarding claim 6, Zad in view of Sen, further in view of Sundararajan, further in view of Yu teach the method of claim 3. Zad teaches:
wherein the data center has a server-based architecture, in which each of the plurality of nodes is respectively associated with a server having a plurality of types of computing resources (Paragraphs 15-16; “Computing system 100 may include one or more servers, storage devices, communications networks, network fabrics, interconnects, network interface cards, switches, routers, etc. In an implementation, computing system 100 is situated in a data center and coupled to other computing systems.”, explicitly disclosing consideration of a server architecture. Paragraph 15 further discloses association with a plurality of types of computing resources, “A “node” or “processing node” generally refers to a computing element. The nodes of a distributed system may be computer systems (e.g., clients, servers or peers) in virtual or physical form, one or more components of a computer system, computing elements, compute engines, hardware devices, software entities or processes, or a combination thereof.”);
wherein the plurality of nodes comprises a plurality of processing nodes each comprising processing resource (Paragraph 15; “A “node” or “processing node” generally refers to a computing element. The nodes of a distributed system may be computer systems (e.g., clients, servers or peers) in virtual or physical form, one or more components of a computer system, computing elements, compute engines, hardware devices, software entities or processes, or a combination thereof.”).
Yu teaches:
wherein the active nodes comprise active processing nodes (Paragraph 119; “target processing node can appropriately increase or decrease the number of processing units for running the script based on the number of requests received corresponding to the same script, so as to minimize the power consumption of the target processing node while ensuring data processing efficiency.”).
Regarding claim 7, Zad in view of Sen, further in view of Sundararajan, further in view of Yu teach the method of claim 3. Zad teaches:
wherein the migration cost associated with the workloads is the migration cost of the workload elements (Paragraph 22; “In a first example invocation of GPU scheduler 108, the GPU scheduler optimally allocates job F 202 to L different vGPUs from the set of vGPUs 116, 118, . . . 120 such that the migration cost and operational cost for computing system 102 are minimized”);
Sen teaches:
wherein the workloads comprise a plurality of distributed workload elements (Paragraph 54; “During the training state 62, the predictors 50 monitor their respective computational resources to evaluate the relative trade-offs in performance and power under the historical operating environment of executing a workload program 20”. Given one workload program and multiple computational resources, it would be obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have the workload elements of the one program distributed among the multiple computational resources.);
Sundararajan teaches:
number of migrations (Paragraph 111; “As described above, based on the fitness function, the best candidate solution is the one that minimizes the standard deviation in CPU and memory residual capacity among the hosts (i.e., equally distributes load among the hosts) and minimizes the number of migrations needed to achieve the mapping in the candidate solution”);
wherein consolidating the workloads comprises migrating one or some of the workload elements (Paragraph 62; “At block 146, a fitness function is applied to each candidate solution. The fitness function scores each candidate solution based on distribution of resource usage among the hosts for each candidate solution and the number of migrations (migration count) from the current mapping that each candidate solution would need to achieve the mappings in each candidate solution.”)
Yu teaches:
active nodes (Paragraph 119; “target processing node can appropriately increase or decrease the number of processing units for running the script based on the number of requests received corresponding to the same script, so as to minimize the power consumption of the target processing node while ensuring data processing efficiency.”);
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have the migrations be performed to reduce the number of active nodes. Because Yu teaches modifying the number of active nodes, Zad teaches optimizing resource allocation, and Sundararajan teaches minimizing the number of migrations, it would have been obvious to a person of ordinary skill in the art to have further optimized to reduce the number of active nodes in order to achieve the goals of Zad of optimizing resource allocation.
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Zad in view of Sen, further in view of Sundararajan, further in view of Yu, further in view of Chesla et al. (US 20080052774 A1) hereafter Chesla.
Regarding claim 4, Zad in view of Sen, further in view of Sundararajan, further in view of Yu teach the method of claim 3. Sundararajan teaches:
a summation associated with number of migrations associated with the workloads (Paragraph 82; “Equation 11 on line 226 indicates an exemplary fitness function that may be used in computing a rebalancing solution. For example, this fitness function may be used by the cloud rebalancing module 156 to determine a fitness score of candidate solutions at block 146. The exemplary fitness function is calculated by adding the standard deviation of the hosts being considered and mig_count, which denotes the number of migrations needed to achieve the mapping of the candidate solution. The candidate solution that minimizes this value is the solution that is chosen.”).
Yu teaches:
a number of active nodes (Paragraph 119; “target processing node can appropriately increase or decrease the number of processing units for running the script based on the number of requests received corresponding to the same script, so as to minimize the power consumption of the target processing node while ensuring data processing efficiency.”).
Zad in view of Sen, further in view of Sundararajan, further in view of Yu does not teach complementary weights with a factor in [0, 1].
However, Chesla teaches:
complementary weights with a factor in [0, 1] (Paragraph 243; “the current expected value is a linear combination of the mean value for the most recent hour and the last expected value, taken with complementary weights: Y.sub.n=.alpha.X.sub.n+(1-.alpha.)Y.sub.n-1, wherein Y.sub.n is the expected value after the nth iteration, X.sub.n is the last mean value of the same parameter, and .alpha. is a weighting constant between 0 and 1.”).
Zad, Sen, Sundararajan, Yu, and Chesla are considered to be analogous to the claimed invention because they are in the same field of digital data processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zad in view of Sen, further in view of Sundararajan, further in view of Yu to incorporate the teachings of Chesla and utilize complementary weights with a factor in [0, 1]. A person of ordinary skill in the art before the effective filing date of the claimed invention would have recognized that number of active nodes and number of workload migrations are recognized optimization objectives in workload placement and resource management, and would have been motivated to simultaneously optimize both metrics in order to balance both energy efficiency, achieved by reducing the number of active nodes, with migration overhead, achieved by reducing the number of workload migrations. It would have further been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have applied the complementary weighting factors of Chesla to the objective function because complementary weights provide a known mechanism for adjusting the relative importance of competing optimization objectives while maintaining a normalized weighting, yielding the predictable result of enabling the optimization to trade off energy efficiency against migration cost according to system requirements. It would have been obvious to apply different values of the weighting factor when optimizing the objective function because the purpose of the weighting factor is to adjust the relative importance of the competing objective terms. A person of ordinary skill in the art before the effective filing date of the claimed invention would have recognized that varying the weighting factor over its defined normalized range permits evaluation of different tradeoffs between optimization objectives, yielding the predictable result of selecting a weighting that best satisfies the desired system performance criteria.
Claims 8-9 are rejected under 35 U.S.C. 103 as being unpatentable over Zad in view of Sen, further in view of Sundararajan, further in view of Yu, further in view of Cannata et al. (US 20200344329 A1) hereafter Cannata.
Regarding claim 8, Zad in view of Sen, further in view of Sundararajan, further in view of Yu teach the method of claim 7. Zad teaches:
each of the plurality of nodes respectively comprises one or more types of computing resources (Paragraph 15; “A “node” or “processing node” generally refers to a computing element. The nodes of a distributed system may be computer systems (e.g., clients, servers or peers) in virtual or physical form, one or more components of a computer system, computing elements, compute engines, hardware devices, software entities or processes, or a combination thereof.”).
Zad in view of Sen, further in view of Sundararajan, further in view of Yu does not teach wherein the data center has a composable architecture; the plurality of nodes are arranged in a plurality of resource pools, the plurality of resource pools are isolated from each other.
However, Cannata teaches:
wherein the data center has a composable architecture (Paragraph 147; “deployments of compute units can be based on independent disaggregated pools, and deployments can enable disaggregation of converged servers. Control software packages can be deployed to converged servers to enable new disaggregation features. The software can be deployed on existing and new hardware deployments, enabling additional usage on server equipment of existing data centers, and converting existing converged servers to composable servers.”, in which an architecture employing disaggregated resource pools and composable serves corresponds to a composable architecture as the computing resources are organized to be independently composed and managed through software.);
the plurality of nodes are arranged in a plurality of resource pools, the plurality of resource pools are isolated from each other (Paragraph 98; “Several machines are shown for each cluster of a plurality of PCIe fabrics 620, with associated machines comprised of physical elements/resources 640 such as CPUs, FPGAs, GPUs, NICs, storage drives, memory devices and other PCIe devices, along with software/configuration data directed or deployed thereto. The clusters are electrically isolated within each fabric 620, and a management system can dynamically pull elements/resources from a pool of free elements, such as seen in FIG. 5.”, in which each cluster of resources constitutes a resource pool because it is a collection of computing resources managed for allocation by the management system. The disclosure explicitly discloses that these clusters are isolated from one another.).
Zad, Sen, Sundararajan, Yu, and Cannata are considered to be analogous to the claimed invention because they are in the same field of resource management. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zad in view of Sen, further in view of Sundararajan, further in view of Yu to incorporate the teachings of Cannata and have the data center have a composable architecture, and to have the plurality of nodes are arranged in a plurality of resource pools, the plurality of resource pools are isolated from each other. A person of ordinary skill in the art before the effective filing date of the claimed invention would have recognized that composable infrastructures are designed to organize computing resources into independent isolated pools that may be dynamically composed and allocated, and would have been motivated to organize the nodes into isolated resource pools to facilitate independent resource management, improve scalability, and enable flexible allocation of disaggregated resources, yielding the predictable result of improving efficiency and manageability of the resources.
Regarding claim 9, Zad in view of Sen, further in view of Sundararajan, further in view of Yu, further in view of Cannata teach the method of claim 8. Zad teaches:
processing nodes (Paragraph 15; “A “node” or “processing node” generally refers to a computing element. The nodes of a distributed system may be computer systems (e.g., clients, servers or peers) in virtual or physical form, one or more components of a computer system, computing elements, compute engines, hardware devices, software entities or processes, or a combination thereof.”).
Sundararajan teaches:
processing and memory resources (Paragraph 12; “the one or more host constraints comprise residual processor resources and residual memory resources”);
wherein consolidating the workloads comprises migrating one or some of the workloads (Paragraph 62; “At block 146, a fitness function is applied to each candidate solution. The fitness function scores each candidate solution based on distribution of resource usage among the hosts for each candidate solution and the number of migrations (migration count) from the current mapping that each candidate solution would need to achieve the mappings in each candidate solution.”).
Yu teaches:
active nodes (Paragraph 119; “target processing node can appropriately increase or decrease the number of processing units for running the script based on the number of requests received corresponding to the same script, so as to minimize the power consumption of the target processing node while ensuring data processing efficiency.”).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have additionally had memory nodes alongside the active processing nodes of Yu. Given Sundararajan considering both processor and memory resources, it would have been obvious to extend the processing nodes of Yu to also encompass memory nodes. Further, it would have been obvious to a person of ordinary skill in the art to have the optimization step reduce the number of active nodes in light of Yu explicitly disclosing the goal of minimizing power consumption and increasing/decreasing the number of processing units, alongside the fitness function of Sundararajan utilized to optimize the workload mapping solution, yielding the predictable result of optimized resource usage. Given the disclosure of Sundararajan teaching both processing and memory resources, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have performed the same processing node steps for memory nodes.
Claims 10-11 are rejected under 35 U.S.C. 103 as being unpatentable over Zad in view of Sen, further in view of Sundararajan, further in view of Yu, further in view of Chesla, further in view of Ritter et al. (US 20200034701 A1) hereafter Ritter.
Regarding claim 10, Zad in view of Sen, further in view of Sundararajan, further in view of Yu, further in view of Chesla teach the method of claim 4. Zad teaches:
the objective function (Paragraph 29; “4) The weights the system administrator chooses for the objective functions ϵ.sub.1, ϵ.sub.2 (where ϵ.sub.1 represents the operational cost and ϵ.sub.2 represents the migration cost); 5) The required number of virtual GPUs R.sub.i ∀i; for each job; and 6) The total number N of physical GPUs in the system.”).
Zad in view of Sen, further in view of Sundararajan, further in view of Yu, further in view of Chesla does not teach optimizing based at least in part on a reinforcement learning method.
However, Ritter teaches:
optimizing based at least in part on a reinforcement learning method (Paragraph 63; “In one embodiment, an action and a state space may be provided, using a special form of reinforcement learning, Q-learning, which may use value iteration to determine the optimal policy. The optimal policy can provide an optimized action value function, which in turn can provide an optimized provisioning result when the dynamic provisioning agent is following this policy”).
Zad, Sen, Sundararajan, Yu, Chesla, and Ritter are considered to be analogous to the claimed invention because they are in the same field of resource management. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zad in view of Sen, further in view of Sundararajan, further in view of Yu, further in view of Chesla to incorporate the teachings of Ritter and have optimized the objective function based at least in part on a reinforcement learning method. A person of ordinary skill in the art before the effective filing date of the claimed invention would have recognized reinforcement learning as a known optimization technique applicable on resource allocation decisions to maximize/minimize an objective function through interaction with the compute environment, and would have been motivated to apply reinforcement learning to the objective function in order to automatically learn resource allocation policies that optimize the objective while adapting to changing workload and system conditions, yielding the predictable result of improving optimization performance further compared to heuristic approaches.
Regarding claim 11, Zad in view of Sen, further in view of Sundararajan, further in view of Yu, further in view of Chesla, further in view of Ritter teach the method of claim 10. Ritter teaches:
wherein the reinforcement learning method is a Q-learning based reinforcement learning method (Paragraph 63; “In one embodiment, an action and a state space may be provided, using a special form of reinforcement learning, Q-learning, which may use value iteration to determine the optimal policy.”, explicitly disclosing Q-learning based reinforcement learning.).
Claims 12-13 and 15-18 are rejected under 35 U.S.C. 103 as being unpatentable over Zad in view of Sen, further in view of Sundararajan, further in view of Yu, further in view of Chesla, further in view of Ritter, further in view of Cannata.
Regarding claim 12, Zad in view of Sen, further in view of Sundararajan, further in view of Yu, further in view of Chesla, further in view of Ritter teach the method of claim 11.
Ritter teaches:
receiving an input comprising an action set (Paragraph 94; “The training environment may be initialized at 304. Initializing the training environment may include setting any environment parameters, such as the KPI formulas, KPI weights, set of possible states, set of possible actions, setting the number of cycles to run, or putting the machine-learning system (e.g. dynamic provisioning agent) into a training state. Initializing the training environment may also include obtaining any additional input data generally required but not provided by the input vectors (e.g. data center availability, job component/resource availability).”, where it would be obvious to have received the set of possible actions as an input parameter.);
epoch number (Paragraph 184; “training the algorithm includes providing a single set of training data inputs to the machine-learning algorithm, running the algorithm with the generated training data inputs, obtaining the results of the algorithm from processing the inputs, comparing the results to expected results (e.g. the generated expected results for the given training data set), and updating the algorithm based on the differences between the output and the expected results. This process may be repeated for all available generated training data sets as part of training the machine-learning algorithm. Training may continue until the differences between the output from the algorithm and the expected results are below a certain threshold, below a threshold for a given number of training cycles, or for a given number of training cycles or episodes.”, which teaches controlling training based on a number of training cycles. A person of ordinary skill in the art would have understood such repeated training cycles through available data sets to correspond to successive epochs of training because an epoch represents a repetition through training data during ML training. In the context of repeated processing of all trained data sets, the disclosed cycles correspond to epochs.);
performing a learning operation based at least in part on the received input to determine an optimal placement (Paragraph 95; “The machine-learning cycle is executed at 306. The learning cycle is generally repeated for a given number of cycles, which may be an input parameter or the number of available sets of inputs in the training data. Executing the machine-learning cycle may include, for each cycle, obtaining the environment data, processing a single set of training data input vectors through the dynamic provisioning agent, saving the acquired knowledge, and analyzing a scoring function of the cycle (which may include making changes or adjustments to the machine-learning process based on this analysis). The learning cycle may include the training process 320 as shown in FIG. 3B. The learning cycle is completed after running for a given number of cycles.”);
a weight factor (Paragraph 71; “optimizing the provisioning function includes determining the highest total score. In the example, the total scoring function may be S=a.sub.1v.sub.1+a.sub.2v.sub.2+a.sub.3v.sub.3, where a.sub.1, a.sub.2, and a.sub.3 are weighting coefficients for each of the KPIs. Through these weighting coefficients, the different KPIs may be given varying levels of importance (including adjusting values to be more directly comparable, including so that the KPIs can be equally weighted accounting for different calculation types or result ranges for a particular KPI).”).
Zad in view of Sen, further in view of Sundararajan, further in view of Yu, further in view of Chesla, further in view of Ritter does not teach initial placement of the workload elements, start state.
However, Cannata teaches:
initial placement of the workload elements (Paragraph 120; “In response to the template and policies, management CPU 810 may initially allocate host CPU 864, GPU 863, storage unit 865 to target compute unit 890 coupled over a logical domain. The target compute unit may then initialize and being operating. In response to the aforementioned policy being triggered, management CPU 810 may migrate compute unit 890 to compute unit 891 that includes host CPU 867, GPU 869, and storage unit 868 coupled over a logical domain in the sub-fabric 807.”. A person of ordinary skill in the art would have recognized that using the existing initial allocation as an input to the Q-learning based reinforcement learning method would enable the method to evaluate and optimize workload placement based on the current resource assignment, yielding the predictable result of enabling placement decisions without requiring the reinforcement learning process to reconstruct the initial system configuration.);
a start state (Paragraph 64; “The state of various components or elements of system 100 can be monitored through GUI 114, such as processor/CPU state, network state, storage unit state, PCIe element state, among others.”).
Zad, Sen, Sundararajan, Yu, Chesla, Ritter, and Cannata are considered to be analogous to the claimed invention because they are in the same field of resource management. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zad in view of Sen, further in view of Sundararajan, further in view of Yu, further in view of Chesla, further in view of Ritter to incorporate the teachings of Cannata and have received as input, an initial placement of the workload elements and a start state. A person of ordinary skill in the art would have recognized that providing the initial workload placement and corresponding start state enables the reinforcement learning method to evaluate available actions from an existing system configuration and determine improved workload placement decisions, yielding the predictable result of improving accuracy and effectiveness of the optimization process by allowing the method to account for the current workload arrangement and resource state. Further, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to use the monitored state of the system components of Cannata as an input to the reinforcement learning method because the monitored component state represents the current operating condition of the system at the beginning of the optimization process. Initializing the reinforcement learning method with the current system state would yield the predictable result of enabling subsequent action evaluation based on the existing system configuration.
Regarding claim 13, Zad in view of Sen, further in view of Sundararajan, further in view of Yu, further in view of Chesla, further in view of Ritter, further in view of Cannata teach the method of claim 12. Ritter teaches:
wherein the learning operation is performed for a plurality of epochs corresponding to the epoch number (Paragraph 184; “training the algorithm includes providing a single set of training data inputs to the machine-learning algorithm, running the algorithm with the generated training data inputs, obtaining the results of the algorithm from processing the inputs, comparing the results to expected results (e.g. the generated expected results for the given training data set), and updating the algorithm based on the differences between the output and the expected results. This process may be repeated for all available generated training data sets as part of training the machine-learning algorithm. Training may continue until the differences between the output from the algorithm and the expected results are below a certain threshold, below a threshold for a given number of training cycles, or for a given number of training cycles or episodes.”, which teaches controlling training based on a number of training cycles. A person of ordinary skill in the art would have understood such repeated training cycles through available data sets to correspond to successive epochs of training because an epoch represents a repetition through training data during ML training. In the context of repeated processing of all trained data sets, the disclosed cycles correspond to epochs);
wherein the learning operation comprises, in each epoch:
obtaining a list including a plurality of possible actions (Paragraph 94; “The training environment may be initialized at 304. Initializing the training environment may include setting any environment parameters, such as the KPI formulas, KPI weights, set of possible states, set of possible actions, setting the number of cycles to run, or putting the machine-learning system (e.g. dynamic provisioning agent) into a training state. Initializing the training environment may also include obtaining any additional input data generally required but not provided by the input vectors (e.g. data center availability, job component/resource availability).”, explicitly disclosing a set of possible actions. It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have received the set of possible actions as an input);
performing an action selection operation to select an action from the plurality of possible actions (Paragraph 87; “In FIG. 2B, each time (t) represents an iteration of the decision process. An initial state of the decision process, in which no item has been sourced yet, is represented as t.sub.0. Each time (t) represents an iteration of the decision process. In this example, at each point in time the dynamic sourcing agent selects and performs a random action. Based on this action, the process (or environment) transitions to the next state. P.sub.a(s, s′) represents the probability for a given action or transition.”);
performing an action application operation to apply the selected action to facilitate transition from a start state to a next state (Paragraph 87; “In FIG. 2B, each time (t) represents an iteration of the decision process. An initial state of the decision process, in which no item has been sourced yet, is represented as t.sub.0. Each time (t) represents an iteration of the decision process. In this example, at each point in time the dynamic sourcing agent selects and performs a random action. Based on this action, the process (or environment) transitions to the next state. P.sub.a(s, s′) represents the probability for a given action or transition.”, explicitly disclosing the process/environment transitioning from an initial state, corresponding to the start state, to a next state.);
performing a reward collection operation to obtain a reward in response to the transition (Paragraph 71; “A total score (which may also be referred to as a cost or reward) may be calculated for a given provisioning determination based on the separate KPI scores. Such a total score may be calculated by a weighted evaluation function of the KPI scores; such a calculation may be a weighted sum of the KPI scores, where a higher value indicates a better provisioning solution.”);
performing a Q-value update operation (Paragraph 101; “updating the machine-learning algorithm may include updating Q-values in a Q-matrix based on the calculated score(s)”);
and performing an optimal placement update operation (Paragraph 115; “The selected action is executed at 450. Executing the selected action may include updating the output vector(s) with the provisioning determination from that action (e.g. job component 1 from source 2).”, where the output vector is to hold the current most optimal placement.).
Regarding claim 15, Zad in view of Sen, further in view of Sundararajan, further in view of Yu, further in view of Chesla, further in view of Ritter, further in view of Cannata teach the method of claim 13. Ritter teaches:
wherein for each epoch (Paragraph 184; “training the algorithm includes providing a single set of training data inputs to the machine-learning algorithm, running the algorithm with the generated training data inputs, obtaining the results of the algorithm from processing the inputs, comparing the results to expected results (e.g. the generated expected results for the given training data set), and updating the algorithm based on the differences between the output and the expected results. This process may be repeated for all available generated training data sets as part of training the machine-learning algorithm. Training may continue until the differences between the output from the algorithm and the expected results are below a certain threshold, below a threshold for a given number of training cycles, or for a given number of training cycles or episodes.”, which teaches controlling training based on a number of training cycles. A person of ordinary skill in the art would have understood such repeated training cycles through available data sets to correspond to successive epochs of training.), the start state corresponds to placement of workload elements that yields a minimum value of the objective function obtained so far in the learning operation (Paragraph 115; “The selected action is executed at 450. Executing the selected action may include updating the output vector(s) with the provisioning determination from that action (e.g. job component 1 from source 2).”, where the output vector is updated in each iteration with the provisioning determination that the operation determines is optimal at that step.).
Regarding claim 16, Zad in view of Sen, further in view of Sundararajan, further in view of Yu, further in view of Chesla, further in view of Ritter, further in view of Cannata teach the method of claim 13. Sundararajan teaches:
wherein performing the action selection operation comprises determining feasibility of an action with respect to the workload element (Paragraph 105; “each candidate solution includes a mapping of every migration service set with a randomly selected host. When assigning each (migration) service set to a host, the system checks the CPU and memory constraints of the host and the anti-affinity constraints (if any) of the service set. If the host has some service sets to which the migration service set has an anti-affinity constraint, then that host is skipped and another random host is chosen. In some embodiments, the hosts are randomly placed on a list. In this case, each host in the list is iterated through until a compatible host (with no resource or anti-affinity constraint issues) is found. The service set is then placed on that host. In case the migration service set cannot be placed on any host, a new randomly generated hosts list may be used. If the migration service sets can be successfully placed, then a feasible solution is possible.”, which explicitly discloses an iteration through the service set, corresponding to the action set, until a feasible solution is able to be placed, corresponding to determining feasibility of an action with respect to the workload element.).
Regarding claim 17, Zad in view of Sen, further in view of Sundararajan, further in view of Yu, further in view of Chesla, further in view of Ritter, further in view of Cannata teach the method of claim 13. Sundararajan teaches:
wherein if the action application operation migrates a workload element from a current resource to another resource (Paragraph 111; “As described above, based on the fitness function, the best candidate solution is the one that minimizes the standard deviation in CPU and memory residual capacity among the hosts (i.e., equally distributes load among the hosts) and minimizes the number of migrations needed to achieve the mapping in the candidate solution.”),
Ritter teaches:
updating the list in response to performing the action application operation (Paragraph 81; “The action is executed at 206. Executing the action may include updating any input vectors to indicate changes based on the action, such as reducing available job components at a particular component source or reducing the job (e.g., the number of components left to be provisioned for the job). Executing the action may additionally or alternatively include updating the output (e.g. provisioning or consignment vector) to indicate what action was taken. For job provisioning, updating the output may include indicating which job component was consigned (sourced) from which source. Executing the action may include determining the next state based on taking the action.”, and Paragraph 84; “The state is updated at 212. Updating the state may include replacing the current state with the new state received at 208 from executing the action at 206.”. Since the next action is taken based on having updated the input and output vectors and reducing available job components, it would have been obvious to have updated the components left to be provisioned, corresponding to the list, such that the list only comprises elements corresponding to the same workload based on the previous action taken.);
Cannata teaches:
resource pools (Paragraph 147; “deployments of compute units can be based on independent disaggregated pools”).
wherein updating the list comprises changing the list to include only actions arranged to migrate any workload elements belong to the same workload as the workload element to the another resource pool (Paragraph 118; “The actions can include alterations to the composition of existing compute units, addition of additional compute units to support a given application or workload, or removal of elements back into a free pool of elements.”).
It would have been obvious to a person of ordinary skill in the art to have combined the state updates of Ritter with the intent of Cannata to support a particular workload, yielding the predictable result of changing the list to include only elements that belong to a particular workload.
Regarding claim 18, Zad in view of Sen, further in view of Sundararajan, further in view of Yu, further in view of Chesla, further in view of Ritter, further in view of Cannata teach the method of claim 13. Sundararajan teaches:
wherein if the action application operation migrates a workload element from a current resource to another resource (Paragraph 111; “As described above, based on the fitness function, the best candidate solution is the one that minimizes the standard deviation in CPU and memory residual capacity among the hosts (i.e., equally distributes load among the hosts) and minimizes the number of migrations needed to achieve the mapping in the candidate solution.”),
Ritter teaches:
updating the list in response to performing the action application operation (Paragraph 81; “The action is executed at 206. Executing the action may include updating any input vectors to indicate changes based on the action, such as reducing available job components at a particular component source or reducing the job (e.g., the number of components left to be provisioned for the job). Executing the action may additionally or alternatively include updating the output (e.g. provisioning or consignment vector) to indicate what action was taken. For job provisioning, updating the output may include indicating which job component was consigned (sourced) from which source. Executing the action may include determining the next state based on taking the action.”, and Paragraph 84; “The state is updated at 212. Updating the state may include replacing the current state with the new state received at 208 from executing the action at 206.”. Since the next action is taken based on having updated the input and output vectors and reducing available job components, it would have been obvious to have updated the components left to be provisioned, corresponding to the list, such that the list only comprises elements corresponding to the same workload based on the previous action taken.);
Cannata teaches:
resource pools (Paragraph 147; “deployments of compute units can be based on independent disaggregated pools”).
wherein updating the list comprises changing the list to include only actions arranged to migrate any workload elements belong to the same workload as the workload element to the another resource pool (Paragraph 118; “The actions can include alterations to the composition of existing compute units, addition of additional compute units to support a given application or workload, or removal of elements back into a free pool of elements.”).
It would have been obvious to a person of ordinary skill in the art to have combined the state updates of Ritter with the intent of Cannata to support a particular workload, yielding the predictable result of changing the list to include only elements that belong to a particular workload.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have, instead of removing all other actions except those belonging to the selected workload element, remove actions belonging to the selected workload element and leaving actions associated with the non-selected workload element. A person of ordinary skill in the art would have recognized that the selected workload element would have already had actions taken thereof, and that updating the list with regard to remaining resource states to indicate possible next actions for states that have yet to be taken would be a known method for ensuring that epochs do not repeat the same steps during training and evaluation.
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Zad in view of Sen, further in view of Sundararajan, further in view of Yu, further in view of Chesla, further in view of Ritter, further in view of Cannata, further in view of Chen et al. (US 20210218639 A1) hereafter Chen.
Regarding claim 14, Zad in view of Sen, further in view of Sundararajan, further in view of Yu, further in view of Chesla, further in view of Ritter, further in view of Cannata teach the method of claim 13. Sundararajan teaches:
a summation associated with number of migrations associated with the workloads (Paragraph 82; “Equation 11 on line 226 indicates an exemplary fitness function that may be used in computing a rebalancing solution. For example, this fitness function may be used by the cloud rebalancing module 156 to determine a fitness score of candidate solutions at block 146. The exemplary fitness function is calculated by adding the standard deviation of the hosts being considered and mig_count, which denotes the number of migrations needed to achieve the mapping of the candidate solution. The candidate solution that minimizes this value is the solution that is chosen.”).
Yu teaches:
a number of active nodes (Paragraph 119; “target processing node can appropriately increase or decrease the number of processing units for running the script based on the number of requests received corresponding to the same script, so as to minimize the power consumption of the target processing node while ensuring data processing efficiency.”).
Chesla teaches:
complementary weights with a factor in [0, 1] (Paragraph 243; “the current expected value is a linear combination of the mean value for the most recent hour and the last expected value, taken with complementary weights: Y.sub.n=.alpha.X.sub.n+(1-.alpha.)Y.sub.n-1, wherein Y.sub.n is the expected value after the nth iteration, X.sub.n is the last mean value of the same parameter, and .alpha. is a weighting constant between 0 and 1.”).
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have utilized complementary weights with a factor in [0, 1]. A person of ordinary skill in the art before the effective filing date of the claimed invention would have recognized that number of active nodes and number of workload migrations are recognized optimization objectives in workload placement and resource management, and would have been motivated to simultaneously optimize both metrics in order to balance both energy efficiency, achieved by reducing the number of active nodes, with migration overhead, achieved by reducing the number of workload migrations. It would have further been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have applied the complementary weighting factors of Chesla to the objective function because complementary weights provide a known mechanism for adjusting the relative importance of competing optimization objectives while maintaining a normalized weighting, yielding the predictable result of enabling the optimization to trade off energy efficiency against migration cost according to system requirements. It would have been obvious to apply different values of the weighting factor when optimizing the objective function because the purpose of the weighting factor is to adjust the relative importance of the competing objective terms. A person of ordinary skill in the art before the effective filing date of the claimed invention would have recognized that varying the weighting factor over its defined normalized range permits evaluation of different tradeoffs between optimization objectives, yielding the predictable result of selecting a weighting that best satisfies the desired system performance criteria.
Zad in view of Sen, further in view of Sundararajan, further in view of Yu, further in view of Chesla, further in view of Ritter, further in view of Cannata does not teach a negative constant multiplied by a reward function.
However, Chen teaches:
a negative constant multiplied by a reward function (Paragraph 29; “In reinforcement learning (RL), a reward is a value in the objective function representing system wide performance. Reinforcement learning uses reward and punishment as signals for positive and negative behavior, wherein a larger value of reward indicates better performance. As an example, using a constant, 1, as the reward factor for reinforcement learning, a call originating within 100 milliseconds results in 100/100×1=1 reward”).
Zad, Sen, Sundararajan, Yu, Chesla, Ritter, Cannata, and Chen are considered to be analogous to the claimed invention because they are in the same field of resource management. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zad in view of Sen, further in view of Sundararajan, further in view of Yu, further in view of Chesla, further in view of Ritter, further in view of Cannata to incorporate the teachings of Chen and have utilized a negative constant times the delta of the objective function to obtain the reward function. Using changes in the number of active nodes and migrations rather than absolute values would have been obvious to a person of ordinary skill in the art because reinforcement learning evaluates the effects of an action on the environment. Measuring the change caused by an action yields the predictable result of providing the feedback necessary to determine whether the action improved or degraded the objective. Further, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to implement the objective function as a negative constant times the delta of the objective function. A person of ordinary skill in the art would have recognized that a negative weighted cost function provides an equivalent reward formulation, where reductions in active nodes and workload migrations yield the predictable result of decreasing the punishment, which is being minimized by the reward formula.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Jung et al. (US 20250045104 A1) discusses the use of a Pareto front (Paragraph 39) for optimizing an objective function and utilizing parameters such as power and performance (Paragraph 73) and using ML for optimization based on the workload type to maximize performance (Paragraphs 73, 82).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KENNETH P TRAN whose telephone number is (571)272-6926. The examiner can normally be reached M-TH 4:30 a.m. - 12:30 p.m. PT, F 4:30 a.m. - 8:30 a.m. PT, or at Kenneth.Tran@uspto.gov.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, April Blair can be reached at (571) 270-1014. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KENNETH P TRAN/Examiner, Art Unit 2196
/APRIL Y BLAIR/Supervisory Patent Examiner, Art Unit 2196
Prosecution Insights